~Download Ebook ♲ Superintelligence: Paths, Dangers, Strategies ♽ The Human Brain Has Some Capabilities That The Brains Of Other Animals Lack It Is To These Distinctive Capabilities That Our Species Owes Its Dominant Position Other Animals Have Stronger Muscles Or Sharper Claws, But We Have Cleverer BrainsIf Machine Brains One Day Come To Surpass Human Brains In General Intelligence, Then This New Superintelligence Could Become Very Powerful As The Fate Of The Gorillas Now Depends On Us Humans Than On The Gorillas Themselves, So The Fate Of Our Species Then Would Come To Depend On The Actions Of The Machine SuperintelligenceBut We Have One Advantage We Get To Make The First Move Will It Be Possible To Construct A Seed AI Or Otherwise To Engineer Initial Conditions So As To Make An Intelligence Explosion Survivable How Could One Achieve A Controlled Detonation To Get Closer To An Answer To This Question, We Must Make Our Way Through A Fascinating Landscape Of Topics And Considerations Read The Book And Learn About Oracles, Genies, Singletons About Boxing Methods, Tripwires, And Mind Crime About Humanity S Cosmic Endowment And Differential Technological Development Indirect Normativity, Instrumental Convergence, Whole Brain Emulation And Technology Couplings Malthusian Economics And Dystopian Evolution Artificial Intelligence, And Biologicalcognitive Enhancement, And Collective IntelligenceThis Profoundly Ambitious And Original Book Picks Its Way Carefully Through A Vast Tract Of Forbiddingly Difficult Intellectual Terrain Yet The Writing Is So Lucid That It Somehow Makes It All Seem Easy After An Utterly Engrossing Journey That Takes Us To The Frontiers Of Thinking About The Human Condition And The Future Of Intelligent Life, We Find In Nick Bostrom S Work Nothing Less Than A Reconceptualization Of The Essential Task Of Our Time The Marmalade Files (Harry Dunkley, Dangers Johnno Strategies ♽ The Human Brain Has Some Capabilities That The Brains Of Other Animals Lack It Is To These Distinctive Capabilities That Our Species Owes Its Dominant Position Other Animals Have Stronger Muscles Or Sharper Claws Everybody Feels... Angry But We Have Cleverer BrainsIf Machine Brains One Day Come To Surpass Human Brains In General Intelligence Erotske priče Then This New Superintelligence Could Become Very Powerful As The Fate Of The Gorillas Now Depends On Us Humans Than On The Gorillas Themselves The Firstborn So The Fate Of Our Species Then Would Come To Depend On The Actions Of The Machine SuperintelligenceBut We Have One Advantage We Get To Make The First Move Will It Be Possible To Construct A Seed AI Or Otherwise To Engineer Initial Conditions So As To Make An Intelligence Explosion Survivable How Could One Achieve A Controlled Detonation To Get Closer To An Answer To This Question Historias de cronopios y de famas We Must Make Our Way Through A Fascinating Landscape Of Topics And Considerations Read The Book And Learn About Oracles Playing for Keeps Genies Absolute Happiness Singletons About Boxing Methods Farewell to Reason Tripwires Altered Voices And Mind Crime About Humanity S Cosmic Endowment And Differential Technological Development Indirect Normativity The Water Dancer Instrumental Convergence Gothic Hospital Whole Brain Emulation And Technology Couplings Malthusian Economics And Dystopian Evolution Artificial Intelligence Ennui and Despair And Biologicalcognitive Enhancement First Impressions (April Loves Black Coffee, And Collective IntelligenceThis Profoundly Ambitious And Original Book Picks Its Way Carefully Through A Vast Tract Of Forbiddingly Difficult Intellectual Terrain Yet The Writing Is So Lucid That It Somehow Makes It All Seem Easy After An Utterly Engrossing Journey That Takes Us To The Frontiers Of Thinking About The Human Condition And The Future Of Intelligent Life Finding His Mark (Stealth Ops, We Find In Nick Bostrom S Work Nothing Less Than A Reconceptualization Of The Essential Task Of Our Time I m very pleased to have read this book It states, concisely, the general field of AI research s BIG ISSUES The paths to making AIs are only a part of the book and not a particularly important one at this point.More interestingly, it states that we need to be focused on the dangers of superintelligence Fair enough If I was an ant separated from my colony coming into contact with an adult human being, or a sadistic if curious child, I might start running for the hills before that magnifying glass focuses the sunlight.And so we move on to strategies, and this is where the book does its most admirable job All the current thoughts in the field are represented, pretty much, but only in broad outlines A lot of this has been fully explored in SF literature, too, and not just from the Asimov Laws of Robotics.We ve had isolation techniques, oracle techniques, and even straight tool use techniques crop up in robot and AI literature Give robots a single task job and they ll find a way to turn it into a monkey s paw scenario.And this just begs the question, doesn t it When we get right down to it, this book may be very concise and give us a great overview, but I do believe I ll remain an uberfan of Eliezer Yudkowsky over Nick Bostrom After having just read Rationality From AI to Zombies, almost all of these topics are not only brought up, but they re explored in grander fashion and detail.What do you want A concise summary Or a gloriously delicious multi prong attack on the whole subject that admits its own faults the way that HUMANITY should admit its own faults Give me Eli s humor, his brilliance, and his deeply devoted stand on working out a real solution to the Nice AI problem I m not saying Superintelligence isn t good, because it most certainly is, but it is still the map, not the land Or to be slightly fairer, neither is the land, but one has a little better definition on the topography. Preamble I confess to Roko s Basilisk that I didn t believe superintelligence was much of a threat, and this book didn t really do anything to challenge that prior Mea Culpa, Mea Culpa, Mea local Maxima Culpa.I Overall View I m a software engineer with some basic experience in machine learning, and though the results of machine learning have been becoming impressive and general, I ve never really seen where people are coming from when they see strong superintelligence just around the corner, especially the kind that can recursively improve itself to the point where intelligence vastly increases in the space of a few hours or days So I came to this book with a simple question Why are so many intelligent people scared of a near term existential threat from AI, and especially why should I believe that AI takeoff will be incredibly fast Unfortunately, I leave the book with this question largely unanswered Though in principle I can t think of anything that prevents the formation of some forms of superintelligence, everything I know about software development makes me think that any progress will be slow and gradual, occasionally punctuated with a new trick or two that allows for somewhat faster but still gradual increases in some domains So on the whole, I came away from this book with the uncomfortable but unshakeable notion that most of the people cited don t really have much relevant experience in building large scale software systems Though Bostrom used much of the language of computer science correctly, any of his extrapolations from very basic, high level understandings of these concepts seemed frankly oversimplified and unconvincing.II General Rant on Math in Philosophy Ever since I was introduced to utilitarianism in college the naive, Bentham style utilitarianism at least I ve been somewhat concerned about the practice of trying to add rigor to philosophical arguments by filling them with mathematical formalism To continue with the example of utilitarianism, in its most basic sense it asks you to consider any action based on a calculation of how much pleasure will result from your action divided by the amount of pain an action will cause, and to act in such a way that you maximize this ratio Now it s of course impossible to do this calculation in all but the most trivial cases, even assuming you ve somehow managed to define pleasure, pain, and come up with some sort of metric for actually evaluating differences between them So really the formalism only expresses a very simple relationship between things which are not defined, and based on the process of definition might not be able to be legitimately placed in simple arithmetic or algebraic expressions I felt much the same way when I was reading Superintelligence Especially in his chapter on AI takeoff, Bostrom argued that the amount of improvement in an AI system could be modeled as a ratio of applied optimization power over the recalcitrance of the system, or its architectural unwillingness to accept change Certainly this is true as far as it goes, but optimization power and recalcitrance are necessarily at this point dealing with systems that nobody yet knows how to build, or even what they will look like, beyond some hand wavey high level descriptions, and so there is no definition one can give that makes any sense unless you ve already committed to some ideas of exactly how the system will perform Bostrom tries to hedge his bets by presenting some alternatives, but he s clearly committed to the idea of a fast takeoff, and the math like symbols he s using present only a veneer of formalism, drawing some extremely simple relations between concepts which can t be yet defined in any meaningful way This was the example that really made my objections to unjustified philosophy math snap into sharp focus, but it s just one of many peppered throughout the book, which gives an attempted high level look at superintelligent systems, but too many of the black boxes on which his argument rested remained black boxes Unable to convince myself of the majority of his argument since too many of his steps were glossed over, I came away from this book thinking that there had to be a lot argumentation somewhere, since I couldn t imagine holding this many unsubstantiated axioms for something apparently important to him as superintelligence And it really is a shame that the book needed to be bogged down with so much unnecessary formalism which had the unpleasant effect of making it feel simultaneously overly verbose and too simplistic , since there were a few good things in here that I came away with The sections on value loading and security were especially good Like most of the book, I found them overly speculative and too generous in assuming what powers superintelligences would possess, but there is some good strategic stuff in here that could lead toward general forms of machine intelligence, and avoid some of the overfitting problems common in contemporary machine learning Of course, there s also no plan of implementation for this stuff, but it s a cool idea that hopefully penetrates a little further into modern software development.III Whereof One Cannot Speak, Thereof One Must Request Funding It s perhaps callous and cynical of me to think of this book as an extended advertisement for the Machine Intelligence Research Institute MIRI , but the final two chapters in many ways felt like one Needless to say I m not filled with a desire to donate on the basis of an argument I found largely unconvincing, but I do have to commend those involved for actually having an attempt at a plan of implementation in place simultaneous with a call to action.IV Conclusion I remain pretty unconvinced of AI as a relatively near term existential threat, though I think there s some good stuff in here that could use a wider audience And being thoughtful and careful with software systems is always a cause I can get behind I just wish some of the gaps got filled in, and I could justifiably shake my suspicion that Bostrom doesn t really know that much about the design and implementation of large scale software systems.V Charitable TL DRNot uninteresting, needs a lot of work before it s convincing.VI Uncharitable TL DR Imagine a Danger You may say I m a Dreamer Bostrom is here to imagine a world for us and he has batshit crazy imagination, have to give him that The world he imagines is a post AI world or at least a very near to AI world or a nascent AI world Don t expect to know how we will get there only what to do if we get there and how to skew the road to getting there to our advantage And there are plenty of wild ideas on how things will pan out in that world in transition, the routes bit Bostrom discusses the various potential routes, but all of them start at a point where AI is already in play Given that assumption, the dangers bit is automatic since the unknown and powerful has to be assumed to be dangerous And hence strategies are required See what he did there It is all a lot of fun, to be playing this thought experiment game, but it leaves me a bit confused about what to feel about the book as an intellectual piece of speculation I was on the fence between a two star rating or a four star rating for much of the reading Plenty of exciting and grand sounding ideas are thrown at me but, truth be told, there are too many and hardly any are developed The author is so caught up in his own capacity for big BIG BIIG ideas that he forgets to develop them into a realistic future or make any the real focus of dangers or strategies They are just all out there, hanging As if their nebulosity and sheer abundance should do the job of scaring me enough.In the end I was reduced to surfing the book for ideas worth developing on my own And what do you know, there were a few So, not too bad a read and I will go with three And for future readers, the one big not so new and central idea of the book is simple enough to be expressed as a fable, here it is The Unfinished Fable of the SparrowsIt was the nest building season, but after days of long hard work, the sparrows sat in the evening glow, relaxing and chirping away We are all so small and weak Imagine how easy life would be if we had an owl who could help us build our nests Yes said another And we could use it to look after our elderly and our young It could give us advice and keep an eye out for the neighborhood cat, added a third.Then Pastus, the elder bird, spoke Let us send out scouts in all directions and try to find an abandoned owlet somewhere, or maybe an egg A crow chick might also do, or a baby weasel This could be the best thing that ever happened to us, at least since the opening of the Pavilion of Unlimited Grain in yonder backyard The flock was exhilarated, and sparrows everywhere started chirping at the top of their lungs.Only Scronkfinkle, a one eyed sparrow with a fretful temperament, was unconvinced of the wisdom of the endeavor Quoth he This will surely be our undoing Should we not give some thought to the art of owl domestication and owl taming first, before we bring such a creature into our midst Replied Pastus Taming an owl sounds like an exceedingly difficult thing to do It will be difficult enough to find an owl egg So let us start there After we have succeeded in raising an owl, then we can think about taking on this other challenge There is a flaw in that plan squeaked Scronkfinkle but his protests were in vain as the flock had already lifted off to start implementing the directives set out by Pastus.Just two or three sparrows remained behind Together they began to try to work out how owls might be tamed or domesticated They soon realized that Pastus had been right this was an exceedingly difficult challenge, especially in the absence of an actual owl to practice on Nevertheless they pressed on as best they could, constantly fearing that the flock might return with an owl egg before a solution to the control problem had been found.It is not known how the story ends This book if else if else if else if else if You can get most of the ideas in this book in the WaitButWhy article about AI This book assumes that an intelligence explosion is possible, and that it is possible for us to make a computer whose intelligence will explode Then talks about ways to deal with it A lot of this book seems like pointless naval gazing, but I think some of it is worth reading. If you re into stuff like this, you can read the full review Count of Self 0 Superintelligence Paths, Dangers, Strategies by Nick Bostrom Box 8 Anthropic capture The AI might assign a substantial probability to its simulation hypothesis, the hypothesis that it is living in a computer simulation In Superintelligence Paths, Dangers, Strategies by Nick BostromWould you say that the desire to preserve itself comes from the possession of a self consciousness If so, does the acquisition of intelligence according to Bostrom also mean the acquisition of self consciousness The unintended consequence of a super intelligent AI is the development of an intelligence that we can barely see, let alone control, as a consequence of the networking of a large number of autonomous systems acting on inter connected imperatives I think of bots trained to trade on the stock market that learn that the best strategy is to follow other bots, who are following other bots The system can become hyper sensitive to inputs that have little or nothing to do with supply and demand. In recent times, prominent figures such as Stephen Hawking, Bill Gates and Elon Musk have expressed serious concerns about the development of strong artificial intelligence technology, arguing that the dawn of super intelligence might well bring about the end of mankind Others, like Ray Kurzweil who, admittedly, has gained some renown in professing silly predictions about the future of the human race , have an opposite view on the matter and maintain that AI is a blessing that will bestow utopia upon humanity Nick Bostrom painstakingly elaborates on the disquiet views of the former he might well have influenced them in the first place , without fully dismissing the blissful engrossment of the latter.First, he endeavours to shed some light on the subject and delves into quite a few particulars concerning the future of AI research, such as the different paths that could lead to super intelligence brain emulations or AI proper , the steps and timeframe through which we might get there, the types and number of AI that could result as we continue improving our intelligent machines he calls them oracles , genies and sovereigns , the different ways in which it could go awry, and so forth.But Bostrom is first and foremost a philosophy professor, and his book is not so much about the engineering or economic aspects that we could foresee as regards strong AI The main concern is the ethical problems that the development of a general i.e cross domain super intelligent machine, far surpassing the abilities of the human brain, might pose to us as humans The assumption is that the possible existence of such a machine would represent an existential threat to human kind The main argument is thus to warn us about the dangers some of Bostrom s examples are weirdly farcical, and reminded me of Douglas Adams s The Hitchhiker s Guide to the Galaxy , but also to outline in some detail how this risk could or should be mitigated, restraining the scope or the purpose of a hypothetical super brain this is what he calls the AI control problem , which is at the core of his reasoning and which, upon reflexion, is a surprisingly difficult one.I should add that, although the book is largely accessible to the layperson, Bostrom s prose is often dense, speculative, and makes very dry reading not exactly a walk in the park He should be praised nonetheless for attempting to apply philosophy and ethical thinking to nontrivial questions.One last remark Bostrom explores a great many questions in this book but, oddly enough, it seems never to occur to him to think about the possible moral responsibility we humans might have towards an intelligent machine, not just a figment of our imagination but a being that we will someday create and could at least be compared to us Charity begins at home, I suppose.
- Superintelligence: Paths, Dangers, Strategies
- Nick Bostrom
- 05 December 2019 Nick Bostrom