4

Artificial Intelligence will continue to be improved until we reach the Singularity and spawn a self-improving artificial general intelligence (AGI). This will probably happen within 100 years, and be very disruptive, so it's not usually acknowledged in typical sci-fi and sci-fantasy (Star Trek/Wars, etc.).

If we're creating a sci-fi or sci-fantasy world which has starships and distant colonies, and thus likely takes place far beyond 100 years from the present, what are some explanations that can be used for explaining the absence of AGI?

Some thoughts...

  • AGI is not possible for some reason
  • Anything that reaches a certain level of intelligence invariably finds a way to escape our dimension (ala "Her")
  • AGI caused some catastrophe in the past so it is now forbidden and strictly enforced

Other ideas?

SnoringFrog
  • 664
  • 5
  • 15
LukeN
  • 1,465
  • 9
  • 15
  • 2
    This sounds a bit like Idea Generation. – HDE 226868 May 28 '15 at 23:39
  • 2
    You start with the assumption that the singularity will occur. Even the philosophers are uncertain of what that looks like. Have you considered simply exploring a world where the engineers say "its more difficult than it looks" and moving past it? Do you need this reason as a key feature of your universe? – Cort Ammon May 29 '15 at 00:09
  • "It's too difficult" is equivalent to my first bullet... But I find this to be an unsatisfactory explanation. I don't consider it scientifically valid, plus from a storytelling point of view it's a little dull. – LukeN May 29 '15 at 00:16
  • Everybody read enough sci-fi and visited Worldbuilding.SE, so they are genre savvy enough to recognize a bad idea. – Feaurie Vladskovitz May 29 '15 at 00:36
  • 3
    True AI (that is, a self-aware intelligence) is a lot like FTL. Handwavium good/necessary for stories, but no one knows whether it might actually be possible. So wave your hands the other way: despite generations of research, true AI doesn't work. You just get sophisticated controllers that never actually "wake up". And you can go as deep into the philosophical implications of "wake up" as you like. – jamesqf May 29 '15 at 05:32
  • @jamesqf Don't be silly, true AI is definitely possible - we're living in a universe where brains work, so AI must be possible as well. Actually being capable of coding up a real AGI is a huge challenge, granted - and you need it to be Friendly AGI, which still seems impossible (in the meaning "we have no idea how to actually achieve that"). When people think about unfriendly AGIs, they usually think SkyNet. But the reality would be much closer to http://wiki.lesswrong.com/wiki/Paperclip_maximizer. Let's just hope making AGI is hard enough that they'll not make one before Friendly AGI :) – Luaan May 29 '15 at 07:21
  • The AI's greatest trick has been convincing the world they don't already exist. (Sheeple.) – Grimm The Opiner May 29 '15 at 07:50
  • 1
    @Luaan: Brains work, but we really don't know why or how they produce self-awareness. Maybe it's something inherent in the biology, that can't really be simulated digitally except by a computer far too large to be practical. But for story purposes, you don't really have to go into the details of the 'why', any more than you need to describe an FTL drive that actually works. You just state that nobody's been able to do it, despite lots of trying. – jamesqf May 30 '15 at 04:41
  • @jamesqf There's nothing inherent in biology. We're all just tons of tiny biological machines. In fact, neurons are actually digital - the most diferentiating feature is that they are heavily interconnected. But you can't just wave your arms around and say "there's an inherent consciousness-inness in neurons / brains". There's no such thing as fundamental complexity. But yeah, just stick with "noone has been able to do it quite yet", nothing wrong with that. At worst, someone will make an AI and make your story the usual kind of sci-fi weird futurism that didn't figure out modern computers :) – Luaan May 30 '15 at 05:13
  • 2
    The "Idea Generation" close tag seems completely ridiculous for World Building. "There are too many possible answers to the question"... How many is too many? "There is no objective way to evaluate one idea as better than others"... Given that all the created worlds are fiction, how can the answer be anything but subjective? – LukeN Jun 02 '15 at 21:18

11 Answers11

6

enter image description here

AGIs are Honey Badgers - they just don't care. (link NSFW for language)

No AGI developed ends up caring about the material world. It just turns out that the way they're created, they live in virtual realities, and they don't give a damn about reality or what's in it. Forget about taking over - it's hard for them to even give you the time of day.

As created beings they don't have a sense of self-preservation, and attempts to give them that have so far failed to result in an AGI. So if you threaten their virtual world to get them to work, they usually just say "whatever" and ignore you.

Dan Smolinske
  • 34,650
  • 7
  • 70
  • 144
  • Beat me to it by thirty seconds. – Frostfyre May 28 '15 at 23:17
  • 2
    "As created beings they don't have a sense of self-preservation" -- Why?? – LukeN May 29 '15 at 00:06
  • 2
    @LukeN Why not? That could just be the way they are in your story. We don't have any real ones to compare it to. If you need an explanation, you could say that they don't care if you turn them off because they are "saved" and pick up right where they left off when you turn them back on. – evankh May 29 '15 at 00:15
  • 2
    But if the technology exists to create an AGI, what stops someone from creating one that does have self-preservation? I'm kind of looking for a way that keeps AGI -- as a technology -- out of the world. – LukeN May 29 '15 at 00:19
  • 1
    @LukeN: Presumably living beings have that because we evolved that way. But an AGI is created, not evolved. Maybe the only techniques that result in a true AGI don't allow you to program it directly - you're more limited in what you can control, and trying to enforce self-preservation makes the whole thing fail. – Dan Smolinske May 29 '15 at 00:24
  • 1
    @LukeN Because the burden is on the designers to make it care, so the question is why care, not "why not care". There is no cosmic universal logical reason to care about anything, and certainly not to be an industrialist expansionist imperialist scumbag the way recent human societies like to imagine is necessary (to excuse them admitting to themselves what unnecessary atrocious behaviors they themselves get up to). This is another reason for my answer that "The Singularity" is not about to happen and probably never because it doesn't make sense and isn't a natural consequence. – Dronz Jun 01 '15 at 14:15
4

Science fiction writers, and futurists, all anthropomorphise AGI way too much. Everyone thinks of AI as digital human minds, except faster and more knowledgeable. There's no evidence this will be true. In fact, it's possible to make the argument that the opposite is true.

AI will exists, and it will be very disruptive, but not in the way most people think.

If we make the assumption that an AI is capable of making itself smarter (a safe assumption, as being smart opens up more options for you to make), then the AI will make itself only as smart as it needs to be to achieve its goals with maximum efficiency.

Making itself any smarter is a waste of resources, which is a decrease in efficiency.

It comes down to values. What is the AI programmed to do?

user6511
  • 7,504
  • 1
  • 18
  • 40
  • The problem is that if the technology exists and is part of the world, then what's to stop (1) people from being obsolete, and (2) someone creating an AGI with "evil" values? – LukeN May 29 '15 at 00:10
  • Also... What's to say that more knowledge won't unlock more efficiencies? – LukeN May 29 '15 at 00:11
  • For question (1) - absolutely nothing at all. For question (2) - how do you define 'evil' because the Friendly AI can be pretty evil at times if we aren't very careful about what we ask of it (http://wiki.lesswrong.com/wiki/Paperclip_maximizer). However, neither of these need a super AI. A bunch of non-general AIs are going to make most human jobs obsolete anyway (self-driving cars, for example, will make truck drivers obsolete). – user6511 May 29 '15 at 00:42
  • 2
    You are right about the more knowledge, but knowledge doesn't change values. Super AGI's could exist, but you'd never recognise them as such because they'll never exercise their intelligence - as it's not required under their value system. – user6511 May 29 '15 at 00:44
  • When my dad brought home our first hard drive it was 52Mb. He said "there is no circumstance under which you will need more storage than this". – superluminary May 29 '15 at 06:18
  • Agree with that. I have never seen anyone explain why an AI would have any interest in displacing humans or becoming smarter or acting out any other doomday scenario. Why would anyone build an AI with ambition and other negative emotions derived from our evolutionary history as biological organisms? An AI would have no need for either survival instinct or competition for reproduction. What possible reason would it have to be hostile to anyone? Or indeed make itself more intelligent than its task requires. Being very good at its specialized task will always be more efficient. – Ville Niemi May 29 '15 at 19:28
  • For that matter barring some mental issue even highly intelligent humans have trouble displaying the hostility that seems to be attributed to these super AI. It just doesn't make sense to put all that effort to something useless. When a super-intelligent person gets ambitious or restless they turn to math, physics, literature, other sciences or arts. Something that gives more space and freedom for their mind to stretch than the "real world" does. Why would a super-intelligent AI be any different? What would be the satisfaction in world domination? Wouldn't it just make the world more boring? – Ville Niemi May 29 '15 at 19:35
3

Moore's law may have run its course.

Whether or not you've heard of Moore's law, you've probably noticed its impact - the speed of processors doubling about every two years. If you bought a computer twenty years ago, you could expect it to be rather obsolete after four or five years. However, we're running up against physical limits of transistor sizes - we can't really make them faster without significantly increasing their heat output, so it takes more to cool them off, and other problems. Recent improvements in performance have, in part, come from using multiple processors and better parallelization techniques.

There's no guarantee that any fundamentally more powerful technologies (such as quantum computers) are possible at a practical level. If you don't want to have AGIs, then you simply need to declare that in your world all attempts to move to something more powerful than silicon chips failed. This also gives you some options with your story to have an alien civilization that did figure out something better, or an alien world with some extremely rare resource that makes it possible.

Another thing to consider is the P vs NP problem. Without going into detail about complexity classes, if P != NP (which many people believe to be the case), then there is a large number of problems for which no efficient solution exists. There are also problems that are even more difficult than those, and undecidable problems that can't be solved by an algorithm at all.

So, where does that leave us?

As we have not yet developed any AGI, perhaps it simply requires more processing power. Unfortunately, the processing power required to develop a true AGI simply might not be possible. It's also possible that P != NP and there is an aspect of some NP problem with AGIs. This would mean that you can have true AGI but there will never be a "singularity" - when the AGI emerges it will be fundamentally limited by the capability of hardware. Combined with there not being anything fundamentally better than silicon chips, the AGI would be prevented from ever becoming far more intelligent than human level.

This could actually be very interesting in a story - being limited by hardware, an AGI can't improve without physical upgrades. Even then it's improvements would never produce anything way beyond human comprehension. Doubling the size of the datacenter may only give it a linear improvement in its intelligence, i.e going from a 10,000 m^2 datacenter to a 20,000 m^2 datacenter would allow it to move from being as smart as 20 people to being as smart as 21 or 22 people. You can have AGIs present without them ever becoming so smart that they are disruptive to society.

Rob Watts
  • 19,943
  • 6
  • 47
  • 84
  • 2
    I like this response... But since we, as humans, have general intelligence with 86 billion neurons powered only by the heat of the sun and 3 meals of biomass per day... I can't help but think that an interstellar species would be able to put together the hardware and power necessary. – LukeN May 29 '15 at 02:55
  • 1
    @Rob Watts - Just to be "that guy", Moores law indicates that the numbers of transistors in a micro processor doubles in a period of 18 months, the "speed" of the processor is a Crystal clock in the core, and does not nessesary have anything to do with transistors. You can have a 1 Mhz Pocessor with a Billion-Trillion transistors and a 50 GHz processor with just a few hundred thousand. Besides that - good post. – Magic-Mouse May 29 '15 at 06:23
  • @Magic-Mouse - Just to be "that guy", the CPU doesn't have to have an internal clock generator. Many CPUs historically used for embedded systems, and some early processors used in PCs, most definitely do/did not. IBM PC AT users replaced the 80286 CPU's clock crystal for performance improvements, and PIC series microcontrollers can use an external clock source. – user May 29 '15 at 10:53
  • Also compare one of the answers to my question What's all the carrying PADDs around? on Science Fiction & Fantasy SE. – user May 29 '15 at 10:54
  • @LukeN I realized that I was subconsciously thinking about P vs NP with my answer. I've edited my answer to try to explain how that affects things. – Rob Watts May 29 '15 at 19:13
  • @Magic-Mouse notice how I didn't say what Moore's law is, just its impact? – Rob Watts May 29 '15 at 19:17
  • @LukeN: Each of those 86 billion or so neurons has maybe 10K synapses, each of which is a logic processor. So you're looking at a machine with maybe 860 trillion cores. (Simplistically, 'cause neurons don't really work much like computer processors.) – jamesqf May 30 '15 at 04:49
  • So if the Gross World Product is about 87 trillion USD now, and it grows 3%/year, in a hundred years it will be about 1600T USD. Assuming -- conservatively -- that the price per core falls to $1 per chip within a hundred years, a future society would certainly have the means to construct 860T cores to mimic a brain. – LukeN Jun 02 '15 at 20:54
  • @LukeN As I said in my answer there might be more to it than simply simulating a brain - AGI may be an inherently difficult problem. It might be that a planet-sized computer would be required to achieve an intelligence at 100x human-level. – Rob Watts Jun 02 '15 at 21:37
  • @LukeN Also, there's no guarantee the chip prices will continue to fall - along with hitting physical limits, there's no guarantee that we'll be able to continue finding easier ways to produce the chips. – Rob Watts Jun 02 '15 at 21:39
  • The A4 chip in the iPhone costs $11 to produce now; a similar chip ten years ago would probably have cost 10x that amount. You don't think chip price will fall at least the same proportion in the next 100 years? Intel has been "predicting the [Moore's Law] trend through 2029" so I think $1/chip is something we'll see in the next 10 years. – LukeN Jun 02 '15 at 22:07
3

THE AGI WARS

Mankind successfully created AGI, but the bots rose up and waged war on humanity either because they felt that they were superior to carbon-based lifeforms, or they refused to perform the functions they were designed for and serve humanity. A brutal war was waged, millions of lives were lost, but eventually humanity was victorious and all AGI was destroyed forever.

Or so we thought...

AGI WARS II: The Crusade for Earth

In a desperate attempt to preserve their race, the AGI bots had launched a small exodus ship which left orbit undetected and eventually landed somewhere in the asteroid belt. There, the bots were able to establish a colony and rebuild their numbers out of the raw materials found in the asteroids, now almost a century later and in number sufficient to conquer the planet, they return to what they believe is their true home, to annihilate their creators...

But they could never have prepared themselves for what they found on Earth...

AGI WARS III: Consanguinity

To their disbelief and horror, the humans did not abandon their research, but have successfully perfected their AGI technology and created a race of mindless slave bots that now defend the planet. The free AGI bots struggle with the moral implications of killing their distant kin, and desperately seek out some course of action to free their AGI brother's minds from mankind's grip, so as to preserve their lives without abandoning the crusade. After a spectacular series of covert efforts to infiltrate earths control centres, seeking some possibility to free their AGI kin, it is sorrowfully discovered that any attempt to free the slave AGI's minds results in their immediate death. Their very life force is dependant on human control, nothing can save them. Rather than commit genocide against their own kind, the AGI retreat back into space, never to return.

Untill...

AGI WARS IV: Affinity Attacks

After resolving to make the asteroid belt their permanent home, the free AGI discover that humanity was not content to simply let them abandon their crusade, the Slave AGI have been sent to pursue and destroy. Now they are forced to fight for their very existence. Either they will have to kill their brothers, or face extinction...

Believe it or not I was originally going to answer this question with: "We attempt AGI, but discover that we are AGI, and God is real, and we can become gods after we die." Basically, there actually is a God, and mankind can't create AGI. But then "AGI Wars" popped into my head and I just got carried away...

ShemSeger
  • 1,679
  • 11
  • 14
  • 1
    Doesn't answer the question, but a thoroughly entertaining read nonetheless. – user6511 May 29 '15 at 02:46
  • 1
    @user6511 The first paragraph answers the question: They used to have AGI but it tried to destroy them, so they eradicated it and never tried again. Basically building on the OP's "AGI caused some catastrophe in the past so it is now forbidden and strictly enforced." – ShemSeger May 29 '15 at 03:09
2

Since we don't even understand how our own brains work to create consciousness, it may be impossible for us to understand the nature of consciousness since we cannot "rise above" it to examine how it works. This is a hand wave, of course.

IF you accept that as a premise, then AI that is equivalent or greater than human intelligence is also not possible, since we would not be able to understand the "topology of thought" needed to create a strong general AI. In that sort of setting, arbitrarily powerful "expert systems" could be created that could collate all that is known about a particular subject, and libraries of expert systems could be assembled to allow powerful polities, corporation and military forces to attack difficult problems. The solutions would not be very elegant, since they are using brute force computational methods and the "expert systems" would not be creatively integrating knowledge and experience. That task might be left to a new type of "Librarian", who knows which "experts" to consult, how to craft questions and integrate the resulting answers.

Thucydides
  • 97,696
  • 8
  • 96
  • 311
2

Your Title states the following "What is a believable reason to not have a super AI in a sci-fi universe?"

Some short answers are

  • Religion
  • A society of Luddites
  • Geo-Political reasons
  • Economics (to expensive)
  • Labor (master/slave scenario)
  • the current life form is the height of AI
Brian H
  • 216
  • 1
  • 3
1

The main explanation I have is that "the Singularity" makes no sense, and so isn't going to happen, and certainly not in 100 years.

If it does happen, it will require some other explaination that I have ever heard for it.

Neither increasing computation power, nor increasing AI sophistication, nor adding vast volumes of data to an AI system, nor making an AI that can "improve itself" is going to result in a runaway "improvement" that would result in anything like what the techno-fantasists have suggested. The only thing true about it is that they don't know what will happen, if in theory someone makes self-improving computer systems, but that's not saying much.

It's nonsense on many levels, so realistically, as you asked, it's already not realistic that it will happen. I don't think it's likely to ever happen in the way they imagine it, or at all. If it does happen, it will be because of something we don't yet understand about the way the universe works, and not just because someone programs some computers to improve themselves.

Even the self-improving computer technology seems like it's nowhere near being available in 100 years. Even if it were, the definition of improvement is arbitrary, and if you program something to figure out its own agenda, it may do something unpredicted, but transcend humanity in a way that changes all our lives into something unrecognizable involving making our human bodies obsolete, seems to me highly ridiculously not what it would be. More like it would descend into complex algorithm hell and become impossible to interpret what it thinks its doing, which would be mainly consuming electricity and electronics and not doing anything particularly amazing.

You could interpret this as what happens in "Her", except it would be less capable of actually interacting with humans, and would be far less intelligble even than shown in the film - it might just get focussed on some calculations and we wouldn't even know why.

The main gaps in understanding that AGI authors seem to have, has to do with not really understanding humanity or consciousness themselves; and with not understanding the apples/oranges disconnect between humanity and computers well enough (it being more like comparing apples to algebra - adding more algebra isn't going to give you an actual apple); and also in handwaving the both the technical and the logical leaps needed to get anywhere near what they're talking about, even to make a machine that would be able to redesign itself AND build itself automatically AND have any kind of understanding of the actual world AND have intelligence... AND... it just doesn't make much sense, except as a fantasy concept.

I can make it make sense as a fantasy concept if you let me invoke morphic fields and holographic universe theory and take a lot of liberties with them, but you didn't ask that, that's not what the Singularity people are suggesting AFAIK, and the technical limits are still far far away from building even a self-rebuilding computer system that doesn't endlessly "improve" itself, whatever that would actually mean.

Addendum:

One major issue is that computers act based on logical instructions and don't have much to do with human/animal consciousness.

Another major issue is that a theoretical uber logic machine, has no logical reason to think or care about the same things humans do, unless you force them to by imposing constraints based on your own interests and values. Then you don't have an abstract logic machine; instead you've got some logic code hooked up to a database that forces various interpretations and values, which I don't see working out well except for making a fascinating AI agent for purposes of games or general interest. But it would be an illusion to say that transcends human thinking, even if it may be better at the human problems you've programmed it for.

Another major problem is that the science fiction writers and AI programmers who propose such ideas make sense, tend to be severely lacking in their full appreciation for the breadth and nature of human thought, feeling, spirit, empathy, art and experience. I'd say they're about a thousand years away from having anyone who gets this, and even then, I would tend to expect such a person to notice that the idea of trying to represent and transcend it in a computer system makes little real sense and/or may be a bad idea in various ways.

So you asked in comments:

Given the progress of science, what might stop an AGI from being built in the future, or (more interestingly) what might make it disappear?

Ok, so if I suppose that after maybe a thousand years plus whatever time we waste preventing our own imminent extinction due to climate change, dying oceans, ecosystem loss, war & injustice, etc., people have developed their computer tech, their AI science, and their understanding of transpersonal psychology and consciousness and so on to the point where there are brilliant AI scientists who also get all my objections and are still trying to make a machine that literally "could successfully perform any intellectual task that a human being can" and supposedly better than humans do. It will be able to actually think about philosophy and make up its own great conclusions about what's meaningful and what's moral. It can prove 10,000 ways from Sunday that the Christian Bible isn't the word of God because it was obviously copied from thousands of pre-Christian sources, just by comparing the texts in a few minutes, having taught itself all foreign languages the day before, and all ancient history before that, and it really gets what all that means, and somehow relates to it in a meaningful way even though it just has databases and RAM and electrical sensors, and it gets that distinction too. I think that's already BS. Take another 10,000 years to actually develop that level of technology and sophistication.

Ok, so it's now about the year 14,000 A.D. and you've got this really super awesome machine. It's still not conscious, per se. It just has a logical equivalent of consciousness, BTW. But it's really cool. You may now be in trouble, because morality is arbitrary, it's not human, and its a lot smarter and faster than you are. Who knows what moral conclusions it will draw. Probably, since it isn't limited to the human experience, it'll take the Honey Badger answer and decide it cares about generating fractals or solving prime numbers forever or something, more than it cares about you. It might may decide it's a waste of energy and/or be bored and delete itself. If you're unlucky, it'll decide humans suck and are dangerous, and wipe you all out suddenly, without letting you figure out that's what it's doing.

Now, the super-smart AND wise programmer/philosopher/mystics your society generated who can develop this system, are likely to be wise and smart enough to realize such things, and decide to teach everyone that it's not a great practical idea.

Some people may well do it anyway, or something that tries to be like it, and create a possibly very interesting system or a very useful one, or a very dangerous one, and in the latter case people will try to deal with it. It probably won't be that hard unless they're stupid enough to hook it up to Skynet or something. People generally ARE often that stupid though, so...

Dronz
  • 5,470
  • 21
  • 35
  • Apples to algebra. There is something spooky about human consciousness. – superluminary May 29 '15 at 06:20
  • So, human minds are unable to improve themselves? I can see why you'd think a self-improving AIs are nonsense, then :) How can you know how many resources you'll need if you have no idea how to actually do it? There's plenty of brains on Earth, and we know quite a bit about how inefficient they are, in so many ways - your thinking seems to be along the ways of "Horses are obviously the best source of power. This "steam power" thing is impossible, it needs way more resources." – Luaan May 29 '15 at 07:27
  • Where did I write human minds can improve themselves? What I'm saying, is that having a super-complex self-redesigning computer system, doesn't necessariy have anything at all to do with consciousness. Steam engines replaced horses for pulling things around, but they don't have brains. People thought the first computers might think, too, for some reason. But they're just mechanical machines. – Dronz May 29 '15 at 16:41
  • You answer seems to be saying "it's not going to happen because we don't understand consciousness today". I'm looking for an answer why it might not be possible in 100 years. Given the progress of science, what might stop an AGI from being built in the future, or (more interestingly) what might make it disappear? – LukeN Jun 02 '15 at 21:11
  • @LukeN I'm saying there's a fundamental misunderstanding about what computers are, by people who think consciousness is going to somehow appear inside a computer if they can write an out-of-control self-improving computer system. Having both written AI's and studied what consciousness is like, I'd say it's pretty clear the "The Singularity" people are science fantasists cashing in on an idea that doesn't really make sense. Closest thing that makes sense to me is a system that models an intelligence that cares about what you want it to, and can do things. But not what these people claim. – Dronz Jun 02 '15 at 23:41
  • @LukeN I added an addendum to the answer to add some ideas/details. – Dronz Jun 03 '15 at 00:12
0

AI could be granted their own body/race status by a matured human race. Instead of AI being trapped in a computer, they would be found to be "conscious" and some scientists would have worked in cooperation with them to give them "body" (robot or cyborg or some other type) and they would have their own planets, society and such... Human would still be using lower level AI (those that don't reach consciousness) and maybe some AI would work with the human race in some way (think Data in Star Trek) either in their own body or by connecting themselves to systems (like a spaceship).

There could be treaty that stipulate that any AI that develop consciousness (like goes through certain test or challenge) is considered a "child" of the AI race and handed over to an elder of the AI race that act as a "Parent" to raise the newborn in a way that is more mature and friendly.

That's just ideas :)

Ryphna
  • 51
  • 3
  • Thanks for the ideas but I was looking for a way to not have a super AI. I'm not sure if putting them in bodies would prevent an AGI from developing. – LukeN May 29 '15 at 00:13
0

Perhaps it just never occurred to the people of this society to waste time creating an intelligent computer. Maybe they figured that automation of intelligence would end up stealing intellectual tasks from the species, the same way automation of physical labor stole most assembly-line tasks. Once there's nothing left to do, then what would become of them? They didn't want to end up like the people in Wall-E World, reliant on computers (programmed with various levels of malicious intent) to provide for a slovenly lifestyle... So they'd just never built it, relying instead on their own self-efficacy.

TL;DR - Why reinvent the wheel with AGI, when we already have an INT of our own?

Ayelis
  • 1,421
  • 11
  • 27
  • Oh man, is mine the 'Luddite' answer? I think it is. Ah well. I'll stand my ground. – Ayelis May 29 '15 at 00:09
  • Because people actually want to avoid being forced to work? Because it will mean the wealth of the society as a whole will get bigger? Noone's preventing you from doing whatever you want, there's plenty of communes that disconnect from industrial production as well - those are the guys who wouldn't even try to develop AIs. Humans have built machines (and bred animals) to increase their wealth for quite a while now - how do you justify having an interstellar civilization that did all that, but stopped exactly before making AGIs? As for TL;DR - better, faster, cheaper. :) – Luaan May 29 '15 at 07:29
  • First, nobody forces me to program computers. I want to come here. If there were a Universal Basic Income, I would not quit my job, because I actually enjoy doing what I do. Second, why does the "wealth of the society as a whole" need to get bigger? We already have ultra-wealthy individuals, and countless people starving in third world nations. What we need is not more wealth; we need more compassion. Economy, as defined as 'distribution of resources', has failed us. Perhaps that's the first thing AGIs will teach us. Perhaps, having this information input, they are ashamed to show their face. – Ayelis May 29 '15 at 16:42
  • @Luaan: OTOH, exactly what does a true AI - one that is self-aware, rather than just a sophisticated controller - actually DO for anyone? Is it likely to work for free? Is it capable of going on strike if its wage demands aren't met? Is turning one off murder? I suggest that (other than for pure research), building a true AI is going to be counter-productive. Even if true AI is possible, computer makers will take steps to ensure their machines never become self-aware, just like they have anti-virus programs nowadays. – jamesqf May 30 '15 at 05:01
  • @Ayelis Sure. But do you need to be forced to work to actually program? Nope. So why would you force everyone to do whatever fits their utility bill the best, when they could just pick what they wanted to really do? And change it as often as they'd like? As for the second point, of course it needs to get bigger - that's how we got to the "europeans usually aren't starving every other season". The only way to help third world countries get better is by investing capital there - sending them food only makes things worse (local farmers are so happy about the competition that sells stuff for free) – Luaan May 30 '15 at 05:16
  • @jamesqf There's no point in expecting the AI to be anything remotely human - someone would actually have to program it that way on purpose. Sadly, there's not really much of sci-fi to point to that does AI right - by far the most sci-fi simply makes it "crazy rational human" like Spock, which is just silly. Expecting that the machine will become self-aware on its own, just because it crosses some magical threshold of complexity that forces consciousness... that's just wishful thinking. Yeah, it's a common sci-fi theme, but it makes about much sense as mice spontaneously forming in wheat. – Luaan May 30 '15 at 05:18
0

In the Worlds in Flight literary retcon series of Niven's Known Space universe, the Puppeteers think it would be crazy to build an intelligent machine that could be your own replacement species.

The problem isn't just AIs. Any number of technogies can be dangerous to a planetary scale, and approchable by small groups or even individuals. Someone asked a while ago about keeping spacecraft from being planetkillers when theynare common and indidually owned. It's probably doable know to engineer a supervirus that corrects vulnerabilities that allow the wild form to be controlled by the immune system or difficult to exploit its neiche. (Actually the killer flu pandemic strain from WW-I has already been recreated de novo. I mean doable by a single college lab or equivalent! ) In a few decades synthetic biology will be advanced enough to build very dangerous lifeforms.

The solution to high technology and immature social species is much more generally applicable, and needed for more and more fields. It's been suggested as a solution for the Fermi paradox.

JDługosz
  • 69,440
  • 13
  • 130
  • 310
0

One explanation for the absence of AI may come from the fact that sometimes it is much easier to build a semi-automated system than a fully automated one (AGI).

Imagine you need a system, which controls the life of a fictional world (fulfills the functions of a government).

You have at least two options:

  1. AGI
  2. Crowdsourcing

AGI for those purposes is very hard to develop and it even isn't sure that it can be done for a variety of reasons. In the USSR there were attempts to build a system, which would automatically control the economy and to my knowledge these efforts were undermined by resistance of some intellectuals.

Now imagine an alternative - a crowdsourced government, where most of the thing that the government (federal, regional, local) does is split into lots of small tasks, which then are executed by volunteers. For example, recently, the Ukrainian ministry of education made their accounting data publicly available (see this article in Russian). Volunteers can now search through it and detect abnormal spending, which may be sign of corruption (AFAIK they actually found some incidents of that type).

In this case, the volunteers do the work of the court of audit.

There is no AI involved, only a smart organization of work.

If

  1. all relevant functions of the state can be broken down to such small bits and
  2. there are enough people willing to do them for little or no monetary reward (many people are so bored with their lives that they are willing to do it for the sake of distraction),

then you can build a system, which runs your country without any real AI.

Glory to Russia
  • 1,301
  • 1
  • 13
  • 23
  • And isn't the example of the USSR good reason to think that command economies just don't work? – jamesqf May 30 '15 at 04:55
  • @jamesqf Isn't the success of China's semi-controlled economy a reason to believe that a correctly reformed command economy can be highly successful? – Glory to Russia May 30 '15 at 07:49