Michael Anissimov has a nice post up about Rouge AI over on Accelerating Future. He ended his post as follows:
"I have considered several solutions to the challenge of rogue AI, but before I mention them, do you yourself have any? If so, please describe them in the comments."
Much, much ink has obviously been spilled over this one.
My personal view is, firstly, that if the military is driving AI research, then things will go badly. If there's someone else - e.g. academia, or a nonprofit like SIAI in the driving seat, I think things look much better.
Secondly, I don't think we know enough about minds-in-general to usefully discuss solutions to the rouge AI / unfriendly AI problem. I mean we can try, but I feel that human natural language is simply not up to the task.
For example, is there a "qualitative difference" between a narrow military AI and a general AI, or is it merely quantitative? The Israeli system you linked to is clearly not complex enough to be dangerous to anyone other than its intended targets, but systems will get more and more complex over time. Perhaps more complex military systems will simply be even more focused on the specifics of war, and will never be able to engage in this sort of reasoning:
"An AI programmed to eliminate violence may come to the conclusion that the best way to reduce violence permanently would be to destroy all humanity, judging the immediate violence to be a necessary compromise for avoiding long-term violence over thousands of years. Given access to its own manufacturing hardware, this scenario is feasible."
On the other hand, perhaps further complexity in military AI will lead to AIs with this kind of ability.
I don’t know, and I don’t think that any present scientific theory has the power to answer this kind of question. We certainly won’t be able to get to the answer by talking about the issue. I’m going to study artificial intelligence next year – in fact I should probably go home and fill in my application form right now – and hope to one day be able to formalize these questions and give scientifically rigorous answers to them. I would encourage others from a math/computing background to go and do the same.
Posts
9 comments:
At the moment talk about rogue AI is cheap, because nobody - if they're being honest - really knows what a rogue AI will look like. he state of the art today remains fairly primitive compared to the lofty ambitions.
From ancient times onwards the military has always had the technology first, before it spreads out into the wider society, so it is reasonable to assume that AGI could first emerge as part of a military project - as the internet and the digital computer both did.
For the present I remain undecided about the true motivations and future path of the SIAI organisation. We'll just have to wait and see.
In the foreseeable future I think it's human intelligence amplified by technology which is going to be the biggest villain. For instance, a large number of relatively dumb military telerobots could be orchestrated under supervisory control by a single individual. This could mean that in future individuals or small groups of people could build (relatively cheaply) what amounts to a private army and engage in warlord type activities.
"For the present I remain undecided about the true motivations and future path of the SIAI organisation. We'll just have to wait and see."
Really?
can you expand on this, Bob?
by the way, I like the little facebook thingy you've got on your blog. Where can i get that?
I’m going to study artificial intelligence next year
Sounds like an excellent plan!
and hope to one day be able to formalize these questions and give scientifically rigorous answers to them.
If you're up for it, I would recommend that you start working on these part-time as well, concurrent with your formal A.I. education. I'm sure you know by now that there's nothing magical about educational systems; much of what you'll need to know you'll need to pick up outside of University anyway, so you might as well get an early start.
@ Rolf: "If you're up for it, I would recommend that you start working on these part-time as well, concurrent with your formal A.I. education. I'm sure you know by now that there's nothing magical about educational systems; much of what you'll need to know you'll need to pick up outside of University anyway"
Yes, this is very true. I was actually just talking to a girl on my maths course, and how she thought that the approach of our lecturer was really boring and overly formal. I, on the other hand, had been supplementing my formal education with stuff off the web, like the excellent site of john baez, so I had other perspectives on the material.
I have actually been reading Reasons and Persons, which is an excellent scholarly work; it outlines the "state of the art" of consequentialist theories of ethics. Sound advice, Rolf, I'd recommend the same to anyone else.
Hello --
Rogue Machine Life an AI are in fact already a huge problem on the Internet. More than a few bright yet misguided "hackers" have long since discovered the exponential advantages of building viruses and worms using software that evolves, remembers, mutates, learns, adapts - and is able to flee from danger. Several of these extremely nasty little bugs have wreaked havoc on the Web already, causing billions in damage this past two years.
Understanding how to deal with maelevolent Machine Life and AI is a battle that has already been joined by those such as Google or eBay. And it will get worse, because now these little buggers evolve and mutate so putting their creator in jail is no longer a useful tactic - the bugs will go on, wave after wave - long after the virus writer is gone.
And I remind everyone that if an advanced AI were to either sieze, damage - or destroy the world wide web the consequences for humanity would be disasterous.
SINCERELY -- Christopher Doyon
---------------------
MLAI Foundation
www.MLAIFoundation.info
@christopher:
I'm totally with you on the point that something needs to be done about viruses and worms, and I think that you bring up a very good point - we ought to see viruses as the precursor to what might happen with AI gone wrong.
I wouldn't say that contemporary viruses constitute "general artificial intelligences", though. Their behavior is far too brittle and narrow.
Also, you said:
"these little buggers evolve and mutate"
As far as I am aware, this is simply false. I have actually worked closely with someone who was trying to create evolving computer viruses, and it is very, very difficult to do, for a number of reasons which I won't go into here.
[note: the aim of the project was to see whether evolving viruses were possible in order to develop countermeasures, there was no malicious intent]
I think my point was simply this; we don't need to fear roving gangs of armor plated cannon firing robots. No physical AI is ever going to do much damage. The real danger is a rogue AI either taking control of or simply destroying the Internet.
Not only are there viruses that evolve, but at least one group is working on a virus with fight or flight. When the virus detects possible attempts to dislodge or destroy it, it clones itself to a new machine AND deletes it's local copy. Nifty trick, and they are close to perfecting it.
YOURS -- Chris
---------------------
MLAI Foundation
www.MLAIFoundation.info
@chris: "Not only are there viruses that evolve... "
can you link me to something that gives some kind of verification of this claim? I was totally under the impression that it had never been done...
The military will probably like a good degree of control. I'd say at least as likely a source for rogue AI is a virus-hacker mentality releasing or hacking some self-replicating machinery. Once such has been developed, and once it can support decent levels of AI and flexibility. Whether it'll be a real threat depends on where it can get to and how fast it replicates.
But whatever technology the rogue uses, human society can use as well, which is a point I think often lost, buried beneath a false instinct that AI will have to have some intrinsic self-motivation beyond obeying human wishes.
Post a Comment