vetta project

Currently the foremost thinker on the ethics and safety of artificial general intelligence is Eliezer Yudkowsky of the Singularity Institute for Artificial Intelligence.  On a few occasions I have tried to read some of his writings on this topic.  Every time, however, I would give up after about ten pages.  I found the ideas to be very jumbled up: a kind of patch work that didn’t flow together to produce any kind of a whole.  I would read something that made good sense to me, followed by what I considered to be an unjustified leap in reasoning.  I also didn’t like his style of writing.  Rather than a dry academic style his writing was more folksy.  Maybe this style appeals to Hofstadter fans, though I’ve never been a fan of the latter myself.  Moreover, I felt there was an unpleasant underlying tone: an attitude that seemed to say “if you object to this statement it is because you are either not intelligent enough or have not studied enough to understand why I am right”.

A few months ago I started reading the Overcoming Bias blog, on which Yudkowsky was discussing physics (I’m still not sure why he’s so involved with physics now, but he’s slowly getting around to explaining this).  Anyway, I have found his writings here to be much more to my liking.  His ideas seem clearer, more focused and organised and I find the style and tone to be much improved.  If you like some interesting philosophical discussions and you haven’t seen the blog already, you might want to check it out.

Encouraged by this I decided to have another look at Yudkowsky’s writings on the ethics and safety of artificial general intelligence.  This time I went for one of his most recent pieces: Artificial Intelligence as a Positive and Negative Factor in Global Risk, a book chapter he did for Global Catastrophic Risks.  If, like me, you were put off by some of his earlier writings, you might want to have a look at this newer document.  While I naturally don’t agree with everything in the chapter, in my opinion the points are clearly argued and fit together well.  Indeed, I found myself agreeing with most of his points.  In short, if you are new to the safety of powerful AI technologies, I’d suggest that you put this document at the top of your reading list.

In part 3 I’ll get back to my own thoughts on the matter…