My Halloween lecture has been uploaded to youtube. The basic outline is:

* what is intelligence?
* Solomonoff induction
* Hutter’s AIXI
* Monte Carlo AIXI (here’s the missing video of it playing pac-man)
* universal intelligence measure
* what neuroscience can teach us about AGI design
* early 2020′s: the Halloween scenario

You can get the slides here. I talked for 2 hours, so it’s broken up into many parts on youtube: Part 1 Part 2 Part 3 Part 4 Part 5 Part 6 Part 7 Part 8 Part 9 Part 10 Part 11 Part 12

Thanks to David Wood at ExtroBritannian for organising this, and all the people who attended — especially those who travelled from other cities and countries, the intelligent questions during my talk, and all the positive feedback I’ve received since. Thanks also to Anders Sandberg for the picture of me speaking that I stole from his flicker stream.

What I will remember most clearly about this summit was Peter Thiel. Firstly, the pre-summit party at his penthouse apartment. That was a treat: a tiny peak into the world of the ultra-rich. His mix of intelligence, focus and energy was quite something to behold and he left a real impression on me. His talk was also among the most engaging in my opinion. No slides, no fluffy stuff, just a straight delivery of ideas and analysis seemingly off the cuff with no notes. In his talk and comments afterwards, the main thing that stuck in my mind was his concern that the singularity wouldn’t arrive quickly enough. Really?

I don’t understand this. If this is really his main concern, why isn’t he using at least a tiny part of his huge financial resources to try to make it happen sooner? He’s funding SIAI, but they aren’t exactly trying to make the singularity happen sooner. If time frame is his primary concern, and I can’t see any reason why he would say this if it were not the case (it’s easier to think of reasons why he wouldn’t) why the inactivity? Just a few percent of his wealth spread across a handful of small projects over the next few decades would make a night-and-day difference to funding in this area. There is essentially no money available to do AGI research and thus we spend our time working on the related topics in the areas of machine learning and theoretical/computational neuroscience that actually are funded. I guess he’s thought carefully about this, but at least I can’t see how his actions can be consistent with his stated beliefs.

Many of the other talks I’d heard before. In some cases so many times I’m sure I could give them myself. I hadn’t heard about the work of Gregory Benford before where they are breeding up very long lived flies and then, if I understand correctly, looking at how the different bio-chemical pathways in these flies change to produce long life. As humans share many of these pathways, they then make pills that reproduce some of these effects in humans in the hope that we too will live longer. I don’t know enough biology to be able to comment further, but it sounds like an interesting long-shot idea to at least try. I guess it will be some time before we know whether it has any effect.

As per my last summit, the most interesting thing for me was meeting and talking to people one on one, or in small groups. I ended up giving my 20 minute spiel on what I consider to be the most promising approach to AGI at least 10 times. It was quite positively received — I’d expected it to be a harder sell. Many of these people seemed to be revising their expectations after talking to me. One exception, and a person I was especially interested in talking to, was Moshe Looks from Google Research. He strikes me as a pretty sharp and temperate thinker who also has a fair amount of hands on experience working on AI/AGI both academically and commercially. Strangely, we seemed agreed on just about everything regarding which approaches were the most promising, in what degrees and why. I was more bullish on the time line for developments, but not radically so. If I add an extra 50% to my time line, which historically appears to be my degree of miscalibration when forecasting technology developments, then we even come up with the same time estimates. Given that I’ve only met him once before, I found this degree of agreement between us quite striking.

My other impression was the scale of the summit this time: over 800 people which I think was roughly twice the size of the last summit. There also seemed to be a more diverse group attending. Along with the usual mix of geeks, nerds and all too obvious Aspergers cases, there seemed to be more general public interest as far as I could tell. I think the gender ratio might have improved too, indeed there was even a fair number of attractive young women. Year by year, is the singularity idea slowly starting to go mainstream? Perhaps it is and I suspect this summit is one of the driving forces. Well done.

As I do at the start of each year, I’ve spent some time reconsidering when I think roughly human level AGI will exist.  I’ve again decided to leave it at 2025, but now with a reduced standard deviation of 5 years.  Computer power is a limitation as researchers typically have limited hardware budgets, unlike the DOD guys and their monster supercomputers.  From what I’ve read, computer power should continue to grow exponentially for at least the next 5 years, and probably the next 10.  So I don’t see this as being too much of an issue in the coming decade.  On the algorithm side, I think things are progressing really well.  I know a number of very talented people who are working on what I think are the key building blocks required before the construction of a basic AGI can begin.  I’m certain these problems are solvable, but whether it takes 2 years or 10 years is hard to guess.  This is my main source of uncertainty.

UPDATE 11 April 2009: Note that these predictions do not take into account my apparent bias towards predicting that things will happen faster than they actually do (see previous post). The required compensation for technology events appears to be about 50% more time. Thus if you want the “Shane meta predictor”, then take 2033 as the expected date, perhaps with a standard deviation of 7 years. At least with financial markets I trust my meta predictor more than my straight predictions and thus I buy and sell accordingly to it. So I suppose that if I had to put money on a date, I should go with 2033. But don’t ask me why it’s going to take that long: I really don’t know.

One example of this has been trying to predict the medium term direction of the stock market over the last 15 years.  The evidence so far shows that I’m consistently good at predicting what will happen, but that I predict that it will happen much sooner than it actually does; I roughly need to double my time estimates.  I’m now trying to mentally correct for this bias in the trades I make, but it will take some years to see if this is working.

In technological matters, I’ve generally done quite well.  My picks for Sun, java, digital music, linux, MySQL and open source in general were pretty much on the mark.  I thought machine learning use in industry would be bigger than it is today, but I wasn’t too far off.  My biggest mistake was to badly underestimate how much Microsoft’s revenues would grow over the last 10 years — I thought they’d already almost saturated the market and its ability to pay.  Like stock markets, my most consistent error has been to be to predict that things will happen faster than they actually do.  I typically need to add about 50% to the time required.

As I don’t have a lot of my own technological predictions to look at, and some remain in the future, I’ve recently been looking at predictions made by others.  I found a few of my old computer and science magazines from the early 80′s through to the late 90′s which contained predictions, and I also dug up Kurzweil’s “The age of spiritual machines” written in 1999 in which he has a whole chapter about 2009.  There were a lot of hits and misses, but if I stand back and try to see the big picture, a pattern becomes clear: Predictions about basic hardware performance, even one I saw in a magazine from 25 years ago, are amazingly accurate.  But you probably knew that already.  Predictions about what would be technologically possible to do at a given point in time were not as accurate, but were still pretty good.  Where things really started to go wrong was when they tried to predict not what would be possible, but what the majority of people would actually be doing.

Perhaps some examples would best explain this.  State of the art speech recognition systems, such as some of the systems that were being developed at IDSIA when I was there, work impressively well.  However, once you’ve learnt to touch type it is typically easier, quieter, more convenient (especially when editing or coding) and far more private to use a keyboard.  I don’t care how good speech recognition is, I don’t want to sit in a room full of people talking out loud to their computers all day.  I only know one person who routinely uses speech recognition to input text.  The fact that speech recognition is technologically doable, doesn’t translate into it being practically useful for many everyday situations.

There are plenty of predictions that fail in this way: the prediction that everybody now would be making video calls on their cellphones.  It’s certainly technologically possible, I saw a guy with a phone that could do it two years ago, but almost nobody does it.  Or that most long distance air travel would be in supersonic jets.  Again, technologically possible, has been for a long time, but not done in practice.  Or that all mice would be wireless by now.  Technologically possible, has been for years, but as far as I can tell most new mice still have cords.  Or that most people driving long distance on freeways would get their car to automatically drive itself.  I’m sure that’s technologically possible, but I don’t see anybody doing it.  Or that your computer would log you in based on recognising your face or your voice.  Technologically possible today, but not done in practice.  And so on.

In short: Predicting raw performance is surprising accurate.  Predicting what will be possible using the knowledge and technology of some future date can also be done with moderate success.  Predicting what the population will routinely do, however, is much harder.  The latter is largely decided by habit, cost and convenience.  Simply being possible isn’t enough.  Note that predicting the development of the first powerful AGI is of the second type.

I’m not convinced that ethics can’t be derived from some deeper unifying principle.  I’m also not convinced that it can, lest you misunderstand me.  What I do accept is that if this is possible then finding such a principle and convincingly arguing for it is likely to be difficult in the extreme, and probably not something that is likely to happen before the singularity.  Nevertheless, I haven’t yet seen any argument so devastating to this possibility that I’m willing to move it from being extremely difficult to certainly impossible.  Any system of ethics that does derive from some unifying metaethical principle is almost certainly going to be different to our present (western?) ethical notions.  I think some degree of this is acceptable, given that our ethical ideas do change a bit over time.  Furthermore, no matter how human we try to make the ethical system of a powerful AGI, post-singularity we are still going to be faced with ethical challenges that our pre-singularity ethics were never set up to deal with.  Thus, our ethics are going to have to be modified and updated in order to remain somewhat consistent and viable, otherwise we’ll end up with this kind of nonsense.

Anyway, let’s assume that this unifying principle either does not exist, or at least can’t be found.  How can we tell if an AGI is ethical given that we can’t explicitly and completely specify what this means?  This seems like the problem Turing faced when trying to determine whether a machine is intelligent or not.  He figured that he couldn’t explicitly and completely say what intelligence is, unlike the research by Hutter and myself, and thus he tried to dodge the issue in the obvious way by setting up an imitation game that doesn’t require an explicit description of intelligence.

Here we can do something similar: set up a group of people and the AGI and ask them ethical questions from a panel of expert judges.  If the judges cannot tell which the machine is, then it passes. Given that the morality function varies between people, and that we can’t say explicitly and completely what our own function is, this seems to be about the best we could hope for.  Naturally, this doesn’t prove that the AGI, or indeed any of the humans participating, are “good”.  An evil genius could probably pass such a test.  Rather, it is simply designed to test whether the AGI is at least able to compute a version of the human morality function which is sufficiently similar to ours that it is able to pass as being human.  Whether the AGI (or human) actually takes its human-passable morality function and reliably and consistently seeks to follow it into the future is a whole other set of problems.  Thus, passing such a test is perhaps a necessary, but certainly not a sufficient condition for having an ethical AGI.

I’m sure somebody must have proposed this idea before, but at least my half hearted attempt to find the idea on Google didn’t turn up anything.  I should also point out that in order for this test to work you’d probably want the AGI to pass a more general Turing test first so that it doesn’t get singled out by the judges for various other reasons.  Only then should you bring in a group of expert ethicists to try to judge which of the test subjects was ethically inhuman.  We would also want to include in the test subjects a few very nice people and a couple of professional ethicists as we wouldn’t want the AGI to be able to “fail” for being too nice or consistently ethical.

Yeah, well, that was 3 weeks ago.

Today, Radeon’s new HD 4870 X2 graphics card has 1600 stream processors that crunch 2.4 Tera FLOPS.

The award is in “recognition of [my] efforts to improve AI theory” and is worth CAD $10,000.  This will certainly help my budget over the next two years while I study at the Gatsby Unit in London.  So, thank you to SIAI Canada, and to all the Canadians whose donations made this money available!

Speaking of my research, after a long weekend of final edits, corrections, formatting, indexing, embedding fonts and other complexity (I’ll write a blog post about what I had to do at some point), I’ve finally uploaded my thesis “Machine Super Intelligence” to lulu.com and have ordered a test copy.  Once I’ve checked that everything is ok I’ll let you know where copies can be ordered.  Copies should be USD $18 plus shipping for a 200 page casewrap hardcover.  Probably about in a month…

The next generation of Nvidia GPUs will support enhancements such as double precision floating point in order to make them more suitable for general purpose highly parallel computation. There will also be cards with no graphics interface and greater maximum RAM designed specifically for low cost supercomputing applications.

A few months ago I started reading the Overcoming Bias blog, on which Yudkowsky was discussing physics (I’m still not sure why he’s so involved with physics now, but he’s slowly getting around to explaining this).  Anyway, I have found his writings here to be much more to my liking.  His ideas seem clearer, more focused and organised and I find the style and tone to be much improved.  If you like some interesting philosophical discussions and you haven’t seen the blog already, you might want to check it out.

Encouraged by this I decided to have another look at Yudkowsky’s writings on the ethics and safety of artificial general intelligence.  This time I went for one of his most recent pieces: Artificial Intelligence as a Positive and Negative Factor in Global Risk, a book chapter he did for Global Catastrophic Risks.  If, like me, you were put off by some of his earlier writings, you might want to have a look at this newer document.  While I naturally don’t agree with everything in the chapter, in my opinion the points are clearly argued and fit together well.  Indeed, I found myself agreeing with most of his points.  In short, if you are new to the safety of powerful AI technologies, I’d suggest that you put this document at the top of your reading list.

In part 3 I’ll get back to my own thoughts on the matter…