AGI 10 and FHI

AGI 10, Lugano

It’s certainly been quite a week. Last Friday I headed back to my old home town of Lugano in Switzerland for the AGI 10 conference. I had a great time. In a way it was a bit like the Singularity Summit for me in that the talks were not where the interesting stuff was going on — all the action was taking place in the coffee breaks, over lunch and over dinner. That’s when you get to see what people really think!

Overall, my (biased) summary is that there seemed to be a greater sense of focus among the community than I’ve picked up before, plenty of passion and even a tiny touch of excitement creeping in there. That said, there also seemed to be some significant disconnections in the community, with instances of people being shocked to discover than what they thought was a discredited minority view wasn’t a minority view at all. To me that’s a good sign that the conference is working. Another theme that I picked up was how central Hutter’s AIXI and my work on the universal intelligence measure has become: Marcus and I were being cited in presentations so often that by the last day many of the speakers were simply using our first names. As usual there were plenty of people who disagree with our approach, however it was clear that our work has become a major landmark in the area.

I think the most important missing ingredient of the conference was a lack of discussion about AGI safety issues. From what I recall, during the main conference presentations Mark Waser was the only person to directly take on the topic. During the final workshop session Roko Mijic appeared out of the blue and gave a talk on Yudkowsky style Friendly AI. A show of hands revealed that while half the audience had heard of SIAI, few had heard of CEV. Roko then kicked off his talk by describing the creation of an AGI as likely being worse than the Sicilian mafia combined with grey goo. It’s hard to say what the audience who’d never encountered CEV etc. were thinking at this point, but I’d hazard a guess that they’d written him off as some kind of paranoid crackpot. In any case, what did become clear is that a sizable part of the AGI community is not familiar with FAI thinking.

FHI, Oxford

On Wednesday I gave a talk at the Future of Humanity Institute at the University of Oxford, along with Robin Hanson. My talk was on what neuroscience can teach us about how to build an AGI. Going by all the praise afterwards, it seems that my talk went over very well indeed. I have to say that it was a real pleasure for me too: the topic is a passion of mine and getting to visit Oxford, meet everybody there and stay in one of the colleges was great fun. I often complain when visiting churches (to admire the building and art) that it’s a pity that researchers don’t get such nice buildings. Oxford seems to be one of the exceptions in this regard.