It’s now over a year later so I guess some of you are wondering what happened to AIQ! I’ve been very busy working on other cool stuff, however Joel Veness and I have been tinkering with AIQ in our spare time.  We’re pleased to report that it has continued to perform well, surprising well in fact.  There was some trickiness to do with getting it to work efficiently, but that aside, it worked perfectly straight out of the box.

We recently wrote a paper on AIQ that was accepted to the Solomonoff Memorial Conference.  You can get the paper here, the talk slides here, and we have also released all the Python AIQ source code here.  It’s designed to be easy to plug in your own agents, or other reference machines, if you fancy having a go at that too.

If you’re not sure you want read any of that, here’s the summary:

We implemented the simple BF reference machine and extended it in the obvious ways to compute RL environments.  We then sampled random BF programs to compute the environments, and tested against each of these.  This can be a bit slow, so we used variance reduction techniques to speed things up.  We then implemented a number agents.  Firstly, MC-AIXI, a model based RL agent that can learn to play simple games such as TicTacToe, Kuhn poker and PacMan, but is rather slow to learn.  Then HLQ(lambda), a tabular RL agent similar to Q learning but with an automatic learning rate.  Then Q(lambda), a standard RL agent, and Q(0), a weaker special case. Finally, Freq, a simple agent that just does the more rewarding action most of the time, occasionally trying a random action.  There was also a random agent, but that always got an AIQ of zero, as expected.  The results appear below, across various episode lengths:

The error bars are 95% confidence intervals for the estimates of the mean.  As you can see, AIQ orders the agents exactly we would expect, including picking up the fact the MC-AIXI, while quite powerful compared to the other agents, is also rather slow to learn and thus needs longer episode lengths.  We ran additional tests where we scaled the size of the context used by MC-AIXI, and the amount of search effort used, and in both cases the AIQ score scaled sensibly.  See the talk slides for more details, or the paper itself.

I’ve been very busy with practical AI matters over the last 6 months so the following (crazy?) ideas have been parked in a folder. That’s a bit boring, so I thought I’d throw it out there to see what people think…

While working on computing a Cumulative Prospect Theory model of investor behaviour a couple of years ago I came across the work of Pothos and Busemeyer (2009) where they argue that quantum probability provides a better model of human decision making, rather than taking the Prospect Theory type of approach by Kahneman, Tversky and co. That just struck me as quirky at the time: why on earth would humans implement something as exotic as quantum logic/probability? They don’t actually think that the brain is operating in the quantum regime do they, or is it computing the quantum probabilities from essentially classical computations? Surely not? I decided to pass.

Then about a year ago I was reading up on search and classification systems when I came across a book by Dominic Widdows in which he introduces quantum logic (and quantum probability) as a useful generalisation of classical logic for this domain. Still being somewhat averse to things either quantum or logic I was naturally a bit sceptical about the utility of learning more about this. It’s not that I disagree with either of these areas, the problem is that they are huge monsters that don’t seem too relevant to what interests me… and so I save my meagre mental powers for other endeavours. Nevertheless, Widdows provided some interesting arguments for why this idea might deserve a closer look, and somehow I took the bait.

What I discovered is that, while classical logic can be seen as a special limiting case of quantum logic, or conversely that quantum logic is a “softened” classical logic, there is quite a bit more subtlety to it than that. For example, Busemeyer and Trueblood (2009) argue that quantum inference can be seen as a generalisation of Bayesian inference, the two coinciding only when compatible measures are involved and thus we have a single Boolean algebra of events. Busemeyer, Wang and Townsend (2006) argue that this often isn’t the case in intelligent agents when one type of judgement interferes with another. That’s interesting, but what got me even curious was when I found Pitowsky (2003) in which he shows that the rules of rational betting imply all the main features of quantum probability. Surely that can’t just be a quirk? Piotrowski and Sladkowski (2002) even argue that we can use it to provide a solution of Newcomb’s paradox. Is something really going on here?

What really caught my interest, however, was when I picked up the book “A New Approach to Quantum Logic” by Engesser, Gabbay and Lehmann (2007 – if you google a bit there is a mostly complete draft online, but hey, it’s also pretty cheap to by). In this they discuss the dynamic and holistic aspects of quantum logic, roughly, the way in which it can describe partially observable complex systems that evolve over time and that have a kind of strong global consistency guarantee. Moreover, it is non-monotonic, self-referentially sound, complete and has so called No Windows Theorems which imply that it’s, in some sense, a completely self-contained logical entity. As far as the authors are aware, and they are experts in the area so I’ll take their word for it, this is unique.

At least to me this seems like the kind of system you’d want if you were looking for a foundation for a recursively self-improving artificial intelligence. Perhaps it’s not the way to try to build the first human level artificial intelligence, but if you were super intelligent and really knew what you were doing, maybe this is what you should be looking at…

Now let’s step back get really wild for a moment. If quantum probability/logic is the right way to build a fully self-referential and consistent, recursively self-improving super-intelligence, and it’s also the fundamental mathematical structure of the physical world… surely that can’t be a coincidence.

Seems that we’re now running a couple of years ahead of even Kurzweil’s predictions.

http://dx.doi.org/10.3390/e13061076

Audience: So when do you expect human level AI?

Rich: Roughly 2030.

Whether or not you agree, views like this seem to be becoming more common in academia.

At the time that I made my teenies predictions I thought they seemed a bit like predicting the obvious. But I’m now starting to wonder whether many of my predictions could have been more tightly assigned to the following 3-4 years, rather than the next decade.

One thing I’ve noted in the past is that it’s usually easier to predict fundamental things like FLOPS per dollar than is it to predict how these technological fundamentals will translate into applications. That might be true, but knowing that your computer of five years hence will have X bytes of storage and perform Y computations per second is a bit abstract for most purposes. What will be the new toys, the new applications, the new businesses? These are the things that impact people.

If predicting specific applications is a bit much to ask for (and if I could I might not want to tell you!), perhaps the next best is to predict the general nature of applications during a period of time. What you might call the “technological theme” of a period.

1980 to about 1995 was the period of the PC. Starting with hobbyists and niche applications and spreading to take over a large chunk of the office. The IBM PC marked the point at which this went mainstream. The defining characteristic was that the communication was typically local, if the machines were networked at all.

1995 to about 2010 was the period of the internet. First emails and basic web pages, search, then ordering online, online banking, music, video, etc. Netscape marked the point at which this went mainstream. The defining characteristic was that the communication was now global but the interface with the world was usually pretty traditional: keyboard, mouse, monitor.

So what’s the next theme? Mobile internet might be an answer, but I think that it’s more general than that. As great as the internet is, most of the important stuff still occurs in that other place called reality. Maybe it’s a new house with a swimming pool, throwing a party with friends or coming down with a serious illness. I think the next theme will be for technology to interface more effectively with the world, being mobile is only one aspect of that. If I had to pin the start of this going mainstream on one thing, it’d say it was the iPhone as that’s when the internet started to show up in the day to day moments of people’s lives as they’re out and about doing things.

Once the location, state and function of many everyday objects starts to spread onto the internet, all sorts of creative efficiencies become possible. Need to pay for the coffee? Just press a button on your phone. Not sure where your car is? Ask your phone to show you the way. Need a cab or a pizza or… just select what you want on some menus on your phone. Prices, special deals, time you’ll need to wait — it will all be there. Need to keep a close eye on your health? Get a small sensor implanted that monitors your blood insulin, oxygenation, pressure, cholesterol, heart rate and so on and wirelessly updates this information to your phone. Should a problem arise, your phone can let medics know where you are and what the problem seems to be.

I don’t expect this to be a sudden change, but rather a gradual absorption of goods, services and various everyday objects into an all pervasive information network. I think this will be a hot area until about 2025. Yeah, it’s going to take a while, not so much for many of these things to become possible, but rather for them to become cheap enough to be economic.

What’s my pick for the theme that comes after that? Well, once you have so much of the economy automated and hooked up together with vast amount of information about anything and everything swirling around, the key leverage point becomes how well you can intelligently process all this in order to control and coordinate things. Thus my pick for the theme from 2025 to 2040 is machine intelligence.

It’s been a very eventful year for me, both personally and on the work front. I keep my personal life off this blog, and as for work… um, significant things are happening but I’m not ready to talk about them yet Thus, I’ll just stick to my general predictions this time around.

First of all, my set of predictions for the teenies. We’re only 1 year in so it’s not surprising that I’m still pretty comfortable with the predictions I’ve made. The only tweak I’ll make is that over the last year I’ve become slightly more confident that we’ll have a decent understanding of how cortex works before the end of the decade. That’s my only update.

My longest running prediction, since 1999, has been the time until roughly human level AGI. It’s been consistent since then, though last year I decided to clarify things a bit and put down an actual distribution and some parameters. Basically, I gave it a log-normal distribution with a mean of 2028, and a mode of 2025. Over the last year computer power has increased as expected, and so it looks like we’re still on target to have supercomputers with 10^18 FLOPS around 2018. In terms of neuroscience and machine learning, I think things are progressing well, maybe a little faster than I’d expected. I was toying with the idea of moving the prediction very slightly closer, but decided to play it safe and keep the prediction unmoved at 2028. With many people thinking I’m too optimistic, showing restraint is perhaps wise I can always move my prediction nearer in a year or two.

One thing I screwed up last year was the 90% credibility region. Going by a log-normal CDF for my predicted mean and mode that David McFadzean did (see bottom of this page) the upper end should be a bit higher at 2045, i.e. at a CDF of 0.95. It seems that I got the lower end right, however, as the CDF is about 0.05 at 2018. With 5% at each end, that gives the 90% interval.

I don’t know if anybody has thought of a theme for next year’s conference yet, but I’d like to make a suggestion: ethics and AGI safety. The conference has been around for a few years now and had attracted some fairly big names and serious academics. How about a return to the core mission of SIAI? As I think AGI is approaching, we seriously need much deeper and broader thinking on these topics. One other suggestion: while big names draw the crowds, in my opinion they often give the least interesting talks. How about a couple of the most popular and accessible LessWrong posts get selected and their authors present them as Summit talks?

Sorry about this slip up, and thanks again to all 75 of you who have purchased copies. If there are any further problems please let me know.

UPDATE: after a little reorganisation, a colleague of mine here at the Gatsby Unit, Demis Hassabis will be talking about how systems neuroscience can inform AGI design. I’ll then give a follow up talk on measuring machine intelligence.

“In 2008, the National Academy of Engineering chose as one of its grand challenges to reverse-engineer the human brain. When will this happen? Some are predicting that the first wave of results will arrive within the decade, propelled by rapid advances in both brain science and computer science. This sounds astonishing, but it’s becoming increasingly plausible. So plausible, in fact, that the great race to reverse-engineer the brain is already triggering a dispute over historic firsts. The backdrop for the debate is one of dramatic progress.”

You can find the whole article here. For those who don’t know, Sejnowski is one of the most respected neuroscientists in the world.

It’s certainly been quite a week. Last Friday I headed back to my old home town of Lugano in Switzerland for the AGI 10 conference. I had a great time. In a way it was a bit like the Singularity Summit for me in that the talks were not where the interesting stuff was going on — all the action was taking place in the coffee breaks, over lunch and over dinner. That’s when you get to see what people really think!

Overall, my (biased) summary is that there seemed to be a greater sense of focus among the community than I’ve picked up before, plenty of passion and even a tiny touch of excitement creeping in there. That said, there also seemed to be some significant disconnections in the community, with instances of people being shocked to discover than what they thought was a discredited minority view wasn’t a minority view at all. To me that’s a good sign that the conference is working. Another theme that I picked up was how central Hutter’s AIXI and my work on the universal intelligence measure has become: Marcus and I were being cited in presentations so often that by the last day many of the speakers were simply using our first names. As usual there were plenty of people who disagree with our approach, however it was clear that our work has become a major landmark in the area.

I think the most important missing ingredient of the conference was a lack of discussion about AGI safety issues. From what I recall, during the main conference presentations Mark Waser was the only person to directly take on the topic. During the final workshop session Roko Mijic appeared out of the blue and gave a talk on Yudkowsky style Friendly AI. A show of hands revealed that while half the audience had heard of SIAI, few had heard of CEV. Roko then kicked off his talk by describing the creation of an AGI as likely being worse than the Sicilian mafia combined with grey goo. It’s hard to say what the audience who’d never encountered CEV etc. were thinking at this point, but I’d hazard a guess that they’d written him off as some kind of paranoid crackpot. In any case, what did become clear is that a sizable part of the AGI community is not familiar with FAI thinking.

FHI, Oxford

On Wednesday I gave a talk at the Future of Humanity Institute at the University of Oxford, along with Robin Hanson. My talk was on what neuroscience can teach us about how to build an AGI. Going by all the praise afterwards, it seems that my talk went over very well indeed. I have to say that it was a real pleasure for me too: the topic is a passion of mine and getting to visit Oxford, meet everybody there and stay in one of the colleges was great fun. I often complain when visiting churches (to admire the building and art) that it’s a pity that researchers don’t get such nice buildings. Oxford seems to be one of the exceptions in this regard.

I’m a bit busy for blogging at the moment, but I will be at AGI 2010 this weekend. I’ll also be giving a talk at Oxford next week on how developments in theoretical neuroscience are giving us significant insights into AGI design.

Another interesting paper is Coherence Potentials: Loss-Less All-or-None Network Events in the Cortex by Thiagarajan, Lebedev, Nicolelis and Plenz. They have evidence that above a certain threshold level of activity LFP information is sometimes transmitted across regions of cortex with surprisingly high fidelity.

Another cool recent paper is Rewarded Outcomes Enhance Reactivation of Experience in the Hippocampus by Singer and Frank.  They show that, well, basically what the title says.  This is not surprising, but until now there hasn’t been good evidence to show that this was happening.  If this can be replicated, and some people I know here are considering doing this, it would fill out another part of our understanding of reinforcement driven learning in the brain.

I’m coming across so many interesting neuroscience papers these days I can hardly keep up with reading them, let alone blogging about all of them.  The thing that amazes me is how the architecture of the brain is so logical — it almost looks designed.