With the summit still fresh in my mind I thought I’d put a bit of a summary together — or perhaps more a collection of random thoughts and observations. For a less personal overview, read the Reason magazine article.
What I will remember most clearly about this summit was Peter Thiel. Firstly, the pre-summit party at his penthouse apartment. That was a treat: a tiny peak into the world of the ultra-rich. His mix of intelligence, focus and energy was quite something to behold and he left a real impression on me. His talk was also among the most engaging in my opinion. No slides, no fluffy stuff, just a straight delivery of ideas and analysis seemingly off the cuff with no notes. In his talk and comments afterwards, the main thing that stuck in my mind was his concern that the singularity wouldn’t arrive quickly enough. Really?
I don’t understand this. If this is really his main concern, why isn’t he using at least a tiny part of his huge financial resources to try to make it happen sooner? He’s funding SIAI, but they aren’t exactly trying to make the singularity happen sooner. If time frame is his primary concern, and I can’t see any reason why he would say this if it were not the case (it’s easier to think of reasons why he wouldn’t) why the inactivity? Just a few percent of his wealth spread across a handful of small projects over the next few decades would make a night-and-day difference to funding in this area. There is essentially no money available to do AGI research and thus we spend our time working on the related topics in the areas of machine learning and theoretical/computational neuroscience that actually are funded. I guess he’s thought carefully about this, but at least I can’t see how his actions can be consistent with his stated beliefs.
Many of the other talks I’d heard before. In some cases so many times I’m sure I could give them myself. I hadn’t heard about the work of Gregory Benford before where they are breeding up very long lived flies and then, if I understand correctly, looking at how the different bio-chemical pathways in these flies change to produce long life. As humans share many of these pathways, they then make pills that reproduce some of these effects in humans in the hope that we too will live longer. I don’t know enough biology to be able to comment further, but it sounds like an interesting long-shot idea to at least try. I guess it will be some time before we know whether it has any effect.
As per my last summit, the most interesting thing for me was meeting and talking to people one on one, or in small groups. I ended up giving my 20 minute spiel on what I consider to be the most promising approach to AGI at least 10 times. It was quite positively received — I’d expected it to be a harder sell. Many of these people seemed to be revising their expectations after talking to me. One exception, and a person I was especially interested in talking to, was Moshe Looks from Google Research. He strikes me as a pretty sharp and temperate thinker who also has a fair amount of hands on experience working on AI/AGI both academically and commercially. Strangely, we seemed agreed on just about everything regarding which approaches were the most promising, in what degrees and why. I was more bullish on the time line for developments, but not radically so. If I add an extra 50% to my time line, which historically appears to be my degree of miscalibration when forecasting technology developments, then we even come up with the same time estimates. Given that I’ve only met him once before, I found this degree of agreement between us quite striking.
My other impression was the scale of the summit this time: over 800 people which I think was roughly twice the size of the last summit. There also seemed to be a more diverse group attending. Along with the usual mix of geeks, nerds and all too obvious Aspergers cases, there seemed to be more general public interest as far as I could tell. I think the gender ratio might have improved too, indeed there was even a fair number of attractive young women. Year by year, is the singularity idea slowly starting to go mainstream? Perhaps it is and I suspect this summit is one of the driving forces. Well done.
There is essentially no money available to do AGI research and thus we spend our time working on the related topics in the areas of machine learning and theoretical/computational neuroscience that actually are funded.
Close enough, what would you see as the main differences in focus if AGI were directly funded?
That’s a good question.
I think the main thing would be trying to develop AGI systems that bring various aspects of machine learning together into a functioning whole system. This takes longer to do than more limited and focused narrow AI work, and also produces results in narrow test domains that are inferior to existing methods, at least to start with. I think this kind of integration to produce a good more general system is one of those problems that sounds much easier to do than it is, and is also critical to getting an AGI that really works.
> I don’t understand this. If this is really his main concern, why isn’t he using at least a tiny part of his huge financial resources to try to make it happen sooner? He’s funding SIAI, but they aren’t exactly trying to make the singularity happen sooner.
Because he wants positive singularity, as opposed to being converted into paperclips?
That wasn’t the impression he gave me. The impression I got, which might be wrong (I’ll watch the videos again when they come online) is that he thinks there are risks with powerful AGI, but he isn’t as concerned about this as many SIAI people are. His bigger concern is that technology generally isn’t progressing fast enough towards a singularity which he thinks is likely to be positive.
Is there a post here capturing your 20min spiel, or have you considered doing the spiel on youtube? I know you’ve worked on AIXI, and apparently you’re currently interested in “hierarchical generative temporal models”…
Writing a post about this has been on my to do list for the last month, but I haven’t gotten around to it. It would take a bit of time to write properly and over the next few months I’m going to be very busy, so I’m guessing it won’t happen for a while.
From what I remember: machine learning techniques of certain particular kinds, such as temporal difference learning, are actually found in the human brain. The correspondence between theoretical results and what you see in experiments is very good. Therefore we will soon be able to build a mouse-level AI based upon these ML techniques.
Pingback: Accelerating Future » Shane Legg on Singularity Summit 2009
Please do so as soon as you have the time! Until then I will read your thesis.
Thanks for the link.
It might be a while as the next few months are scary busy…
People like Peter Norvig, Barney Pell, James Harris Simons appear to have secured funding.
Organisations like those are unlikley to fund academic research in the area, though – such research would be likely to provide your results to your competitors.
There does appear to be some government funding available from military sources.
Norvig believes that AGI is too far off to be that worried about it. (Moshe told me this based on personal conversations he’s had with Norvig, and it’s consistent with what I’ve heard from Norvig himself in talks).
I don’t know about Pell.
Simons is plenty rich enough, but has he ever expressed any interest in AGI? Does he even know what it is?
Plenty of people are rich enough to try to build an AGI. I’ve read that over 100,000 people have in excess of $30 million, and I think that $10 million would fund a small team for a decade to tackle many of the core issues. You don’t need to be ultra rich like Thiel to go for this, it’s just that people in his category can try without risking a noticeable amount of their total wealth. They could even fund a few competing teams if they’re not sure who to back. Though even they need to be careful: if AGI moves before they expect it to, or it’s not a team they are backing, it may be impossible to buy in even with all their wealth.
I think that $10 million would fund a small team for a decade to tackle many of the core issues.
Given the HUUUUGE amount of, let’s say, moderately useful
research which has already been done in AI, machine learning, statistics, pattern recognition, linguistics, neurophysiology, etc, etc, I see this as utterly delusional, a big cognitive bias which would have to be redressed, by “overcomingbias” may be, LOL…
Why the heck would the small team, however smart and motivated they could be, “magically” stumble on the core issues and solve them?
Have we ever agreed about anything?
> Plenty of people are rich enough to try to build an AGI.
Most are smart enough to see it’s a drip in the ocean, and that realistic impact requires much more. Do you actually believe that any given team has nontrivial chance of getting there? (Where? Overconfident expectation of hitting a wrong target…) All of them is scary, any one of them is a priori insignificant.
It largely boils down to how you think an AGI will be built. If you think it requires a number of radically new ideas and a huge programming effort to build everything. Then, yes, AGI is a long way off and $10 million won’t make any real difference. We might get there in 2050… if we’re really fast.
That’s not, however, my view. I think that an AGI can be built by integrating and extending a number of existing machine learning technologies, using what we know about the brain as an architectural guide. I estimate 10 years before this approach catches on and has a significant probability of success.
Useful ideas tend to be not radically new, yet hard to notice for the first time. It’s on a good day that the lasting output of a research group over a decade is a couple of tools the benefit future progress, while a successful application would be built on understanding of hundreds of such ideas collected all over the history.
Answerbots, oracles, marketbots and security agencies have lots of cash, access to enormous computer systems, laden with mountaines of training data – and have many smart employees.
They face enormously difficult problems, have a lot to gain, and are well placed to make progress. That these folk haven’t pre-announced any product breakthroughs yet means rather little. James Harris Simons doesn’t do announcements. His computer systems toil in secrecy and obscurity – and it’s much the same story with the security agencies.
Oh, if your point is that some group could already be trying this below the radar, then yeah sure, I agree, it could well be happening already. Indeed, I’ve already been approached by a low profile group with very significant resources that is interested in this kind of thing.
The Jim Simmons & als “crowd” certainly has the $10 million you are wishing for (and even more…), kind of an acid test for your optimism, as always “the proof is in the pudding”!
BTW, I am an optimist too somehow, I think that the “key issues resolution” may spring from about anywhere, in the public view or not, but that conventional researchers, well funded or not, are too obtuse(*) to recognize the deal, may be even after it will have emerged.
And since I don’t believe in the exponential burst of AI improvment I think it may go unnoticed for a while.
*) Look how very brilliant people got embroiled in “metaphysics haggling” over a seemingly sensible idea.
In a sense, the singularity is already here. Each of us is a small singularity, a small exceptional 1/x point where different universes meet and worlds collide.
I really wish I could have made it! I enjoyed the 2008 Summit (and got to meet you extremely briefly and the pre-summit event), but this one seems a lot more exciting from the responses I’ve seen online.
I’m working with Ben Goertzel on OpenCog still, and currently we’re in the position of trying to get more funding, so sympathise with your sentiments. It’s proving to be a hard sell, but if all else fails I’ll head back into academia and get a narrow AI post-doc
Yeah, it was good. Quite a lot bigger than last year and the workshop afterwards meant a lot more time to talk to people. Ben mentioned to me that he was trying to arrange some more funding.