The first is an editorial in Nature called A risk worth taking.  I think this quote sums it up, “Researchers long ago learned that the last people they should tell about their big ideas are their sources of financial support.”  It then goes on to describe the radical approach that the Bill and Melinda Gates foundation is taking to overcome this problem.  Good on them, but even if this works it only solves part of the problem: if you do obtain funding to undertake radical research and your research fails, which is rather likely, what then becomes of you?  Will you get the next job/grant, or will the guy who did less risky research and got some not-altogether-surprising results that were then published in a mainstream journal?

I know this dilemma.  I guess you could call me a Black Swan Researcher by nature.  I mostly do fairly conventional research because that’s at least close to what I’d really like to be doing, and it keeps me in this line of business by delivering semi-regular conventional results.  I think the only work I’ve ever done that was purely based on interest was the work on defining intelligence and the limits of prediction algorithms.  The rest has been selected based on some combination of interest and necessity.  I wondered if I’d ever fall in love with doing conventional research, and yeah, I do like it, but no, deep down I’m just far too radical.  It’s the stories of Black Swans in strange lands that get my eyes shining and it’s too much a part of my character to ever change.

One thing I admire about my PhD supervisor, Marcus Hutter, is his ability to successfully walk this line.  One of his strategies is to have a diversified portfolio of research that ranges from crazy stuff like AIXI and radical new ideas on model selection, to far more conventional work on… actually, come to think of it, most of his work is a bit radical!  I guess the point is that some of his work is high risk, while some is less so.  Thus, if some of his more risky ideas don’t work out, then he still has a bunch of less risky bits of research that almost certainly will produce publishable results.  It takes some real self discipline and planning to pull this off.  One method, that he used to enforce on me, was to “time box”, that is, to set a deadline for some research into some crazy idea.  If, after say a week of work, I didn’t have something fairly convincing to justify working more on the idea, it had to stop.  But during that week, I had free reign to go crazy.  That’s how I initially developed the limits of prediction work: I thought I had a workable idea, I came back a couple of weeks later and it seemed like it was going somewhere, and so I continued with it.  Other ideas didn’t work out and got stopped.  It takes self discipline, but I think it’s a really good idea.  Another approach that I’ve heard researchers talk about is to have ongoing high risk research, but to only allow it to occupy a specific amount of time each week.  Say, Fridays after lunch.  I remember reading that Andrew Wiles, the guy who proved Fermat’s Last Theorem, did something like this due to the very high risk of failure with this ground breaking piece of work.

A related article, again from Emerson’s twitter feed, is the Physics World article In search of the black swans by Mark Buchanan.  As the article notes, “The publish-or-perish ethic, in particular, strongly rewards those scientists doing more or less routine technical work in established fields, and punishes more risky work exploring unproven ideas that may take a considerable period of time to reach maturity.”  I think this is especially true of AGI research.  With this kind of high risk but potentially high impact research, most of us can only play around at the edges if we want to keep our lines of funding in operation.  Somehow, science needs to get out of this conservative mindset and into a more risk tollerant culture.  The model suggested by the article is that of the venture captial companies in Silicon Valley where they expect most of their investments to fail, but to occasionally have huge successes that more than make up for the losses.  I particularly like the ideas on all this from the physicist Lee Smolin, but I’ll let you read the article for that and other thought provoking ideas.