There seems to be a general trend in AGI towards more cortical inspired designs (also going hand in hand with all the advances in computational neuroscience). Actually, I think a full virtual rat is an immediately achievable milestone (we have the computational power already). If a team could produce that, and more importantly, get a simulacra of a rat body that looks like a rat and does rat-like things in a detailed simulation (some of these bio-AI teams should really team up with a full game simulation engine & team), it could really prove the cortical algorithm scalability, and then things could move *rapidly* after that – attracting funding to scale quickly to cat/dog like brains which then can be built into custom hardware for robotics. Right now there isn’t enough money in AGI for it to attract billions of dollars, but a couple of key advances like that could change that almost overnight.

That would be a rapid, sudden breakthrough – and once that is achieved, you’re very close to an infant posthuman brain. That by itself is not scary or profound. But once the cortical model is developed, you can pour money into translating it to custom hybrid digital/analog hardware, and you can run it 1000x real-time. That *is* profound, and that will quickly bring about a Singularity (a time of near-instantaneous change). Smart human minds that can think a 1000x faster than us are not the traditional ‘Big Bad AI’, but they will change the world over night (literally) and are potentially extremely dangerous.

Making them ‘friendly’ will be very important, but that won’t involve programming (its not a technical challenge), it will involve careful social planning, parenting, education, and a controlled knowledge environment.

Wow, for once I am (almost) entirely in agreement with a singularitarian.
No matter the route it will take, brain simulation, machine learning, quantum computing, whatever, Artificial General Intelligence will take ages to develop up to and beyond human intelligence.
Intelligence isn’t magic, it doesn’t sprout out of thin air once you’ve guessed the magic spell, it takes collecting knowledge and the more elaborated the knowledge the more expensive it is to learn (computing wise).
This is the main reason I don’t “believe” in the Singularity and I am not afraid of “Big Bad AI”, I think those scares are just wishful thinking and nightmares of a handful of paranoid monkeys.

If the brain is fairly well optimized, then you need to get a network close to the brain’s connection capacity for brain-like intelligence, and to run it real-time, you probably have to be able to simultaneously update all or most of the nodes at say 100hz, perhaps 1000hz. (conceivably there are algorithm improvements that improve locality and get some gains, but I doubt it would be orders of magnitude) Upcoming fast GPU’s (such as Nvidia’s Fermi) are just about there with a 1GB of memory each and 100GB/s of bandwidth. So if the synapse roughly equals a byte, you only need a million GPU’s – very expensive indeed, but perhaps not unrealistic in a decade. Without moving to custom ASIC, you could climb above 1000kz by running the whole thing from cache – but at astronomical cost. On the opposite side of the spectrum, you could use CPUs hooked up to flash SSD’s, and build a network of the same capacity for orders of magnitude less. But it would run about 1/100th of real time. The bandwidth/memory ratio isn’t changing much across this spectrum – moore’s law is only reducing the total cost.

Now here’s the rub: running at 100-1000hz, it takes the biological brain about a couple of years just to get to primitive speech, 5-8 years to develop to the point where it can use a small vocabulary and read and so on – and thats with supervised learning (parents). So the speed of the simulation is of crucial importance, and it will dominate large scale simulations.

The problem is a full test cycle of an AGI requires a full developmental training cycle – from infant to adult! Thats so incredibly expensive that moving to more custom hardware to speed it up far beyond real-time will be important. And it will be important long before you get to a full human model. Its almost like writing code that takes 30 years to compile. There are many pitfalls during development – most routes usually leading to low intelligence or worse.

You could say that your particular AGI model will reach human level with less training cycles, but thats essentially a bet, an expensive one, and with some evidence against (nature has already ran an astronomical large number of full tweak & test cycles).

Anyway, perhaps from reading the Singularity is Near, or the Blue Brain Project, I’ve thought that the computational power for full brain emulation is still a ways off, and that it would happen a decade or so out due to Moore’s Law, not a new hardware breakthrough. Looks like the latter is actually far more likely. To echo Nathan’s point, more recently I’ve been reading up on progress in neuromorphic hardware designs and its pretty amazing – this is probably going to reach brain level hardware capability soon – if it was scaled up to the high tech, high density fab processes it could happen today (the hardware at least, the exact wiring schematics – the chip layout is still a reverse engineering effort). The FACETS project’s prototype chip and wafer-scale integration system is especially interesting (turn the entire wafer into a usable circuit, no need to cut it up into chips, defects are ok!). Whats even more incredible is that since the synapses are emulated directly with just a handful of transistors (instead of digitally simulating with hundreds of thousands and many clock cycles), they can run the hardware at many many times real-time – 1,000 to 10,000 while still using low power. The implications of that are rather mind boggling. By the time full scale models are built and the brain’s wiring architecture is understood and emulated, these neuromorphic systems will be able to think thousands of times faster than humans.

Its possible that some more traditional software based AGI approach could reach the goal 1st, but the brain reverse engineering and emulation approach looks like its getting the fast track. And more importantly, from day 1 it will have an unbelievable speed advantage.

Regarding Markram’s work. I’m not sure how valuable this will turn out to be. On one hand he’s made perhaps the most detailed cortical model so far. On the other hand, is it detailed enough? There are various things it misses and they could prove to be decisive in some subtle way. Or maybe not. Also, even if his model is correct, we still may not really understand what the underlying algorithm is doing.

My current bet is on something half way along your scale. Not a detailed brain simulation, but rather a partly brain inspired general design with key insights or even components of the system coming from algorithms in machine learning.

My rationale is that there is a sort of sliding scale of elegance, efficiency, and unpredictability versus brute-force and steady progress in AGI research. I’ll try to lay it out here:

Most elegant, and unpredictable – some very clever and well implemented software cuts straight to the chase and enables AGI with a minimum of brute computing power for machine learning.

Medium on both — increased computer power enables scaling of Hierarchical Temporal Memory or some similar machine learning algorithm with only a moderate amount of cleverness in the software to emulate & surpass human intelligence.

Slow, inelegant, brute-force computing combined with slow painstaking reverse engineering of the brain enables emulation of the human brain on future supercomputers. (e.g. Markhram’s Blue Brain project)

So, the reason this new brain-on-a-chip is game changing (in my inexpert worldview) is that it enables the slow, inelegant-but-likely-to-succeed method to happen without significant further improvement in either technology or software. It’s simply a question of funding and scaling this technology up enough. (My best estimate would be a few tens of millions of dollars) And if technology does improve (as it likely will), then the funding will be quite attainable in just a few years (a couple million or less). In ten years from now, if this project pans out, a common desktop computer attached to one of these chips could emulate a super-intelligent human.
Since my upper-limit on likely time-frame of the singularity occurring was based on the fast method (hare) not working out, and the slow method (tortoise) setting the pace. Now it appears to me that the slow method’s timeframe has been reduced from 25-30 years from now (my personal estimate of Markhram’s project turning into a feasible super-human emulator) to a mere 10 years or less.
This is both exciting and rather shocking. Also, if braingate’s technology (now at Braingate 2) continues to improve, ten years should be sufficient for safe, effective high-bandwidth brain-computer interface which could be linked to Boahen’s (or FACETS’ similar, but less impressive, chip) allowing for brain upgrades. So there wouldn’t be any need for software at all, just programming the brain-chip to learn from the brain-interface like normal neurons do…..

Okay, I’ve gone on too long. If you’ve gotten this far, thanks for reading.
-Nathan

“…all the important research will happen in these fields – ie specialized intelligences designed to do the jobs humans don’t have the intelligence capacity for.”

In a sense that’s already true. Computers are vastly super human at numerical calculations, remembering data, playing checkers, and many other things. I see it as a frontier that is slowly getting pushed back.

What limits further increasing the economic value of computers isn’t so much what they already do well, but rather what they can’t yet do. Things that you need a human for.

Anyway, it’s not that I’m wanting to build an artificial human. The reason that I’m interested in how the brain works is that this is a real working system that solves many of these problems that computers can’t yet solve. I want to learn some of the brain’s techniques and tricks. Some say that the brain is too complex to ever understand in this way. I think they are wrong.

My thoughts are in line with Carl’s.

Carl

For a direct simulation approach we’ll need a lot of resources. I think that’s partly because we don’t really know what we’re doing. If we manage to work out the fundamental algorithm we might be able to get away with a fraction of the resources. For example, simulated synapses and neurons can be made far more reliable than real ones. Parts of the algorithm might be able to be done far more accurately and efficiently with just a few numerical computations, rather than using a neural network structure. Of course until we have a working system this is just speculation…

They released an updated version of their visual recognizer demo earlier this week, in fact.

I went to their conference last year and played with their SDK a bit, and it turns out to be just a toolkit of standard machine learning stuff. If there was anything novel in Hawkins’ book, it didn’t make it into the final product.

One of the things Hawkins insisted on in his book is a prediction-based approach. Unfortunately the Numenta toolkit doesn’t do prediction yet. However, they said they’re working on it, and it will be interesting to revisit their stuff when that’s done. -Carl

(I have no information at all about his progress after the book, and would sure like to know more. Does he have a blog?)

That said, if we want to indulge, Modha’s group is doing 1/10-realtime mouse cortex on Blue Gene (as of 2007). Modha estimates mouse cortex at 45 TFLOPS and human cortex at 350 PFLOPS, using a highly simplified model. Markram’s more nuts-and-bolts approach wants 1 PFLOP for mouse. Finally, we should keep in mind that we can expect no more than 1/2 the rated performance of supercomputers in a real application, and often much less than that.

I went to have a look at that paper you link too, sounds interesting. Unfortunately, I get a 404 page not found error.