Artificial neural network renaissance?

September 28th, 2006 · 2 Comments

You have probably all read about the 80 core chip Intel has been talking about at their current developers’ forum. In some ways it’s a bit like the 8 core CELL CPU that’s going into the PS3. The best article I’ve seen so far is here. They predict that 1 teraflop of performance should be possible.

A key question is, how to you feed so many cores? Here’s where a key difference to the CELL lies: They propose to stick 2.5 GB of DRAM to the bottom of the chip, with each core having direct access to its own 32 MB of RAM on its own 40 GB/s connection.

Aside from being generally cool, it occurred to me that 32 MB per core on an extremely fast local connection would be fantastic for simulating neural networks. Perhaps a 1000 times more efficient than what most of us currently do when trying to simulate a neural network on a standard PC. If this technology went mainstream, I think it could create a renaissance in research into artificial neural networks, as well as other easy to parallelize methods such as genetic algorithms.

By the way, more thoughts on Friendly AI will be forthcoming when I have time…

Tags: Neural Networks

2 responses so far ↓

1 David // Sep 29, 2006 at 12:20 am

A very interesting development. From my post yesterday on the AGI list:

It’s my understanding that NM (and probably the whole class of similar AGI architectures) is largely memory-performance limited due to the nature of processes which continuously iterate over large portions of system memory.

What NM-like AGI architectures really need is a 10x-1000x memory performance improvement, moreso than processor horsepower growth. On-die system memory with SRAM performance would fit the bill. The future looks promising, as more transistors squeeze onto chips, the industry is moving to modular microarchitecture approaches to reduce overall design complexity, and such an approach will make possible cheap[er] specialized chips such as those dedicating 90% of transistors to memory and 10% to processor cores.

I think the next 5-10 years of microarchitecture design will be a fun ride; we’ll see radical change from decades-old practices which will bring some wild stuff… then comes spintronics — pushing around electrons is so 20th century!
2 mathemajician // Sep 29, 2006 at 9:58 am

I think the key thing is that the computation model is moving away from the classical von Neumann architecture towards a more distributed model. That is a problem for inherently serial tasks, but for naturally parallel tasks, which includes various things in AI, it’s really great news.

vetta project

Pages

Blogroll