vetta project

Currently the foremost thinker on the ethics and safety of artificial general intelligence is Eliezer Yudkowsky of the Singularity Institute for Artificial Intelligence.  On a few occasions I have tried to read some of his writings on this topic.  Every time, however, I would give up after about ten pages.  I found the ideas to be very jumbled up: a kind of patch work that didn’t flow together to produce any kind of a whole.  I would read something that made good sense to me, followed by what I considered to be an unjustified leap in reasoning.  I also didn’t like his style of writing.  Rather than a dry academic style his writing was more folksy.  Maybe this style appeals to Hofstadter fans, though I’ve never been a fan of the latter myself.  Moreover, I felt there was an unpleasant underlying tone: an attitude that seemed to say “if you object to this statement it is because you are either not intelligent enough or have not studied enough to understand why I am right”.

A few months ago I started reading the Overcoming Bias blog, on which Yudkowsky was discussing physics (I’m still not sure why he’s so involved with physics now, but he’s slowly getting around to explaining this).  Anyway, I have found his writings here to be much more to my liking.  His ideas seem clearer, more focused and organised and I find the style and tone to be much improved.  If you like some interesting philosophical discussions and you haven’t seen the blog already, you might want to check it out.

Encouraged by this I decided to have another look at Yudkowsky’s writings on the ethics and safety of artificial general intelligence.  This time I went for one of his most recent pieces: Artificial Intelligence as a Positive and Negative Factor in Global Risk, a book chapter he did for Global Catastrophic Risks.  If, like me, you were put off by some of his earlier writings, you might want to have a look at this newer document.  While I naturally don’t agree with everything in the chapter, in my opinion the points are clearly argued and fit together well.  Indeed, I found myself agreeing with most of his points.  In short, if you are new to the safety of powerful AI technologies, I’d suggest that you put this document at the top of your reading list.

In part 3 I’ll get back to my own thoughts on the matter…

I don’t trust philosophical arguments all that much. I think this comes from studying mathematics. To start with I try to see or feel a result and I then use this to either understand the essence of an existing proof, or to give me guidance in constructing a proof myself. I remember a proof in the same way: I recall its intuitive essence rather than the mechanics. Intuition then, at least for me, plays a central, however I’ve also learnt not to completely trust my instincts. Too many times I’ve had a clear intuition that has refused to crystallise into a mechanical proof. Eventually, sometimes after a prolonged struggle, the math contradicts what I thought was obvious. In a way these are the most interesting results because this is where you learn something surprising. However, it also makes you cautious of trusting intuitively clear arguments, which of course is the very stuff that good philosophy is made of.

One approach to this problem is to try to turn everything into formal mathematics. The mathematical definition of intelligence that I developed with Hutter is one example of this. I think this works best when a concept has a somewhat functional character that allows you to capture what the concept encompasses without becoming too entangled in its “meaning”.  Unfortunately, many important concepts do not seem to fall into this category. For example, in many instances what is or is not ethical is both debatable and loaded with intuitive meaning. If you can’t formalise these things, the best you can manage is to have arguments that are hopefully intuitively clear — we’re back in the land of philosophy.

The vagaries of ethics already causes plenty of difficulties in the world.  New technologies add new pressures, sometimes causing seemingly robust ethical principles to be stretched to the point of absurdity or neatly sliced down the middle.  Naturally, the more powerful and subtle a technology is, the more it demands of our system of ethics.  What about the ultimate in powerful and subtle technology, the technology of intelligence?  What kind of an ethical system could robustly support such a weight?  If I don’t entirely trust philosophical arguments and I can’t see how the problem can be formalised, this creates a difficult situation.

To be continued…

Father Funes, director of the Vatican Observatory, recently wrote in the official Vatican newspaper that aliens may indeed exist.  Just as I was trying to wrap my head around how these two world views could be integrated, Funes went on to suggest that aliens may even be free from original sin.

At that point I think my mind imploded.

It has a kind of Zen kōan quality about it.  If the sound of one hand clapping isn’t doing it for you, just try contemplating how the doctrine of original sin applies to aliens.

With over 20,000 visitors in 2 days I discovered what my poor web server’s maximum capacity is: about 70 visitors a minute.  It doesn’t sound like that much to me, but a number of times I had to ask my ISP to resuscitate the thing.  I guess all the interpreted PHP code and database access required to build each page is quite computationally expensive.    Anyway, I thought I’d write a short addendum for anybody who is still interested.

First of all, things like SciPy are obviously good tools for certain types of problems. If you’re writing a front end system for a corporate database system or a text editor — forget about it.  That said, plain Python is a pretty good way to code most things in terms of being a language that is somehow both concise and simple.  In my opinion the only real downside is speed: it’s about 100 times slower than C, or around 50 times slower than java.  Anyway, that’s another topic, here I’m more focused on using Python in combination with SciPy.  If you’re dealing with complex transformations on data that has many dimensions, for example simulating networks or machine learning, SciPy is likely to be good tool for the job.

One of the most popular comments on reddit basically said that while writing a program to use matrices can produce very short code, it’s far too difficult to debug. I don’t buy this. Firstly, in my experience, if you get the matrix computations wrong the output is normally complete junk. That’s good news, because the really hard bugs to fix are ones where the output is almost right, or is right most of the time. You also can be pretty confident that all the fancy algorithms doing efficient matrix multiplications etc. are correct. Furthermore, because the code is so short there are only a few places where you have to look to work out what you’re doing wrong. Finally, if you’re working in areas such as machine learning or finance then the normal way to describe things is already matrix notation. If you want to, say, construct a non-parametric density estimate by using a multivariate Gaussian kernel, then your starting point is going to be a set of matrix equations.

What are the problems with SciPy?  As pointed out by one of the commenters to my last post, one major problem is documentation.  This is common with open source projects it seems: people like to write code but they don’t like to write documentation.  The basics seem to be covered ok, but as you get further into the libraries it’s not so good.  And it’s not just that the documentation is missing, but rather the structure of the whole SciPy system is confusing.  For example, let’s say you want to interpolate in multiple dimensions.  You go to the interpolate module and what you find is that it only does up to 2 dimensions.  So you’re out of luck, right?  No, the trick is to look in the image processing library ndarray which has interpolation is as many dimensions as you want.  For a new user this is all really confusing.  Hopefully, as the whole system matures these things will improve.  As the documentation is wiki style, once I’m a bit more knowledgable I’m going to do my part and help out in a few places.  It’s really the least I can do.

Then there is the question of performance.  I’ve been told that the standard Ubuntu SciPy package doesn’t use matrix libraries that are all that efficient.  If you want to have fast SciPy then you need to install BLAS, LAPACK and ATLAS yourself.  Using ATLAS you can then compile the libraries so that they will be automatically optimised for your particular CPU, memory cache sizes etc.  You can then get SciPy to use these optimised libraries.  Just look under the installing SciPy for all the instructions.  I haven’t done this yet but people who have report significant speed increases.

If your code makes relatively few calls (say tens of thousands or less) on relatively big matrices (say with thousands of elements) then (depending on what calls you’re making) your code will probably spend most of its time busy in the matrix library.  With an optimised library you’re then doing pretty well in terms of performance.  You might want to look at things like Parallel Python to take the performance to the next level.

Where performance becomes a problem is when you have the reverse situation: many calls (millions) on relatively small matrices (hundreds of elements or less).  When what tends to happen then is that all the computation time is spent going from Python into the library and back out again. The time spent actually inside the library doing computations is negligible. In this case your code would be much faster if it was written in C++ and directly called the BLAS libraries. Other than trying to rethink your algorithm so that each matrix operation does a bigger chunk of the total work, I’m not sure what to do about this. Using something like Pyrex doesn’t really help all that much (Pyrex takes Python with a few extras and compiles it into C that can then be compiled into a binary library that Python can use). The problem is that even in Pyrex code you still have the overhead of going into and back out of the SciPy library. You can directly access matrices (ndarrays) from within Pyrex in a way which is very efficient, however you’re then left writing the matrix code yourself — ok if you want to do something trivial, but not a good idea if you need to efficiently multiply two matrices.  So in short, if you need to make a large number of calls on relatively small matrices then I’m not sure how you can do this in a way that can compare to C++ used with BLAS.  If I ever find a solution, I’ll let you know.

I’ve programmed in many languages before, indeed I’ve spent at least a year working in Basic, C, C++, C#, java, assembler, modula-2, powerhouse and prolog.  One thing I’ve never done before is Matlab, well except a few basic exercises for some course I did way back.  A couple of years ago I started using python and more recently I’ve started to use the scipy libraries which essentially provide something similar to Matlab.  The experience has been unlike anything I’ve coded in before.  The development cycle has gone like this:

1) Write the code in python like I would write it in, say, java.  I have data stored in some places, then I have algorithms that iterate over these data structures computing stuff, calling methods, changing values and doing various complex things in order to implement the desired algorithm.  10 pages of code, somewhat general.

2) Then I realise that in a few places I don’t need to iterate over something, I can just use some vectors and work with those directly.  7 pages of code, a little more general.

3) Then I realise that part of my code is really just running an optimisation algorithm, so I can replace it with a call to an optimiser in one of the scipy libraries.  5 pages of code, and a bit faster now.

4) Then I try to further generalise my system and in the process I realise that really what I’m doing is taking a Cartesian space, building a multi-dimensional matrix and then applying some kind of optimiser to the space.  3 pages of code, very general.

5) Finally I’m like, hey, how far can I push this?  With some more thought and spending a few days trying to get my head around all the powerful scipy libraries, I finally figure out that the core of my entire algorithm can be implemented in an extremely general and yet fast way in just a few lines.  It’s really just a matrix with some flexible number of dimensions to which I am applying some kind of n-dimensional filter, followed by an n-dimensional non-linear optimiser on top of an n-dimensional interpolation and finally coordinate mapping back out of the space to produce the end results.  2 pages of code, of which half is comments, over a quarter is trivial supporting stuff like creating the necessary matrices, and just a few lines make the necessary calls to implement the algorithm.  And it’s all super general.

Now this is great in a sense. You end up throwing away most of your code now that all the real computation work is being done by sophisticated mathematical functions which are using optimised matrix computation libraries. The bottleneck in writing code isn’t in the writing of the code, it’s in understanding and conceptualising what needs to be done. Once you’ve done that, i.e. come up with mathematical objects and equations that describe your algorithm, you simply express these in a few lines of scipy and hit go.

It’s not just with my financial software either. I recently implemented a certain kind of neural network using nothing but scipy and found that the core of the algorithm was just one line of code — a few matrix transformations and calls to scipy functions.  I hear that one of the IDSIA guys working on playing Go recently collapsed the code he’s been working on for six months down to two pages.

The downside to all this is that you send months developing your complex algorithms and when you’re done you show somebody the result of all your efforts — a page or two of code.  It looks like something that somebody could have written in an afternoon.  Even worse, you start to suspect that if you had really known scipy and spent a few days carefully thinking about the problem to start with, then you probably could have coded it in an afternoon.  It’s a little embarrassing.

While I’m on the topic of economics, I found Fareed Zakaria’s Newsweek article The Rise of the Rest to be an interesting take on some of the global changes that are taking place.

I found Peter Thiel’s talk from the 2007 Singularity Summit interesting.  Are rapid technological changes a driving force behind some of the world’s financial turbalance?  Perhaps the dot-com bubble was a case of pre-singularity jitters?

When I was looking for a new laptop last year I appreciated reading other people’s candid views on their recent purchases, thus I thought I’d write a post to contribute back.

I ended up getting a Lenovo T61p. Specs: Core2 Duo T7500 at 2.2Ghz, 4 GB of RAM, 1680×1050 screen, nVidia Quadro FX 570M graphics card with 256 MB, and a 9 cell battery. One factor in making this decision was that it was discounted through the university by about 30%. It came with 32 bit Visa Business and I installed Ubuntu 7.10, which I’ve now upgraded to Ubuntu 8.04.

Perhaps the biggest factor for me was reliability. Laptops seem to have a tendency to break and that’s a major pain to have to deal with.  Unfortunately, after a few months of talking to people about their laptops I came to the conclusion that a significant proportion of all major makes and models have problems (and yes, that does include Macs… Jesus, you guys are worse than Jehovah’s Witnesses). All I can say is that after half a year I have had no hardware failures, of course your experience might be different.

The good:

The screen rocks. It’s 15.4″ 1680×1050, which gives you about 130 dpi and is still plenty bright. To start with I wasn’t sure about getting such a high res screen as it might make things too small. However, I just set the dpi in Ubuntu and everything rescaled up, i.e. all the fonts and menus etc. are the same size as normal but are now exceptionally clear and sharp. Sadly these high spec screens are not an option in all countries for some reason.

The keyboard is great and I now understand why people rave about ThinkPad keyboards. To start with I wanted the Fn and Ctrl keys swapped, however after a week I no longer noticed.

The chassis is the best I’ve ever had on the laptop. It’s all reinforced with magnesium so it’s super rigid. Indeed, physically the whole laptop feels very strong.

The wireless is perhaps too good? Every few weeks, usually on a Saturday morning, it’ll drop the wireless connection for a few seconds while in my apartment. However, I’ve noticed that when this happens the wireless system is reporting around 20 available networks, so I guess it starts having problems with conflicts? Seriously, I can pick up wireless networks from people’s houses half way down the street.

The performance seems good, but I haven’t really tested it against anything. As far as graphics performance goes, I know that Crysis is infamously demanding in this area. So long as I turn down the screen resolution and limit the texture size to fit in 256 MB, I find the game quite playable. Thus if it can (just) handle Crysis, I’d expect it to deal with most other games without a problem (though obviously this isn’t a high performance gaming machine).

The battery life is pretty decent. I get 3 to 4 hours depending on what I’m doing. And it’s quiet, very quiet. Even under full load it’s still pretty quiet.

The bad (or at least not-so-good):

The track pad is far too small. I’m used to using the red dot to move the pointer as then I don’t need to move my hands, so this doesn’t really bother me. However, if you’re used to big track pads you’ll hate this one. At around 3 kg it’s a bit heavy. The screen is slightly off centre to the right due to the wireless. I thought that might bother me to start with, actually I never notice it. I also thought that having the cursor arrow keys being slightly smaller than standard keys might bother me, but I got used to it. The only hardware feature I miss is not having a built in webcam for use with skype. Instead I have to plug one in. That’s about it really.

Oh, there’s also the missing DVI output. To start with I couldn’t understand their thinking in going with VGA, however now I get it: If you want a desktop set up, then get a docking station and you’ll get the DVI output from that. On the other hand, if you’re running around somewhere and want to plug into a projector, then you’ll want the VGA output anyway. Unless you manage to come across a very new high end projector, in which case you can skip the cables altogether and use the laptop’s wireless USB.

Finally a word on operating systems. It came with 32 bit Vista Business which is kind of a joke, but you knew that already. I dual boot the 64 bit version of Ubuntu 8.04 and am very happy with it. 7.10 had some issues, but with 8.04 things just work. Really.  Wireless, DVDs, all the special buttons to adjust volume and screen brightness, even the graphics system worked without me touching any config files. Once it was installed it informed me that to get accelerated graphics I’d need the closed source driver from nVidia.  So I clicked ok and it installed it for me and restarted X. The only thing that needed real intervention was getting suspend to work. I googled around a bit and found that I needed to add a line to some file to unload and reload my wireless. That fixed the problem. Even the colour laser printer down the hall now works, including the positioning of staples.  There used to be problems with flash player on 64 bit linux, but now that works too. Skype with video, also no problem. The only thing I know of that doesn’t work is wireless USB. Given that I’ve never met a wireless USB device before, it’s not exactly a problem. I guess that in a year or so when they become more common this will work.

Of course Ubuntu still has a few rough edges.  For example, I couldn’t work out how to get the VGA output to work in order to drive my projector. In the X config when I’d click on “Detect External Monitors” nothing would happen. The problem is that I’m running the nVidia stuff and thus I have to run the nVidia config software to set up more complex things like multiple monitors.  Using this software, everything appears to work.

Small update: One of my friends who’s doing a PhD here just got a new Dell laptop today and installed 64 bit Ubuntu 8.04.  Everything worked perfectly straight out of the box: wireless, suspend, sound configuration, dual monitors… everything.

After all this time, I’m finally getting to the stage of knowing enough to know how much I don’t know.  Or in Rumsfeld’s lexicon: unknown unknowns are slowly becoming known unknowns.  Equations, theorems and whole sub-fields of study that I’d previous heard mentioned, or that I’d studied and largely forgotten as they didn’t seem useful, are turning out to be essential for problems I have to solve: optimal control theory, measure theory, stochastic calculus, variational calculus, martingale theory, mean field theory, perturbation theory, kernel density estimation…  if I’d fully understood all these things to start with, I would have been able to knock off a month of work in just a few days.  :-/

As a child in school my intelligence and knowledge was my greatest asset, but as a researcher I’m finding that it’s the main thing holding me back.  If I could double my knowledge and intelligence, I could easily quadruple my productivity.

Anyway, our proto-AGI is now able to learn and abstract spatial patterns, and it should soon start forming generative temporal abstractions.  A backend for securities trading is also starting to take shape.