Learning to predict the future

One of the things I’ve been thinking about recently is the prediction of the future.  Many people really enjoy doing this and come up with all sorts of wild speculations.  It’s kind of like having the liberty to write your own science fiction, but then taking it a step further by convincing yourself to actually believe it.  Sooner or later the future arrives, and many of the recorded predictions look rather silly.  More cautions people take note of this and often avoid easily falsifiable predictions.  That’s all very well as it avoids them ending up looking like a fool, however it also makes becoming a better predictor problematic as they’re never really forced to contemplate their mistakes.  My preference is to make an honest attempt at specific predictions, along with the reasoning behind them.  Then when the time comes, go back over them and try to work out what went right, what went wrong, and mostly importantly why.  Was it bad luck?  Was I overconfident?  Under confident?  Was some kind of systematic bias at work?

One example of this has been trying to predict the medium term direction of the stock market over the last 15 years.  The evidence so far shows that I’m consistently good at predicting what will happen, but that I predict that it will happen much sooner than it actually does; I roughly need to double my time estimates.  I’m now trying to mentally correct for this bias in the trades I make, but it will take some years to see if this is working.

In technological matters, I’ve generally done quite well.  My picks for Sun, java, digital music, linux, MySQL and open source in general were pretty much on the mark.  I thought machine learning use in industry would be bigger than it is today, but I wasn’t too far off.  My biggest mistake was to badly underestimate how much Microsoft’s revenues would grow over the last 10 years — I thought they’d already almost saturated the market and its ability to pay.  Like stock markets, my most consistent error has been to be to predict that things will happen faster than they actually do.  I typically need to add about 50% to the time required.

As I don’t have a lot of my own technological predictions to look at, and some remain in the future, I’ve recently been looking at predictions made by others.  I found a few of my old computer and science magazines from the early 80′s through to the late 90′s which contained predictions, and I also dug up Kurzweil’s “The age of spiritual machines” written in 1999 in which he has a whole chapter about 2009.  There were a lot of hits and misses, but if I stand back and try to see the big picture, a pattern becomes clear: Predictions about basic hardware performance, even one I saw in a magazine from 25 years ago, are amazingly accurate.  But you probably knew that already.  Predictions about what would be technologically possible to do at a given point in time were not as accurate, but were still pretty good.  Where things really started to go wrong was when they tried to predict not what would be possible, but what the majority of people would actually be doing.

Perhaps some examples would best explain this.  State of the art speech recognition systems, such as some of the systems that were being developed at IDSIA when I was there, work impressively well.  However, once you’ve learnt to touch type it is typically easier, quieter, more convenient (especially when editing or coding) and far more private to use a keyboard.  I don’t care how good speech recognition is, I don’t want to sit in a room full of people talking out loud to their computers all day.  I only know one person who routinely uses speech recognition to input text.  The fact that speech recognition is technologically doable, doesn’t translate into it being practically useful for many everyday situations.

There are plenty of predictions that fail in this way: the prediction that everybody now would be making video calls on their cellphones.  It’s certainly technologically possible, I saw a guy with a phone that could do it two years ago, but almost nobody does it.  Or that most long distance air travel would be in supersonic jets.  Again, technologically possible, has been for a long time, but not done in practice.  Or that all mice would be wireless by now.  Technologically possible, has been for years, but as far as I can tell most new mice still have cords.  Or that most people driving long distance on freeways would get their car to automatically drive itself.  I’m sure that’s technologically possible, but I don’t see anybody doing it.  Or that your computer would log you in based on recognising your face or your voice.  Technologically possible today, but not done in practice.  And so on.

In short: Predicting raw performance is surprising accurate.  Predicting what will be possible using the knowledge and technology of some future date can also be done with moderate success.  Predicting what the population will routinely do, however, is much harder.  The latter is largely decided by habit, cost and convenience.  Simply being possible isn’t enough.  Note that predicting the development of the first powerful AGI is of the second type.

This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

6 Responses to Learning to predict the future

  1. David says:

    Not entirely related to the post but considering you are well positioned to comment:

    What are your thoughts on the AGI developments by adaptive ai? They recently launched their first commercial product, an IVR system. Do you think they will be making a big difference in that market and in their commercial applications of AGI research in general? Is this something to be looking out for in terms of significant steps going forward?

  2. Giovanni says:

    “The evidence so far shows that I’m consistently good at predicting what will happen, but that I predict that it will happen much sooner than it actually does; I roughly need to double my time estimates.”

    Do you realize this makes you immensely valuable for any decent christian adventist sect? :-)

    Btw, your point on fads and fashions in technological trends is quite true. Physicists working on social systems use to say that all one is trying (or would like) to do is to model a bunch of electrons with their charge, their position, their speed — and their opinions. Not very encouraging IMHO …

  3. Shane Legg says:

    David

    It’s a great company. Buy their product. Tell everybody you know about it. Get you friends to buy one.

    Did I mention that I’m a (very minor) shareholder?

    :-)

    Seriously, I’ve haven’t been on the “inside” of the company for some years and I don’t know the IVR market at all. So your guess is as good as mine. One thing I do believe in, however, is Peter’s unbelievable tenacity when it comes to getting things to work. That said, I don’t think they have the kind of major AGI breakthrough that many a waiting for.

  4. Shane Legg says:

    Giovanni:

    Only if these Christians are wanting to predict the general trends in the stock market…

  5. As I’ve been watching the share markets plunge and trying to predict when to resume buying, I’ve found myself reflecting on your post. I think the issue is that technology changes faster than society. So we can predict the rate of change of technology reasonably well, though when we use the same rate of change on society we tend to consistently over-estimate how quickly things will change. I suspect that the society’s mood moves slower again, and therefore our mistakes here are even more pronounced. My biggest consistent mistake has been over-estimating how quickly the mood of euphoria or despair will switch. For example, when the NASDAQ reached 3000 in November 1999 I was so adamant that the bubble would burst that I considered buying PUT options. However, the euphoria continued until the NASDAQ reached over 5000! Another example: I felt there was a significant housing bubble in 2004, but the euphoria continued for many more years and house prices increased significantly in this time. I now feel that the markets have reached a bottom, though trying to use an adjustment factor to account for my over-estimation in the past, I assume that the switch from despair to optimism will be a lot further out than I would normally predict.

  6. Shane Legg says:

    Kelvin

    Ditto for me. I thought the credit crisis was over around the middle of last year! It had already been in the news for quite a while and things seems to be calming down. Just as well I continued to wait before re-entering the stock market.

    Picking the true bottom of the market probably isn’t that important. My guess is that in 5 years time now will look like a pretty good buying period even if the market goes down a bit more in the next year.

Comments are closed.