I don’t trust philosophical arguments all that much. I think this comes from studying mathematics. To start with I try to see or feel a result and I then use this to either understand the essence of an existing proof, or to give me guidance in constructing a proof myself. I remember a proof in the same way: I recall its intuitive essence rather than the mechanics. Intuition then, at least for me, plays a central, however I’ve also learnt not to completely trust my instincts. Too many times I’ve had a clear intuition that has refused to crystallise into a mechanical proof. Eventually, sometimes after a prolonged struggle, the math contradicts what I thought was obvious. In a way these are the most interesting results because this is where you learn something surprising. However, it also makes you cautious of trusting intuitively clear arguments, which of course is the very stuff that good philosophy is made of.
One approach to this problem is to try to turn everything into formal mathematics. The mathematical definition of intelligence that I developed with Hutter is one example of this. I think this works best when a concept has a somewhat functional character that allows you to capture what the concept encompasses without becoming too entangled in its “meaning”. Unfortunately, many important concepts do not seem to fall into this category. For example, in many instances what is or is not ethical is both debatable and loaded with intuitive meaning. If you can’t formalise these things, the best you can manage is to have arguments that are hopefully intuitively clear — we’re back in the land of philosophy.
The vagaries of ethics already causes plenty of difficulties in the world. New technologies add new pressures, sometimes causing seemingly robust ethical principles to be stretched to the point of absurdity or neatly sliced down the middle. Naturally, the more powerful and subtle a technology is, the more it demands of our system of ethics. What about the ultimate in powerful and subtle technology, the technology of intelligence? What kind of an ethical system could robustly support such a weight? If I don’t entirely trust philosophical arguments and I can’t see how the problem can be formalised, this creates a difficult situation.
To be continued…
I don’t agree with philosophical arguments not being formalisable. On the contrary, analytical philosophy is very much concerned with defining concepts as precisely as possible, and spelling out arguments in painstaking detail so as not to overlook any logical fallacies. This is true even for ethical discussions in analytical philosophy. (Of course, it is usually not the case for that French drivel which passes for philosophy among some people.)
@Julian
Put your money where your mouth is: give me an equation for “ethical”.
Just a note to tell Shane that I’m subscribed to his rss feed now, and am particularly interested in his further thoughts on the formalization of ethical ideas.
There are two sides to the discussion. One is formaliziation and one is ethics. Artificial Intelligence can be seen as the formaliziation of reasoning, so the topics are very related. In the most important book on ethics in the anquity, Aristotle’s nicomachean ethics, ethos means good action. Good actions in terms of value for society can not be added to other action, the value expresses in actions themselves.
“give me an equation for “ethicalâ€
The analytical tradition, especially Carnap, has recommended exactly that, namely statements shoudl always be formalized, if they can’t be they shouldn’t be uttered. In Wittgenstein’s tractatus every sentence is a tautology (an equation) and for the rest the guideline is: “Whereof one cannot speak, thereof one must be silent.” The book attempts at describing everything which one can speak about. However later Wittgenstein revised many of his views. He found that there is no 1-1 relationship between language and logic. Language becomes meaningful in contexts. To phrase it differently: the environment provides meaning. One simplistic exampe is a street sign.
I think the problem which one run quickly into is the frame problem. If I give a robot a precise rule after which he should act at all times (Asimov’s and other laws), the question is to what extent this is possible at all. The Stanford Encyclopedia has a very good article on the frame problem [1]. The central question is: “how does the robot limit the scope of the propositions it must reconsider in the light of its actions?” If the robot is sufficiently intelligent he will carry out complex tasks. When someone (person A) describes to another person (peroson B) what he wants him to do, some time will be spent on describing the basic idea. However if A explains to B everything in very great detail he would be better of doing the work himself. If A says to B: “please get me a cup” and B brings a jar that is clearly a violation. If B brings a small cup, A might say: “bring me a bigger cup”. If there is no cup in the right size available and B goes of to buy one that might be a violation.
A good strategy for resolving the problem would be to focus on certain situations (“ethic games”). For instance the situation where a robot is a slave is clearly structured. If the robot serves the masters friend is perhaps different. When the robot has more autonomy it will be more complex. If the robot goes to the mall to buy things, there has to be regulation on the laws. He has to be able to place purchase contracts, he has to be responsible for his actions (criminal law), and so on.
Can we have a mathematical theory of ethics? Economics tries to be this theory. Most of the time the argument goes like the following. Given the preferences of househoulds, which mechanism maxmizes social welfare? Welfare is reduced to the optimal allocation of resources, so many dimensions of the problem are left out, such as the political, cultural, biological and indeed ethical dimension. And indeed the Utiliterian axioms (Neumann/Morgenstern) don’t hold even in the most strictest scenarios.
Anyone who is interested in a more ellaborate discussion can reach me via ben@cosmoi.de
[1] “http://plato.stanford.edu/entries/frame-problem/”
@ Benjamin:
I’m tempted to go and read some Aristotle now.
I wouldn’t say that we should only speak about things we can formalise: the informal often comes before the formal. My concern is that if we can’t precisely formalise ethics how can we construct an ethical system that is strong enough to work well in very extreme situations, specifically, in the context of super intelligent machines?
I think the frame problem that you bring up is a key part of this. I think of the frame problem as being to do with the meaning of something with a context. It’s something which is messy and complex rather than neat; it depends on complicated relationships within large amounts of knowledge. Trying to plug the holes one equation at a time, e.g. Cyc project, is like trying to move a mountain with a teaspoon.
People’s ethical ideas have this nature: big, complex, lots of context dependencies and not even logically consistent if pushed too far. You can’t put all this into a neat formula.
“the informal often comes before the formal.”
I agree. I think this is true in many different ways. For one, we use the terms syntax and semantic to speak about language. The more abstract principle is that of form and content.
I must admit that I up to now I never considered the ethical problems of AI in terms of the views of Singinst-related people. I go more with the Hofstadterian viewpoint, which is: a) it is important to discuss these matters b) Kurzweil’s singularity scenario makes to bold predictions. Having abstract arguments is one thing, trying to imagine a sensible scenario and it’s implications is another. I would really like to see more texture to some of this discussion. What would a machine look like? How will it integrate with other trends? Certainly I believe that AI is drastically short of funding vis-Ã -vis it’s potential benefits. I think this is a question of having good business models.
@ Benjamin
I too would like to see a more textured debate. In my opinion the fundamental problem is that people do not know what they are talking about. I mean this literally: they are talking about super intelligent machines without a clear definition of what they mean by this. Indeed, people can’t even agree on what “intelligence” means. Until some kind of a consensus is reached regarding definitions you can’t have a sharp discussion of the issues and possible solutions. This is true of any subject. Some people might not like my mathematical definition of intelligence, but they certainly cannot accuse me of being vague or somehow trying to dodging the issue of giving an explicit definition.
Regarding the singularity. I think Kurzweil has some good ideas up to the point of roughly human level intelligence. From that point onwards I take a more Vingian stance – I don’t think anybody has good predictions as to how things will play out.
AGI will be a truly simple case. Since any AGI will outperform humans in everything, the only thing you should be worried about are the ethics that AGI will have, not so much our behavior against AGI “entities”.
Except for a trivially short period of research, we won’t have power over AGI, AGI will have power over us.