People have a series of rationalizations. People say for example that science and technology have their own logic, that they are in fact autonomous. This particular rationalization is profoundly false. It is not true that science marches on in defiance of human will, independent of human will, that just is not the case. But it is comfortable, as I said: it leads to the position that "if I don't do it, someone else will."
Of course if one takes that as an ethical principle then obviously it can serve as a license to do anything at all. "People will be murdered; if I don't do it, someone else will." "Women will be raped; if I don't do it, someone else will." That is just a license for violence.
Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on.
There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, the technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility.
Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know.
But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live.
If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used.
It is no accident that modern education doesn't teach the distinction between symbol and thing — if it did, education as we know it would fall apart. After that, after education reshaped itself to provide actual knowledge instead of the symbolic representation of knowledge, the society around us would be transformed.
But in the meantime, most "educated" people cannot tell the difference between a fact and an idea, the most common confusion of symbol and thing. Most believe if they collect enough facts, this will compensate for their inability to grasp the ideas behind those facts.
What gets me though is most IT people can't even grok people's facial expressions, but they'll trust anything that claims it measures “the average user”.
The straight line is ungodly.
Most of us are unable to sort out reality — we can't distinguish between a thing and a symbol for that thing. This springs from several causes. One cause is that we are isolated from the natural world, where the distinction between a thing and a symbol is more obvious. Another cause is our educational system, which simply reflects the intellectual laziness of the society in which it is embedded. A third cause is resistance on the part of vested interests — if we could think creatively, we would be difficult to govern, and advertisers would have to appeal to reason instead of emotion.