People have a series of rationalizations. People say for example that science and technology have their own logic, that they are in fact autonomous. This particular rationalization is profoundly false. It is not true that science marches on in defiance of human will, independent of human will, that just is not the case. But it is comfortable, as I said: it leads to the position that "if I don't do it, someone else will."
Of course if one takes that as an ethical principle then obviously it can serve as a license to do anything at all. "People will be murdered; if I don't do it, someone else will." "Women will be raped; if I don't do it, someone else will." That is just a license for violence.
Other people say, and I think this is a widely used rationalization, that fundamentally the tools we work on are "mere" tools; This means that whether they get use for good or evil depends on the person who ultimately buys them and so on.
There's nothing bad about working in computer vision, for example. Computer vision may very well some day be used to heal people who would otherwise die. Of course, it could also be used to guide missiles, cruise missiles for example, to their destination, and all that. You see, the technology itself is neutral and value-free and it just depends how one uses it. And besides -- consistent with that -- we can't know, we scientists cannot know how it is going to be used. So therefore we have no responsibility.
Well, that is false. It is true that a computer, for example, can be used for good or evil. It is true that a helicopter can be used as a gunship and it can also be used to rescue people from a mountain pass. And if the question arises of how a specific device is going to be used, in what I call an abstract ideal society, then one might very well say one cannot know.
But we live in a concrete society, [and] with concrete social and historical circumstances and political realities in this society, it is perfectly obvious that when something like a computer is invented, then it is going to be adopted will be for military purposes. It follows from the concrete realities in which we live, it does not follow from pure logic. But we're not living in an abstract society, we're living in the society in which we in fact live.
If you look at the enormous fruits of human genius that mankind has developed in the last 50 years, atomic energy and rocketry and flying to the moon and coherent light, and it goes on and on and on -- and then it turns out that every one of these triumphs is used primarily in military terms. So it is not reasonable for a scientist or technologist to insist that he or she does not know -- or cannot know -- how it is going to be used.
Yet another circumstance must be mentioned which proves favorable for the Nazis and their immensely powerful apparatus of oppression: the development of modern technology gives the rulers, as has long been insufficiently understood, an advantage over the ruled. The more effective the weapons become and the less you can protect yourself against them, the more the armed is superior to the unarmed. The Bastille could not be successfully stormed in the age of airplanes and tear gas. Rifles equipped with rifles have no chance against motorized police forces; it makes no sense to build barricades against a government that has tanks. And in the event of a revolution, it is not only weapons development that favors those in power, the state over the individual: modern technical development and the associated sophisticated organization work in the same direction. Traffic has led to the countries becoming small and easy to monitor. How many hiding places there were in a country a hundred years ago! At that time, every power hit natural barriers! Today there is no loophole and no hideout for the rebel anymore. Even the thoughts that are able to penetrate the walls have become "controllable" because they are tied to the mass distribution of news, to radio, film and the press. How long will it take before every house has its own microphone and every private word, like every telephone call today, can be heard? The ant state is at hand. It may not be a coincidence that states like Germany and Russia have elevated technology to the status of a religion. Conversely, this development of modern technology makes the preservation of freedom a human task that is more urgent than ever."Germany: Jekyll & Hyde (1939 - Deutschland von innen betrachtet)"
For the masses, the feeling that technology develops along an inevitable path reflects their lack of agency — the fact that the crucial decisions about the technological conditions of society will be made by a largely self-regulating confraternity of elites. For engineers and scientists, technological development appears to be driven by a combination of what they can imagine, what is technically feasible, and what governments or markets demand. Even those whose particular genius produces the breakthroughs feel this as an inevitability, as if they are possessed by some inner logic that is the real force ushering in this new world.
We live in a society absolutely dependent on science and technology and yet have cleverly arranged things so that almost no one understands science and technology. That's a clear prescription for disaster.
Our entire much-praised technological progress, and civilization generally, could be compared to an axe in the hand of a pathological criminal.letter to Heinrich Zaggler, December 6, 1917