Until now the totalitarian belief that everything is possible seems to have proved only that everything can be destroyed. Yet, in their effort to prove that everything is possible, totalitarian regimes have discovered without knowing it that there are crimes which men can neither punish nor forgive. When the impossible was made possible it became the unpunishable, unforgivable absolute evil which could no longer be understood by the evil motives of self-interest, greed, covetousness, resentment, lust for power, and cowardice; and which therefore anger could not revenge, love could not endure, friendship could not forgive. Just as the victims in the death factories or the holes of oblivion are no longer "human" in the eyes of their executioners, so this newest species of criminals is beyond the pale even of solidarity in human sinfulness.
It is inherent in our entire philosophical tradition that we cannot conceive of a "radical evil." And this is true both for Christian theology, which conceded even to the Devil himself a celestial origin, as well as for Kant, the only philosopher who, in the word he coined for it, at least must have suspected the existence of this evil even though he immediately rationalized it in the concept of a "perverted ill will" that could be explained by comprehensible motives. Therefore, we actually have nothing to fall back on in order to understand a phenomenon that nevertheless confronts us with its overpowering reality and breaks down all the standards we know. There is only one thing that seems to be discernible: we may say that radical evil has emerged in connection with a system in which all men have become equally superfluous."The Origins of Totalitarianism"
The tune had been haunting London for weeks past. It was one of countless similar songs published for the benefit of the proles by a sub-section of the Music Department. The words of these songs were composed without any human intervention whatever on an instrument known as a versificator. But the woman sang so tunefully as to turn the dreadful rubbish into an almost pleasant sound. He could hear the woman singing and the scrape of her shoes on the flagstones, and the cries of the children in the street, and somewhere in the far distance a faint roar of traffic, and yet the room seemed curiously silent, thanks to the absence of a telescreen.
She knew the whole drivelling song by heart, it seemed. Her voice floated upward with the sweet summer air, very tuneful, charged with a sort of happy melancholy. One had the feeling that she would have been perfectly content, if the June evening had been endless and the supply of clothes inexhaustible, to remain there for a thousand years, pegging out diapers and singing rubbish. It struck him as a curious fact that he had never heard a member of the Party singing alone and spontaneously. It would even have seemed slightly unorthodox, a dangerous eccentricity, like talking to oneself. Perhaps it was only when people were somewhere near the starvation level that they had anything to sing about.
He would have liked to continue talking about his mother. He did not suppose, from what he could remember of her, that she had been an unusual woman, still less an intelligent one; and yet she had possessed a kind of nobility, a kind of purity, simply because the standards that she obeyed were private ones. Her feelings were her own, and could not be altered from outside. It would not have occurred to her that an action which is ineffectual thereby becomes meaningless. If you loved someone, you loved him, and when you had nothing else to give, you still gave him love. When the last of the chocolate was gone, his mother had clasped the child in her arms. It was no use, it changed nothing, it did not produce more chocolate, it did not avert the child's death or her own; but it seemed natural to her to do it. The refugee woman in the boat had also covered the little boy with her arm, which was no more use against the bullets than a sheet of paper. The terrible thing that the Party had done was to persuade you that mere impulses, mere feelings, were of no account, while at the same time robbing you of all power over the material world. When once you were in the grip of the Party, what you felt or did not feel, what you did or refrained from doing, made literally no difference. Whatever happened you vanished, and neither you nor your actions were ever heard of again. You were lifted clean out of the stream of history. And yet to the people of only two generations ago this would not have seemed all-important, because they were not attempting to alter history. They were governed by private loyalties which they did not question. What mattered were individual relationships, and a completely helpless gesture, an embrace, a tear, a word spoken to a dying man, could have value in itself. The proles, it suddenly occurred to him, had remained in this condition. They were not loyal to a party or a country or an idea, they were loyal to one another. For the first time in his life he did not despise the proles or think of them merely as an inert force which would one day spring to life and regenerate the world. The proles had stayed human. They had not become hardened inside. They had held on to the primitive emotions which he himself had to re-learn by conscious effort. And in thinking this he remembered, without apparent relevance, how a few weeks ago he had seen a severed hand lying on the pavement and had kicked it into the gutter as though it had been a cabbage-stalk."Nineteen-Eightyfour"
I cannot tell why the spokesmen I have cited want the developments I forecast to become true. Some of them have told me that they work on them for the morally bankrupt reason that "If we don't do it, someone else will." They fear that evil people will develop superintelligent machines and use them to oppress mankind, and that the only defense against these enemy machines will be superintelligent machines controlled by us, that is, by well-intentioned people. Others reveal that they have abdicated their autonomy by appealing to the "principle" of technological inevitability. But, finally, all I can say with assurance is that these people are not stupid. All the rest is mystery.
What gets me though is most IT people can't even grok people's facial expressions, but they'll trust anything that claims it measures “the average user”.
We don't know a perfected totalitarian power structure, because it would require the control of the whole planet. But we know enough about the the still preliminary experiments of total organization to realize that the very well possible perfection of this apparatus would get rid of human agency in the sense as we know it. To act would turn out to be superfluous for people living together, when all people have become an example of their species, when all doing has become an acceleration of the movement mechanism of history or nature following a set pattern, and all deeds have become the execution of death sentences which history and nature have given anyway."Elemente und Ursprünge totaler Herrschaft" p. 683
Just as bombs follow oil, and drones follow drought, so boats follow both: boats filled with refugees fleeing homes on the aridity line ravaged by war and drought. And the same capacity for dehumanising the other that justified the bombs and drones is now being trained on these migrants, casting their need for security as a threat to ours, their desperate flight as some sort of invading army. Tactics refined on the West Bank and in other occupation zones are now making their way to North America and Europe.
The salvation of the world depends only on the individual whose world it is. At least, every individual must act as if the whole future of the world, of humanity itself, depends on him. Anything less is a shirking of responsibility and is itself a dehumanizing force, for anything less encourages the individual to look upon himself as a mere actor in a drama written by anonymous agents, as less than a whole person, and that is the beginning of passivity and aimlessness.
As I write, highly civilized human beings are flying overhead, trying to kill me.
They do not feel any enmity against me as an individual, nor I against them. They are ‘only doing their duty’, as the saying goes. Most of them, I have no doubt, are kind-hearted law-abiding men who would never dream of committing murder in private life.
What is finished... is the idea that this great country is dedicated to the freedom and flourishing of every individual in it. It's the individual that's finished. It's the single, solitary human being that's finished. It's every single one of you out there that's finished, because this is no longer a nation of independent individuals. It's a nation of some 200-odd million transistorized, deodorized, whiter-than-white, steel-belted bodies, totally unnecessary as human beings, and as replaceable as piston rods... Well, the time has come to say, is dehumanization such a bad word. Because good or bad, that's what is so. The whole world is becoming humanoid - creatures that look human but aren't. The whole world not just us. We're just the most advanced country, so we're getting there first. The whole world's people are becoming mass-produced, programmed, numbered, insensate things..."Network"
The propagandist's purpose is to make one set of people forget that certain other sets of people are human."The Olive Tree" (1936)
Software will absorb new humans as just another object, and map them into some form of homelessness, because that's the most efficient use of us.
Personally I'd feel happier if I grew up on a community surrounded by people that I knew and cared for, instead of being routed around like a tinder hookup, endlessly bidding for cheaper container storage.
I think that's something worth sacrificing a little bit of algorithmic efficiency for.
Let nothing be called natural In an age of bloody confusion, Ordered disorder, planned caprice, And dehumanized humanity, lest all things Be held unalterable!"The Exception and the Rule" (1937)
Frankly, most people have a grossly stunted sense of empathy, with exceptions only for those things that they have firsthand life experience with. An instructive contrast is media depictions of sexual violence against women: for a couple of decades, there has been a concerted effort to decry media depictions of violence against women (sexual or otherwise) that are perceived as unnecessary or trivializing. The success of these campaigns is to be lauded, but what they reveal about how empathy-bankrupt monsters most people are is sort of chilling: people en masse can be taught to play-act empathy by rote if it's drilled into their head that a particular case is no longer socially acceptable, but they don't seem to be capable (en masse) of practicing the skill of empathy itself.
For anyone who watches Game of Thrones: Compare the huge outcry against the depiction of Sansa's marital rape to _an entire season_ of infinitely more gruesome torture porn by the same perpetrator against a male victim, which included genital mutilation. Despite being a far more egregious and lengthy display of gratuitous and stomach-turning violence, the latter caused no backlash whatsoever, and is in fact still fodder for jokes about Game of Thrones-themed foods ("Theon's Flayed Sausage"), including from publications like Huffington Post which were some of the loudest complainants about how unforgivable the former scene was.
TL;DR: It's not a particularly popular opinion, but most people are far more depraved and far less capable of basic human empathy than even their own ostensible moral standards would imply. Like you, I can't relate to how people watch that stuff, but I can definitely understand it.