Doc Searls, Linux Journal:
In Re-engineering Humanity, Brett Frischmann and Evan Selinger have dug deeply into what’s going on behind the “cheap bliss” in our fully connected world.
What they say is that we are all subjects of techno-social engineering. In other words, our algorithmic conveniences are re-making us, much as the technologies and techniques of agriculture re-makes farm animals. And, as with farming, there’s an extraction business behind a lot of it.
They say “humanity’s techno-social dilemma” is that “companies, institutions, and designers regularly treat us as programmable objects through personalized technologies that are attuned to our personal histories, present behavior and feelings, and predicted futures.”
And we are not innocent of complicity in this. “We outsource memory, decision-making and even our interpersonal relations…we rely on the techno-social engineers’ tools to train ourselves, and in doing so, let ourselves be trained.”
There are obvious benefits to “delegating physical, cognitive, emotional and ethical labor to a third party”, such as Waze, but there are downsides, which Brett and Evan number: 1) passivity, 2) decreased agency, 3) decreased responsibility, 4) increased ignorance, 5) detachment and 6) decreased independence. On the road to these diminished human states, we have “fetishised computers and idealized computation”.
Surveillance of people is now the norm for nearly every website and app that harvests personal data for use by machines. Privacy, as we’ve understood it in the physical world since the invention of the loincloth and the door latch, doesn’t yet exist. Instead, all we have are the “privacy policies” of corporate entities participating in the data extraction marketplace, plus terms and conditions they compel us to sign, either of which they can change on a whim. Most of the time our only choice is to deny ourselves the convenience of these companies’ services or live our lives offline.
Rather than obsolesce these kinds of contracts, digital technology and the internet allowed companies to automate them. The effects are not good for the human side of each contract. Brett and Evan explain, “Our current online contracting regime is a compelling example of how our legal rules coupled with a specific technological environment can lead us to behave like simple stimulus-response machines—perfectly rational, but also perfectly predictable and ultimately programmable.”
But what happens when our senses extend beyond the metal carapace we wear when we drive a car—outward through the unseen systems guiding our selves and every other car on the road? In this state we are not GPS satellites and Google data centers, but rather puppets at the ends of digital strings pulled by AI puppeteers.
We surely appreciate and rely on what they provide us, but we also yield agency in the process of blurring between our automotive selves and a vast system of dependencies, which even if they are doing good things for us, make us less than human—or, in Brett and Evans’ words, “simple machines under the control and influence of those in control of technologies”.
The engineers among us have a hard job. But it should help to know how high the stakes are, and how we’re already embedded so deeply in a re-engineered dystopia that we can’t see how tragically ironic cheap bliss really is.