More history. The article below explores history of the successful
propping up of a tool to level of science.
I think that this is a major 'elephant in the room' phenomenon that has
permeated everything. Including nettime, of course - I'm referring to
philosophical discourses on primitive and simpleton application of
Merkle trees in Blockchain algorithm, assigning this f*cking idiocy
magic powers to do evil, good, or a combination thereof.
From https://thebaffler.com/salvos/blame-the-computer-pein
Blame the Computer
The fake science that keeps threatening to kill us
Corey Pein
Evidence mounts that the forces of digital civilization have produced a
technological dystopia run by artificially unintelligent algorithms
designed in the interests of greed for maximum efficiency. And true to
the tropes of many a dark sci-fi reverie, these impersonal arbiters of
our collective fate evince neither pity nor mercy—which means, among
other things, that one entirely foreseeable byproduct of their operation
is to inflict maximum terror on the human population, whose
participation in the system is ritualistic at best. Had there been any
residual reason to doubt any part of this glum portrait of our
remorselessly data-engineered vision of the human future, well, it was
rudely laid to rest on Saturday, January 13, at 8:07 a.m. Hawaii local time.
The bad news arrived, as all news seemingly does now, as a smartphone
push notification. But this notification looked and sounded different
than most. It was, in fact, an emergency alert from the state
government. The form of the alert was not a radio wave or an electronic
signal, as in decades past, but computer code. Attached to the alert was
an audio file with a recording of a synthesized male voice, for the
vision-impaired. “The U.S. Pacific Command has detected a missile threat
to Hawaii,” the voice said. “A missile may impact on land or sea within
minutes. This is not a drill. If you are indoors, stay indoors. If you
are outdoors, seek immediate shelter.” The stilted tone of the robot
voice was all the more eerie, tasked as it was with effectively
announcing the impending death of whoever heard it. “We will announce
when the threat has ended,” it said. “Take immediate action measures.”
Take what? And who was “we”? For many, “action measures” meant running
around in panic. More level-headed folks tore through their pantries
searching for bottled water and canned foods, then hid under a pile of
mattresses, or squeezed into bathtubs with their bawling children.
Thousands said tearful goodbyes to loved ones over the phone or, failing
that, to strangers over social media. Some had their possible last words
mangled by autocorrect: “Is a missile ducking coming. Holy shit.”
Ducking hell.
Managers at Starbucks and McDonalds, in the style of British cavalry
officers in Crimea, ordered workers to stay on duty, missiles be damned!
One young woman responded with a definitive anti-endorsement of an Oahu
confectionary chain, writing: “DON’T TELL ME THIS MISSILE THING IS REAL
I AM NOT DYING AT COOKIE CORNER.”
The Wrong Button
A few skeptics wondered why they didn’t hear sirens. Five minutes after
sounding the alarm, and having received confirmation from the United
States military that there was, in fact, no missile headed their way,
Hawaiian civil defense officials attempted to “cancel” the mass alert.
But it was too late. The freak-out signal had been received, and there
was no taking it back. Finally, at 8:46 a.m., almost forty minutes after
receiving the first urgent message, phones across the islands began to
buzz with the morning’s second official announcement: “There is no
missile threat or danger to the State of Hawaii. Repeat. False Alarm.”
Dread gave way to bewilderment, relief, and various forms of catharsis.
The website PornHub released statistics showing a 48 percent increase in
web traffic from Hawaii once the emergency was rescinded (whereas the
initial alert had prompted a 77 percent drop in porn-surfing on the
site). Next came outrage. Heads must roll! But whose?
It took five more hours for Hawaii Governor David Ige to appear on
television to take responsibility for the false alarm. (Later, Ige
confessed he hadn’t known the password to his official Twitter account,
which had further delayed the state’s effort to reverse its error and
send an all-clear signal.) Apologies wouldn’t cut it, however. What the
hell had happened? The public demanded an explanation.
Ige delivered one. It seemed too absurd to be real, but too embarrassing
to be a lie, which gave it the ring of truth. Someone, the governor
said, had “pressed the wrong button.”
“OOPS!” replied the front page of the Honolulu Star-Advertiser, in a
font size typically reserved for actual declarations of war. The news
that no one would be fired for the mistake only aggravated public anger,
and stoked more calls for swift managerial retribution. Was improper
training to blame? Not likely, it seemed—the state said the unnamed,
butterfingered state employee was a “veteran” on the job. (This
characterization later changed dramatically; the newly problematic
employee was duly thrown under the bus, and his boss resigned.)
The System Worked
The quest for scapegoats next turned toward the Trump administration.
The president himself had (thankfully no doubt) been out of the loop,
spending the morning of Hawaii’s false alarm on the links at his
eponymously branded golf course in West Palm Beach, Florida. Well, then,
what about his Federal Emergency Management Agency chief, Brock Long,
who had botched the post-hurricane response in Puerto Rico? In the great
tradition of evasive bureaucratic action, FEMA shifted blame to the
Hawaii Emergency Management Agency, whose administrator, Vern Miyagi,
oversaw the alert system. What about him? Indeed, what about the state
contracting officer who had hired the outfit that made the agency’s
software—Alert Solutions, Inc., of California? What about its Israeli
founder, Efraim Petel? What about BlackBerry, Ltd., the Canadian
multinational that had acquired Petel’s company and inherited its annual
maintenance contract with Hawaii? Did BlackBerry CEO John Chen have
anything to answer for? No one asked.
Eventually, the source of the panic was uncovered—in the clunky,
counterintuitive design of the software that the Hawaii Emergency
Management Agency used. It turns out that, when users engage the missile
alert prompt in the system, they’re greeted by a drop-down menu with
only two choices: one to test the missile alarm, and another to sound
it. A few software designers and “user experience” experts from the tech
industry faulted this dangerously simplistic virtual construction—a task
made no doubt easier by their perfect hindsight.
Managers at Starbucks and McDonalds, in the style of British cavalry
officers in the Crimea, ordered workers to stay on duty, missiles be damned!
But here’s the thing: however much critics and alarmed Hawaiians might
crave the cathartic release of blaming, and cashiering, a fellow human,
there was no plausible scapegoat on offer. In other words, the system
had, in fact, worked as designed—it was just obeying a cosmically
disastrous user prompt.
Only one month prior to the January fiasco, Miyagi’s agency faced stern
criticism from the press after emergency sirens failed to sound in a
nuclear attack drill. Here was something of a photographic negative of
the missile attack scare, in which the emergency alert system didn’t
appear to be stoking enough hypothetical alarm. Miyagi reassured the
Star-Advertiser that despite this testing glitch, his failsafe system
had actually achieved one of its major goals—alarms could be sounded
using “a single button in an emergency operations center in Honolulu.”
Indeed, everyone conceivably responsible in the system’s test run had
performed more or less to specification.
So the panic-stoking false alarm in January was a snafu of software
engineering—and the implicit faith that every problem, even nuclear
apocalypse, awaits a simple, convenient, digital solution. Put another
way, it was a carefully designed byproduct of computer science.
As if to drive this point home for the world at large, a nearly
identical false alarm went off three days later, sending out an
electronic alert about an impending nuclear strike by North Korea. As
before, there was no missile. This time, the purported target was Japan,
where, as in Hawaii, millions felt the same terrifying surge of
adrenaline and apocalyptic dread as they looked to the skies. The state
broadcaster, NHK, blamed an unexplained “switching error” on an
unspecified “device.” Translation: someone pressed the wrong button.
Too Brain-dead to Fail
It’s undeniably a good thing that so far these snafus happened with
alarm systems rather than missile launchers. But it doesn’t take much
imagination to foresee how easily a false alarm could prompt a globally
catastrophic retaliatory strike—especially considering how often
President Mr. Big Button lets it be known that he trusts Twitter and TV
more than his own intelligence officers. In mulling over the multitude
of nightmarish possibilities with a writer for The Atlantic, one arms
control expert grimly concluded that “there is no fail safe against
errors in judgment by human beings or the systems that provide early
warning.”
Those systems are also made by human beings, of course. Many are tempted
to think that the world would be safer if they weren’t—if only somehow
the systems we depended upon could perfect their own design. Perhaps
humans should not be trusted with something so important as crafting the
warning system intended to safeguard the future of the species.
This reflexive distrust of the human mind is the conventional wisdom of
our technological age. It’s most clearly evidenced in the present
tech-industry infatuation with “artificial intelligence” startups. But
almost all political decisions—from the construction of electoral
districts to the drafting of political slogans to the deployment of
military drones—now come before nominal human decision-makers only after
they’ve been filtered through a computerized process. And despite the
abundant evidence to the contrary, many will tell you this is a good
thing. But after spending four years immersed in the madness of Silicon
Valley for a book, I’ve come to a different conclusion. The great peril
we face comes not from an over-reliance on human judgment, but from a
distinct lack of it. Indeed, the most bone-jarring risks before us have
less to do with human error than with engineering hubris—and that hubris
has been synthesized into the uncritically celebrated discipline of
computer science. Even more than “military intelligence,” computer
science is an oxymoron.
The Call-Service Apocalypse
I’m not talking here of the “deep state” or the psyops confecting of a
“fake news” campaign or “false flag” operation sprung on unwitting
American netizens. The grinding rationalization of pre-existing power
relations and bureaucratic prerogative at the heart of computer-science
theorizing is nothing so dramatic. Nor is it secret.
No, the plodding, catechistic precepts of software design mainly serve
to ratify the status quo drudgery of bureaucratic servitude—and indeed
to elevate it into a theory of crudely incentivized mass deference.
Strip away the nomenclature of cybernetic systems theory and software
design, and you have something very close to the plot prospectus for a
Philip K. Dick novel.
The basic setup is as follows: we’re victims of a terrible subroutine
that’s been grinding away for more than seventy years within the larger
program of capitalism. The imperative for economic efficiency has
created irresistible incentives for the automation of thinking. Every
decision that can be made in advance by managers, and reduced by
engineers to a series of switches inside a computer program, has been
dutifully ground down into a binary decision tree. All that’s left for
humans is to make a selection from a drop-down menu.
Thus our day-to-day lives have come to resemble a call-center phone
maze, defined by a wearying succession of false choices—all of them
miserable—and eventually escalating toward a frustrating confrontation
with a powerless authority figure. Inevitably, this person cannot help
because “the computer won’t let me.” Their voice sounds human, but their
words might as well belong to an AI program, smoothly reciting a
self-preserving script amid the specter of apocalyptic ruin: I’m sorry,
Dave. I’m afraid I can’t do that.
Biological evolution took sixty-five million years to produce the human
brain. We outsourced that asset at the earliest opportunity. In a few
short generations, we’ve reached such an atrophied mental state that
nuclear geopolitics works exactly like the customer service department
at Comcast or Wells Fargo.
Engineering Mindlessness
Do I exaggerate? Consider the bloodless, and clueless, way that the
national security intelligentsia responded to the Hawaii fiasco. As
islanders’ panic subsided, the retired U.S. Army Lt. Gen. Mark Hertling,
a CNN commentator, publicly chastised U.S. Rep. Tulsi Gabbard, who broke
protocol by notifying her constituents via Twitter that the missile
alert was mistaken. “For the record, Congresswoman Gabbard inserting
herself into the process . . . is NOT a good thing,” Hertling wrote. How
could it be that the truth—that Hawaiians faced no danger—was too
dangerous to speak?
Well, because such notifications fell well outside the boundaries of
permissible official conduct, as modeled by the software. Hertling
claimed that military simulations had shown that comparable political
“interference” in such an emergency could result in tens of thousands of
additional deaths. Significantly, the final authority in Hertling’s
ideal scenario did not belong to elected officials who might “interfere”
with “the process,” but to the process itself—a semi-automated, fully
computerized system devised and controlled by the military-industrial elite.
How did we get to this seeming endpoint of utterly nonreflective
computer agency? In the perfect world envisioned by Hertling and
countless other natsec apparatchiks, the automation of judgment has
become so thorough, and “the process” so holy, that those authorities
will insist upon deference to instructions from a computer, even when
they know the instructions were made in error. What sort of organization
asks people to behave this way? No participatory government could
withstand the sort of brainless obedience “the process” demands.
However, it would certainly be expected inside any authoritarian cult.
The curious thing about the computer-science cult is that it was first
consolidating in the face of a barrage of criticism warning against just
this sort of outcome. In the middle of the last century, when cybernetic
intelligence was still largely a drawing-board proposition, the inherent
limitations of computers were better understood, even by members of the
cult.
The late MIT professor Joseph Weizenbaum, who taught in the computer
science department, understood the mulish shortcomings of the standard
computer program better than most of his contemporaries. As a result, he
ended his career a heretic and an outcast from the field. His first
heresy was to insist that scientists, who had fallen in love with
computers, didn’t really need them to do their jobs. For example, he
noted, the scientists recruited into the Manhattan Project managed to
invent the atomic bomb without the help of computers. Weizenbaum was
sure, however, that if those same scientists did have access to
computers at the time, they would have sworn the job was impossible. In
other words, he grasped just how seamlessly the power of automation
worked to indulge our laziest tendencies as a species. Fortune Magazine,
1955
Up the Academy
In Weizenbaum’s view, many in his field were no more than “tinkerers
with techniques”—charlatans who had managed to associate themselves with
science in order to “siphon legitimacy from the reservoir it has
accumulated.” This insight seems shocking now, at a moment when computer
science has casually annexed much of our common world: just because most
scientists used computers, didn’t make all computer users scientists.
The curious thing about the computer-science cult is that it was first
consolidating in the face of a barrage of criticism warning against just
this sort of outcome.
Weizenbaum blamed “accidents of history” for the christening of computer
science departments within academia. “All work done in such departments
is indiscriminately called ‘science,’ even if only part of it deserves
that honorable appellation,” he lamented. “Not everyone who calls
himself a singer has a voice.”
As it happened, Weizenbaum got one key point wrong: the elevation of
computer science was not an accident, but a deliberate branding decision
made by veterans of the postwar military-industrial complex. These
grey-suited gadget peddlers banded together to misappropriate the name
of science in the service of sales. They needed a civilian market for
their products, and so naturally gravitated toward a sector with deep
pockets and establishment cachet: the university system.
The first big fish to take the bait was Stanford University provost
Frederick Terman, a radio engineer by training and an administrator by
ambition. Although Terman’s role in the development of early Silicon
Valley has been overshadowed by contemporaries such as William Shockley
(the notoriously racist physicist and semiconductor-company executive),
Terman was arguably more important, given his talent for finding
money—especially grant opportunities from the Defense Department.
In the early 1950s, Terman, then dean of engineering, convinced his
Stanford superiors to set aside a large parcel of land as an “industrial
park,” where private tech companies could enjoy favorable leases and
access to university resources. Hewlett-Packard, cofounded by two of
Terman’s students, was among the first companies to set up shop at
Terman’s park, soon to become renowned as the center of Silicon Valley.
His vision was to position Stanford as an “entrepreneurial university,”
more responsive to the fleeting imperatives of money and power than to
pedagogical traditions. In that respect, he was decidedly ahead of his time.
In addition to cutting real-estate deals, Terman also wanted to raise
the profile of computers on campus. He’d been introduced to the devices
by his former MIT doctoral adviser, Vannevar Bush, who was the
government’s chief administrator on the Manhattan Project (and no
relation to the presidential dynasty). It’s hard to imagine now, with
men such as Elon Musk and Jeff Bezos having attained the status of
billionaire demigods, but in the early 1950s, computer nerds were pretty
much social nonentities. The gadgeteers lacked the standing of their
more prestigious contemporaries—mathematicians wrestling with unsolved
theorems, physicists who tackled cosmic problems by questioning the
basic assumptions of our perceived reality, and social scientists who
delved into the ambiguities of human structures and relationships. In
those departments, computers were seen as mere tools—novelties, even. A
department dedicated to computers made as much sense as a Department of
Slide-Rule Studies.
But Terman was undaunted by such narrow thinking. He saw the pecuniary
gains to be won by catering to the needs of the growing,
government-backed high-tech industry. He also grasped that the study of
computers lacked a certain gravitas. And so he decided to conjure the
field’s animating mission—and far from coincidentally, its fundraising
appeal—out of thin air, with the assistance of another bureaucratic
visionary. In 1958, Terman commissioned a computer salesman named Louis
Fein to study and report on the feasibility of launching a new
university department devoted to computers.
Synnoetics on the Make
In reaching out to Fein, Terman had selected an ideal emissary for the
comp-sci cult’s fundraising gospel. Fein was a former Raytheon engineer
who’d worked on missile guidance systems during the war (yes, the cursed
missile-tracking platform that triggered mass panic in Hawaii more than
half a century later was present at the very creation of computer
science). He’d gone on to found his own business based on Raytheon’s
technology, called the Computer Control Company. Fein also worked as a
consultant for the Stanford Research Institute, another early beachhead
in the military-industrial march on American academia. The institute
essentially laundered publicly funded research through its nonprofit
status, prior to the work’s eventual patenting, privatization, and
profiteering at the hands of savvy middlemen like Fein. Such institutes
created a system whereby the taxpayers would pick up the bill for
research leading to such innovations as the computer mouse and the
internet, but the profits from their commercialization would accrue to a
few lucky insiders.
Fein approached his task by reading narrowly and schmoozing widely. In
1961, he published a kind of manifesto in American Scientist magazine.
Fein’s paper was framed as a fictional speech by a prestigious
university president, set in the future year of 1975, and looking back
upon the tremendous progress of “the computer-related sciences” in their
long slog toward respectability. Fein’s narrator, a thinly veiled alter
ego, proposed that the burgeoning field of “computer-related sciences”
be further elevated with a new moniker, one scarcely heard since:
“synnoetics.”
This new field, Fein insisted, was about more than computers—although
“we were acutely aware of the public relations value of this word.”
Synnoetics would encompass not only computers—which were “but one
species of automata,” he wrote—but cybernetics, “intellectronics,” and
other buzzwords that might as well have come from a Bay Area TED Talk
circa 2010.
Still, Fein’s central neologism hinted at grander ambitions. The word
synnoetics, “derived from the Greek, means pooling together the
resources of the mind,” Fein explained. Synnoetic technologies would
enhance man’s ability to solve problems—“to lift himself by his own
bootstraps.” As he saw it, synnoetics was “supradisciplinary”—its
purported power to improve human mental abilities placed it above other
fields. (Here, Fein echoes the allied academic cult of neoliberal
economics, which has tirelessly sought to promote itself to the
credulous world at large as “the imperial science”—i.e., the discipline
that effectively explains, and rules over, all others.)
As he laid out his vision, it became clear that Fein had grand designs
for the spread of computer-driven inquiry throughout the known
intellectual world. He gushed over how the computers would elevate the
practice of “engineering, law, music, chemistry, physics, medicine,
psychology, and other disciplines”—but especially “management and
control.” Tellingly, the first example he concocted to demonstrate the
power of applied synnoetics involved the deployment of robot Pinkertons
to break a strike. “I am sure you all recall how the famous strike of
1970 was settled when one of our faculty mediators used an automaton to
aid both parties in agreeing to what was at once an optimum settlement
for both sides,” Fein wrote. There should be no mystery why Fein’s
fantasies exerted instant and widespread appeal for administrators and
executives.
The Boss’s Data
It’s important to recall, at this late date in the computerized
enclosure of the American commons, that Fein was not proposing anything
resembling the promiscuously mobile and networked computer scene of the
twenty-first century. In the early postwar period, computers were rare,
expensive, and so big they might take up entire rooms. But in his
manifesto, Fein described an arrangement whereby these apparent
disadvantages could be leveraged to the benefit of the computerized campus.
Universities could buy computers at a discount from the manufacturers,
on the condition that they train a certain number of students, staff,
and faculty in how to operate the devices—thus effectively covering the
cost of workforce training for those companies. As an added sideline,
universities could sell time on their fancy new machines to interested
third parties, especially private companies. Adopting this model, the
university computer lab immediately became a profit center, and faculty
from other departments found themselves competing for resources with
private companies that had become paying customers of the university.
Stanford established its division of computer science in 1963; it became
a full-fledged department two years later. (Purdue’s computer science
department came earlier, in 1962, also at Fein’s urging.) Fein took his
show on the road and began pitching the lucrative new field to
universities all over the United States and Europe. Other
“entrepreneurial” institutions such as MIT hopped aboard the bandwagon,
and as the Cold War escalated, computer science departments sprouted
across the campuses of the land like poisonous mushrooms.
At least 295 U.S. colleges and universities offer degrees in computer
science. Graduates number in the many tens of thousands each year, and
their ranks swell with double-digit annual percentage growth rates as
colleges embrace their new role as glorified job-training centers, and
students flock to the promise of a secure career path. Politicians, too,
have seized upon the purported value of computer mastery as the solution
to all social and economic ills.
Coding with Impunity
The term synnoetics obviously never caught on. But Fein’s concept of
computer science as a “supradiscipline” definitely did. Computer science
is the most exalted field in the new academic paradigm of STEM
supremacy. The profit- fueled fetish for “digital learning” has
coincided with the chauvinist denigration of the humanities and social
sciences. Computer skills have become synonymous with talent and
ingenuity. And the occupation of programming, which in its earliest
iteration carried the stigma of “women’s work,” has become a
high-status, highly compensated, and highly male-dominated field. The
tech bros are all ninjas and rock stars in their own minds and ours. The
most powerful among them, like Google’s Sergey Brin, actually aspire to
become immortal gods.
And yet the problems with computer science as any sort of credible
stand-alone academic discipline are persistent and well known. A 2006
study by Michael J. Quinn, a computer science professor at Oregon State
University, polled a sample of fifty accredited computer science
programs to determine how they taught students about ethical
issues—presuming, that is, they even bothered to try. Most gave ethics
minimal consideration—a single credit hour’s worth, taught either by a
professor inside the computer science department (unversed in ethics) or
an outsider from the philosophy departments (ignorant of computers).
Despite two-plus decades of study by the National Science Foundation on
the marked inadequacy of ethics instruction in the field, ethics and
humanities education in computer science departments has scarcely
improved, even as the tech industry has swallowed an ever-increasing
share of the economy. Even the conservative Stanford Review, founded by
the right-wing venture capitalist Peter Thiel when he was a student,
last year complained that the university’s ethical instruction for
computer science students was “insufficient.”
“Computer science” is something more pernicious than a non-science—it is
an outright enemy of scientific reasoning.
While the plaint about missing ethics curricula may seem like so much
humanist caviling, it actually highlights a deep and abiding flaw in the
conception of computer science. In practical terms, the omission of such
instruction, or any other form of reflexive self-criticism within the
field, means that mercenary military contractors funded the creation of
a pedagogy without ethics, which supplied the labor for a tech industry
without ethics, which powered the rise of state-sanctioned monopoly tech
corporations that exercise unprecedented control over global markets and
have intrusive access to all the digitized data of our lives. In a
nightmare fulfillment of Fein’s original vision, the dogmas of this
industry, branded as computer science, infect everything—even the
proposed solutions to problems created by the industry.
Garbage In
Which opens, in turn, to an awkward question: What if the problems with
“computer science” aren’t fixable? What if the real problem is that the
field never deserved the respect it has obtained—or, more precisely,
purchased? What if the early academic skeptics of “computer science,”
who considered these devices to be mere tools—people like Weizenbaum,
and his MIT colleague Norbert Wiener, a math professor who dismissed the
computer obsessives as incurious “gadget worshippers”—were correct? In
retrospect, it’s clear that their objections were never answered in any
substantive fashion, but merely overruled by profit-minded administrators.
It’s now painfully clear that computer science is not actually a
science, by the simplest definition of that word—a method of obtaining,
organizing, and analyzing knowledge about the universe. Granted,
computers may assist with the tasks of obtaining, organizing, and
analyzing. But “computer science” as a specialized field of
gadget-enabled inquiry is not concerned with the natural universe—it is,
rather, engaged in exploring an entirely fabricated universe that exists
inside the computer. By virtue of their influence in society as tycoons
and technocrats, the computer scientists demand that we must adapt to
fit their models.
Defenders of the field maintain that this myopic concentration of
collective effort is a feature, not a bug—and, what’s more, that
anything that exists outside the machine can be input and modeled inside
it. In this respect, the gadget worshippers again invite comparison with
their dismal cousins, the classical economists. Both disciplines draw
conclusions from fabricated simulacra, models based on how they imagine
things ought to work—rather than through patient, ongoing observation of
how they actually do work. The universe is alive, but every computer
algorithm is dead on arrival.
Strip away the nomenclature of cybernetic systems theory and software
design, and you have something very close to the plot prospectus for a
Philip K. Dick novel.
In sizing up the gross cognitive deficiencies of computer science,
Weizenbaum went even further, noting that every computer system “permits
the asking of only certain kinds of questions” and “accepts only certain
kinds of ‘data.’” To create a computer program is not to enhance one’s
mental abilities, as boosters like Fein claimed, but rather to restrict
one’s options to a set of (always biased and often mistaken)
assumptions. “A computing system has effectively closed many doors that
were open before it was installed,” Weizenbaum wrote.
Because it is devoted to the creation of systems that limit choice,
“computer science” is something more pernicious than a non-science—it is
an outright enemy of scientific reasoning. As digitization has polluted
our conception of reality by shifting our focus to inferior models, it
has crippled our imaginations by restricting what we consider legitimate
“input”: if it’s not online, it doesn’t exist.
Worse, as Weizenbaum noted, the sprawling complexity of any computer
system meant that it “cannot even in principle be understood by those
who rely on it.” How could true scientists put all their trust in tools
they cannot explain?
Thinking Like a Data State
The uninterrogated premises of computer science have now worked, as
critics like Weizenbaum foresaw, to concentrate lethal quantities of
social and military power in the hands of dubiously accountable agents
of the security state and the neoliberal political economy.
Now applied computer science concerns itself with the technological
refinement of the police state, otherwise known as “cybersecurity.” In
civilian commercial applications, the focus is much as Fein foresaw—with
computer-powered startups concerned chiefly with the pillaging of labor.
There’s little need for robot Pinkertons, because the bright minds of
Silicon Valley have done their part to ensure that workers are so
thoroughly atomized by the “gig economy” that organizing to make
collective demands has become almost unimaginable.
And in a development that an arch-disruptor like Fein would have
relished, the all-consuming supradiscipline of synnoetics has even begun
to nibble at the belly of the universities that spawned it. Why bother
funding traditional universities, with vestigial departments promoting
obsolete subjects, when schools could be structured for the sole purpose
of teaching people how to work computers? Hence the recent spread of
for-profit, unaccredited, learn-to-code “boot camps,” where dislocated
workers trade the skills of their former trades and crafts for a
brighter future pushing buttons. There are at least ninety-five
companies running such coding boot camps around the country, graduating
nearly 23,000 students last year and charging an average of $11,400 in
tuition fees for a typical fourteen-week course, according to Course
Report, a startup that tracks and promotes the fast-growing industry.
Many boot camps are pitched as socially beneficial worker-retraining
programs. They’ve been so ineffective in that regard that at least one
coding boot camp, targeting unemployed miners in Appalachia, has
inspired a class-action lawsuit on the grounds that students were
inadequately trained and didn’t receive their promised stipends. What’s
more, the complaint reportedly alleges that not a single student found
work in a tech job, although placement was “guaranteed.”
But even when such programs meet their promises, the enterprise remains
a dubious one. Students pay tuition in order to learn how to write
software that will one day take over their own jobs, without being
taught to question why. Obedience is simply baked into the coders’
curriculum. Indeed, the very name of these courses—“boot camps”—recalls
the martial origins of the industry.
The Great Dictators
Computer science education inevitably promotes authoritarianism. In the
best-case scenario, graduates of these programs will go on to toil as
“code monkeys” in the most despotic corners of capitalism—weapons
manufacturing, robotics and AI, and finance. On a deeper level, they
will absorb the innately authoritarian assumptions of the field. The
binary worldview, with no tolerance for ambiguity, has created some
disturbing mental excretions. Last year a college instructor in Boston
shared with me the following note, written by a student, and apparently
lost or discarded. The context of its creation is a mystery, but
nevertheless, it represents a grim snapshot of our intellectual moment.
• Authoritarian leaders would be more effective for technology,
engineering, and more scientific related companies because those are the
kinds of jobs you are either right or wrong and involve the most
centered and determined employees.
• Democratic leaders would fit inn [sic] in a more political and social
environment because in this industry decisions affect more people and
are better made with the opinions of a group of people.
Of course, it’s not only students who’ve intuited the integral
connection between technology and authoritarianism, while suggesting
that the former justifies the latter. Sam Altman, the reliably pompous
president of the tech venture capital fund Y Combinator, went so far as
to praise the ancillary benefits to “innovation” of China’s notoriously
restrictive, censorship-addicted one-party system in a December blog
post. “I realized I felt more comfortable discussing controversial ideas
in Beijing than in San Francisco,” Altman wrote. “That showed me just
how bad things have become” back home. Bad for who, though? Altman
lamented that “credible people” in his circles had left the Bay Area
because “they found the reaction to their work to be so toxic.” Not so
in China! Techies there, freed from American-style “political
correctness,” may gleefully explore such “heresies” as “pharmaceuticals
for intelligence augmentation, genetic engineering, and radical life
extension,” Altman gushed. Funny, he never mentioned the Chinese
government’s Great Firewall—the world’s most comprehensive and effective
system of internet censorship—or the tens of thousands of
dissent-crushing online speech monitors it employs, or the policy of
re-education through labor, which once upon a time would’ve sent
quasi-libertarian tycoons like Altman into spasms of indignation. But
hey, the Chinese government lets scientists clone primates, so who cares
about the political prisoners? How quickly the techies’ righteous esteem
for freedom of thought vanishes when the powers that be promise them new
toys to play with.
This political tendency is one that dates to the early days of the
field. In his most important book, Computer Power and Human Reason,
published in 1976, Weizenbaum depicts the computer as a reactionary
device. Since its invention, he wrote, the computer “was used to
conserve America’s social and political institutions. It buttressed them
and immunized them, at least temporarily, against enormous pressures for
change.”
The conventional wisdom in his field held that society relied upon
computers to solve increasingly complex problems created by burgeoning
populations and new technologies—especially nuclear weapons. It was said
that computers had arrived “just in time” to help capitalist society
cope with rapidly increasing complexity. “Yes, the computer did arrive
‘just in time,’” Weizenbaum wrote. “But in time for what? In time to
save—and save very nearly intact, indeed, to entrench and
stabilize—social and political structures that otherwise might have been
either radically renovated or allowed to totter under the demands that
were sure to be made on them.”
Weizenbaum believed computers were standing in the way of necessary
revolution. He grew disgusted by his colleagues’ amoral servility before
power. And he was unwilling to let them off the hook for enabling
monstrous abuses by the powers that be, especially warmongers like
Robert McNamara, who carpet-bombed Southeast Asian peasants with
statistical perfection. “The scientist and the technologist can no
longer avoid the responsibility for what he does,” Weizenbaum wrote.
Weizenbaum was also among the first thinkers in the field to recognize
that code was ideology. He saw computers as the natural product of an
imperialistic process that had corrupted and “reduced reason itself to
only its role in the domination of things, man, and, finally, nature.”
In this flattened-out world of instrumental reason, every stroke of the
keyboard is an offering to the war machine, and every swipe of the
touchscreen is a little prayer of thanks to the Pentagon, which made it
all possible.
Meanwhile, as Weizenbaum observed, computers served as dispensers of
moral indulgences for powerful decision-makers. “The computer, as
presently used by the technological elite, is not a cause of anything.
It is rather an instrument pressed into the service of rationalizing,
supporting, and sustaining the most conservative, indeed, reactionary,
ideological components of the current Zeitgeist,” he wrote.
Computerization meant that no one had any incentive to take
responsibility for difficult decisions—and, by the same token, no one
could be held accountable for bad ones. Sound familiar?
Toggling Toward Bethlehem
In the course of researching this story, I slogged through several
hundred pages of federal technical manuals and how-to guides for states
and localities interested in adopting the emergency management system
created by President Bush’s executive order in 2006. As with any
computer-driven process, the most important choices have already been
made. The first order of business on one federal to-do list for local
authorities is to go shopping—rather, to “select IPAWS compatible
software.” As it happens, the feds publish a list of pre-approved
vendors with off-the-shelf solutions, and the importance of “private
sector partners” is frequently stressed in official materials. “It is
clearly in the national interest to ensure private sector
participation,” notes FEMA’s June 2010 Strategic Plan for the Integrated
Public Alert and Warning System (IPAWS) Program. Incidentally, it turns
out, FEMA will not provide technical support. Instead, local authorities
are advised to “contact your vendor.”
The profit-fueled fetish for “digital learning” has coincided with the
chauvinist denigration of the humanities and social sciences. Computer
skills have become synonymous with talent and ingenuity.
Which is exactly what Hawaii state officials did after the false missile
alarm of January 13. They were quick to publicize a short list of fast
fixes, none of which were “ditch this stupid software and go back to
using the telephone.” In the state’s final investigative report on the
matter, conducted by a retired brigadier general in the Hawaii National
Guard and released January 29, the employee who sent the alarm,
initially described as an experienced veteran—who would not lose his or
her job over the incident—was recast as a longstanding “source of
concern” who had more than once seemed “confused” when it came to
distinguishing drills and real emergencies. The employee was fired after
all, and the agency administrator, Miyagi, also resigned. Heads rolled.
Problem solved?
Until the next false alarm, we must reckon with the knowledge that the
fragile, clumsy, harebrained system put in place to alert the public of
impending nuclear disaster closely mirrors the antiquated, harebrained
system that will be used to create a nuclear disaster. Thanks, computer
science! The discipline has given us a system that can only ask, “Shall
we launch the missiles now? Or shall we merely pretend to launch some
missiles?” For all the trillions of dollars poured into military
research and Silicon Valley solutions over the past seven decades, not a
single member of that gadget-worshipping cabal has yet given the
president a “peace” button to push.
That’s because peace, like every important problem we still face as a
species, exceeds the conceptual scope of the servile tinkerers and their
phony “science.” Louis Fein, the scapegoat of this story, disagreed, of
course. In his defense, he seems to have meant well. “What the hell are
we making these machines for, if not to free people?” he told a Time
magazine reporter in 1965. The question came perhaps a little too late.
Programming for Peace
In 1963, a full year after the Cuban missile crisis, Fein published
still another paper on what he saw as the boundless potential of
computer science. In it, he proposed a six-phase program “on the
prevention of nuclear war and the establishment of the basis for future
peace on Earth.” True, war may have plagued every previous generation of
humanity. But those people didn’t have computers—or the next best thing,
computer consultants. “Imagine,” Fein began, “a management consulting
and research type of organization called, say, the Universal Study
Center for the Salvage and Reorganization of Institutions in Imminent
Danger of Destruction Applying Computers Wherever Feasible (USCSRIIDDACWF).”
Perhaps humans should not be trusted with something so important as
crafting the warning system intended to safeguard the future of the species.
Inputs might include Christian teachings about universal love, as well
as (pre-USSR) Marxist doctrines. Upon crunching the relevant data, Fein
wrote, USCSRIIDDACWF’s computers would prescribe “an optimum Earth
reorganization”—revolution at the push of a button.
If only his computers were programmed correctly, and supplied with the
right sort of information, Fein imagined that they would produce
invaluable “strategies and tactics” for the shift to a “democratic
socialist society where the Augustinian slogan ‘from each according to
his ability and to each according to his need’ would be the guiding
policy for the prevention of war and the establishment of peace and
prosperity.”
Fein understood that opposition to this program would be
“all-pervasive.” He conceded that “obtaining moral and financial support
. . . may be extremely difficult.” Maybe, he thought, the United Nations
would help? Or perhaps the March of Dimes could be used as a fundraising
model?
At last the great intellectual forefather of computer science dared to
venture beyond the binary scheme of cognitive deference to synnoetics
and pre-filtered menu choices, into the world that humans actually
inhabit. And tellingly, he could only imagine funding and defending the
effort, not from the largesse of the Cold War national security state,
but via a door-to-door philanthropic appeal. His goals may have been
noble, but how impoverished his imagination had grown by overexposure to
that computerized milieu.
Unwittingly, Fein furnished a sort of parable for the digital age: the
regime of maximally programmed deference to authority can’t magically
remedy, by user command, all the many social pathologies that it has
conspired to create. The problem is, no one’s about to learn that in any
computer-science class.
# distributed via <nettime>: no commercial use without permission
# <nettime> is a moderated mailing list for net criticism,
# collaborative text filtering and cultural politics of the nets
# more info: http://mx.kein.org/mailman/listinfo/nettime-l
# archive: http://www.nettime.org contact: [email protected]
# @nettime_bot tweets mail w/ sender unless #ANON is in Subject:
