The Oracle Effect
http://thedailywtf.com/articles/the-oracle-effect
Even simple rituals can feed into this Oracle Effect. For
example, PayPal doesn't want to handle transactions for ISIS,
which isn't unreasonable, but how do you detect which
transactions are made by honest citizens, and which by
militants? What about just blocking transactions containing
the letters "isis"? This seems like a pretty simple algorithm,
but think about the amount of data flowing through it, and
suddenly, it picks up the air of ritual- we have a magic
incantation that keeps us from processing transactions for
militants. Using algorithms and decision-support systems
isn't bad. It's not even bad if they're complicated! They're
solving a complicated problem, and we'd expect the resulting
system to reflect at least some of that complexity. A recent
conference hosted at NYU Law spent time discussing how we
could actually avoid biases in policing by using well-designed
algorithms, despite also pointing out the risks and dangers to
human rights. These sorts of decision-making tools can make
things better- or worse. They're just a tool.
- - -
--Lauren--
Lauren Weinstein ([email protected]): http://www.vortex.com/lauren
Founder:
- Network Neutrality Squad: http://www.nnsquad.org
- PRIVACY Forum: http://www.vortex.com/privacy-info
Co-Founder: People For Internet Responsibility: http://www.pfir.org/pfir-info
Member: ACM Committee on Computers and Public Policy
Lauren's Blog: http://lauren.vortex.com
Google+: http://google.com/+LaurenWeinstein
Twitter: http://twitter.com/laurenweinstein
Tel: +1 (818) 225-2800 / Skype: vortex.com
I have consulted to Google, but I am not currently
doing so -- my opinions expressed here are mine alone.
_______________________________________________
pfir mailing list
http://lists.pfir.org/mailman/listinfo/pfir
