David Reed brings up the question:
> If the algorithm discriminates against the conversations of (say)
> black people, is it protected speech, or a violation of civil rights?
There have been reports (I've never seen a specific citation, I'm afraid) that
a bank used machine learning to build a decision engine for personal loans.
They had to scrap it because it turned out that it gave a large negative weight
to black applicants. I've heard stories about similar systems for ranking
resumes that were found to give a large positive weight to being unmarried.
There are some very difficult issues coming down the road. Opinions in and of
themselves may be protected, but when those opinions are embedded in a larger
system and have real world consequences, what deserves protection has to be
specified very carefully. While truth is an absolute defense against libel in
the US, a proof that a rating system that discriminated against black loan
applicants or married job applicants actually achieved some acceptable goal
(decreasing loss rates, longer work hours with more output) would not render
the system legally acceptable in the US. Personally, I wouldn't find it
ethically acceptable either.
So ... when do we accept an argument that "it's just what the algorithms came
up with"? We have many centuries of experience with bureaucracies - which
provide exactly this kind of insulation for each individual. "I'm just
following the procedures." There are no absolute rules. If we demand that
every member of an organization be personally responsible for every decision or
action that organization undertakes, businesses and governments become
impossible. But if we go 100% the other way, "It wasn't my job" becomes an
absolute defense. Computers are a much better embodiment of the basic drive of
bureaucracies: The conversion of individual initiative to uniform policies,
uniformly applied. "Government of laws, not men."
-- Jerry
_______________________________________________
nnsquad mailing list
http://lists.nnsquad.org/mailman/listinfo/nnsquad
