----- Forwarded message from Gregory Foster <[email protected]> -----
From: Gregory Foster <[email protected]> Date: Mon, 26 Nov 2012 09:26:21 -0600 To: [email protected] Subject: [drone-list] US DoD Directive on Autonomy User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 Secrecy News (Nov 26) - "Autonomy in Weapon Systems": http://www.fas.org/blog/secrecy/2012/11/weapons_autonomy.html One note. Mention is made in this document about the importance of "verification and validation" (V&V) of systems. Earlier this month, DARPA's High-Assurance Cyber Military Systems (HACMS) program awarded a 4.5 year $18M contract to an international consortium "to develop a complete, formally proven architecture to protect the control and communication systems of an aerial vehicle from compromise by faults and targeted attacks." http://www.nicta.com.au/media/current/nicta_to_develop_critical_software_for_multi-million-dollar_us_government_cyber-security_project Part of that award went to National ICT Australia [ @NICTA ] which toots its own horn for formally verifying 7,500 lines of C code. That tells you something about the challenge of "formal verification" which mathematically proves a system will behave as specified - a much higher bar than "rigorous" testing. http://en.wikipedia.org/wiki/Formal_verification That also tells you something about the complexity of systems which have thus far achieved formal verification. For comparison, the Linux kernel---which sits at the heart of many of the workstations used by the US military's remote pilots---surpassed 15 million lines of code in January, leading the creator of Linux, Linus Torvalds, to lament that he is "afraid of the day" when there will be an error that "cannot be evaluated anymore." http://www.tomshardware.com/news/Linux-Linus-Torvalds-kernel-too-complex-code,14495.html Considering DARPA knows the path to formal verification of larger systems must be cleared - demand it. The challenge of formal verification presents a strong argument for delaying deployment of lethal autonomous systems. gf -- Gregory Foster || [email protected] @gregoryfoster <> http://entersection.com/ _______________________________________________ drone-list mailing list [email protected] Should you need to change your subscription options, please go to: https://mailman.stanford.edu/mailman/listinfo/drone-list If you would like to receive a daily digest, click "yes" (once you click above) next to "would you like to receive list mail batched in a daily digest?" You will need the user name and password you receive from the list moderator in monthly reminders. Should you need immediate assistance, please contact the list moderator. ----- End forwarded message ----- -- Eugen* Leitl <a href="http://leitl.org";>leitl</a> http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE -- Unsubscribe, change to digest, or change password at: https://mailman.stanford.edu/mailman/listinfo/liberationtech
