SECRECY NEWS
from the FAS Project on Government Secrecy
Volume 2017, Issue No. 4
January 11, 2017

Secrecy News Blog: https://fas.org/blogs/secrecy/

ARTIFICIAL INTELLIGENCE "JOLTED" BY SUCCESS

Since 2010, the field of artificial intelligence (AI) has been "jolted" by the "broad and unforeseen successes" of one of its component technologies, known as multi-layer neural networks, leading to rapid developments and new applications, according to a new study from the JASON scientific advisory panel.

The JASON panel reviewed the current state of AI research and its potential use by the Department of Defense. See Perspectives on Research in Artificial Intelligence and Artificial General Intelligence Relevant to DoD, JSR-16-Task-003, January 2017.

"AI technologies are of great importance to DoD missions. Defense systems and platforms with varying degrees of autonomy already exist. More importantly, AI is seen as the key enabling technology (along with human-computer interactions of various kinds) of a 'Third Offset Strategy' that seeks for the U.S. a unique, asymmetric advantage over near-peer adversaries."

The JASON report distinguishes between artificial intelligence -- referring to the ability of computers to perform particular tasks that humans do with their brains -- and artificial *general* intelligence (AGI) -- meaning a human-like ability to pursue long-term goals and exercise purposive behavior.

"Where AI is oriented around specific tasks, AGI seeks general cognitive abilities." Recent progress in AI has not been matched by comparable advances in AGI. Sentient machines, let alone a revolt of robots revolt against their creators, are still somewhere far over the horizon, and may be permanently in the realm of fiction.

While many existing DoD weapon systems "have some degree of 'autonomy' relying on the technologies of AI, they are in no sense a step--not even a small step--towards 'autonomy' in the sense of AGI, that is, the ability to set independent goals or intent," the JASONs said.

"Indeed, the word 'autonomy' conflates two quite different meanings, one relating to 'freedom of will or action' (like humans, or as in AGI), and the other the much more prosaic ability to act in accordance with a possibly complex rule set based on possibly complex sensor input, as in the word 'automatic'. In using a terminology like 'autonomous weapons', the DoD may, as an unintended consequence, enhance the public's confusion on this point."

Nevertheless, even if they are more "automated" than genuinely "autonomous," many existing applications of artificial intelligence "have achieved performance that exceeds what humans typically do."

And while artificial intelligence can have damaging or disruptive implications in other contexts, that may not be so in the military. "Displacement of soldiers' jobs by technology is a benefit, not an economic or social harm," the JASON report said.

Yet the path forward for DoD to adopt new AI applications is not simple or smooth. A major obstacle is the difficulty of validating those applications to establish their safety and reliability.

Current progress in AI "has not systematically addressed the engineering 'ilities': reliability, maintainability, debug-ability, evolvability, fragility, attackability, and so forth. Further, it is not clear that the existing AI paradigm is immediately amenable to any sort of software engineering validation and verification. This is a serious issue, and is a potential roadblock to DoD's use of these modern AI systems, especially when considering the liability and accountability of using AI in lethal systems."

Deep neural networks function in a manner which is "almost unknowably intricate, leading to failure modes for which -- currently -- there is very little human intuition, and even less established engineering practice."

A recent report from the Defense Science Board on Unmanned Undersea Systems observed that "The ability to communicate periodically can reduce some of the technological requirements and risks associated with the autonomous subsystem that often slows the development and subsequent fielding of systems." See Next-Generation Unmanned Undersea Systems, October 2016:

The JASONs recommended continued DoD investment in basic research on artificial intelligence especially since, in all likelihood, "AI, both commercially derived and DoD-specific, will be integral to most future DoD systems and platforms."

This week the Department of Defense announced the demonstration of swarms of "autonomous" micro-drones. "The micro-drones demonstrated advanced swarm behaviors such as collective decision-making, adaptive formation flying, and self-healing," according to a January 9 news release.

A journalistic account of recent breakthroughs in the use of artificial intelligence for machine translation appeared in the New York Times Magazine last month. See "The Great A.I. Awakening" by Gideon Lewis-Kraus, December 14, 2016.

IMPROVING DECLASSIFICATION: NOT YET

A new report on improving declassification procedures in the U.S. intelligence community implicitly suggests that no such improvements are likely to emerge any time soon.

The report, published yesterday by the Office of the Director of National Intelligence in response to congressional direction, is largely devoid of new ideas and instead calls for greater "integration" and "coordination."

"Improving the declassification process across the Community will require an integrated and multifaceted set of initiatives fully coordinated with organizations that have AD [automatic declassification] programs. No single step will suffice in addressing satisfactorily the areas for improvement that have been identified in this report."

See Improving the Intelligence Community's Declassification Process and the Community's Support to the National Declassification Center, ODNI, December 2016:

The core of the report is in a section entitled "Proposals to Improve the IC's Declassification Process." But it does not actually present any declassification policy proposals. Instead, in a near-parody of a government report, it calls for establishment of new working groups to write other reports and generate further recommendations.

Thus, the "Proposal on Process" calls for "a Declassification Improvement Working Group (DIWG) to conduct a zero-based study of the IC's AD process and prepare a report--by a specified deadline-- that includes recommended actions to increase the program's effectiveness and efficiency across the Community."

The "Proposal on Electronic Records" says that "A joint task force [...] should be formed to aggressively pursue the identification, development, and validation of technological capabilities -- tools and infrastructure -- for incorporation in the AD [automatic declassification] process." (Aggressively!)

This is not helpful. In fact, it is practically a declaration of helplessness.

The new report is lacking in specific actionable proposals that could be evaluated, debated, perhaps modified and ultimately adopted in practice. It does not ask or answer any penetrating questions. Such as:

What if agency "equity" in older records, requiring review by those agencies, simply lapsed at some point in time, eliminating the need for such review?

What if certain defunct intelligence compartments could be altogether excused from multi-agency referral and review?

What if a fixed fraction of agency information security expenditures were routinely and predictably allocated to performing declassification?

What if new metrics could be devised to measure the success of declassification programs based on requester demand and disclosure impact, not just on number of pages processed?

Fundamentally, what if intelligence community tolerance for risk were recalibrated to facilitate more expeditious declassification of both current and historically valuable records?

Interestingly, the report notes that agencies favor numerous revisions to President Obama's executive order 13526 on classification policy, so that "updating the E.O. will be a major undertaking."

But those revisions mainly seem geared toward relaxing existing declassification requirements, not strengthening them. So, for example, IC officials believe they could place increased emphasis on declassifying historical records of broad public interest if they could be assured that other records of lesser interest would not be automatically declassified as they become 25 years old, as the Obama order nominally directs.

******************************

Secrecy News is written by Steven Aftergood and published by the Federation of American Scientists.

The Secrecy News blog is at:
      https://fas.org/blogs/secrecy/

To SUBSCRIBE to Secrecy News, go to:
     https://fas.org/sgp/news/secrecy/subscribe.html

To UNSUBSCRIBE, go to:
      https://fas.org/sgp/news/secrecy/unsubscribe.html

OR email your request to saftergood@fas.org

Secrecy News is archived at:
      https://fas.org/sgp/news/secrecy/index.html

SUPPORT the FAS Project on Government Secrecy with a donation here:
      https://fas.org/donate/