Report

Towards Responsible Autonomy - The Ethics of RAS in a Military Context

September 12th 2019 - 10:56

The accelerating arms race in Artificial Intelligence and the diffusion of cheap, technologically advanced military systems among state and non-state actors, compel countries to adopt robotic and autonomous systems (RAS). This is due to not only the prospect of lagging behind allies, but the notion that adversaries use RAS to gain a significant military advantage and escalation dominance. How can countries stay competitive, without violating core national and international ethical principles?

This paper discusses the key ethical challenges spanning human agency, human dignity and responsibility in the operation of RAS. It presents a three-part framework for how to view human control over RAS in an ethical manner:

  • Life cycle perspective
  • Sub-system functionality perspective
  • Observe-orient-decide-act (OODA) loop perspective

Based on the framework above, the paper presents the following concrete recommendations for actors engaging in the design, development, testing, operation and decommissioning of RAS at the strategic and operational levels:

  • The fundamental principle is to work with ‘ethics by design’: ethical considerations are incorporated in not just the operating of RAS, but all throughout a system’s life cycle;
  • Create an institutional culture of shared accountability concerning all actors involved throughout the RAS life cycle;
  • Build a wider understanding of system behavior by involving end users as early as in the design and testing stages. This way, the users within the military can trace, understand and predict a system’s decision-making process;
  • Develop best practice guidelines for (1) outsourcing to external contractors; and (2) interoperability with technologically advanced allied armed forces that co-deploy RAS;
  • Identify within what sub-system functions of RAS increasing automation and autonomy will present benefits to the military without eliciting major ethical concerns, such as movement controls and computer vision;
  • Embed core rules of engagement (ROEs), along with an open architecture that allows for mission-specific ROEs by mission command;
  • Improve transparency on the uses and contexts of use of RAS in the military domain with the general public and inform policymakers of realistic opportunities and limitations of complex system design.

Dowload the report here.

 

Disclaimer: Responsibility for the content and for the opinions expressed rests solely with the authors. Publication does not constitute an endorsement by the Netherlands Ministry of Defence.

Patrick Bolder is an officer in the Royal Netherlands Air Force. Since January 2019 he is seconded to The Hague Centre for Strategic Studies. His expertise lies in all Defence matters, with a focus on Military Space, Unmanned systems, European Defence issues and Nuclear Policy.
Bianca Torossian is a Strategic Analyst at HCSS. She contributes to a range of projects commissioned by the Dutch Ministry of Foreign Affairs, the Dutch Ministry of Defence and NATO. She primarily focusses on the intersection between emerging technologies and conflict, with multiple publications to her name on hybrid and cyber conflict, and is also heavily involved in the Water, Peace and Security (WPS) partnership.
Michel Rademaker is the deputy Director of HCSS. He has fifteen years of hands-on experience as an officer in The Royal Netherlands Army, where he held various military operational and staff posts and also served a term in former Yugoslavia. He has a masters degree obtained at the University of Tilburg. After leaving the armed forces, Mr. Rademaker went on to work at the Netherlands Organisation for Applied Scientific Research (TNO) as a project and program manager and senior policy advisor on security topics for ten years. Eg. as NATO RTO project leader, he and his team developed serious gaming assessment methods and conducted several assessments of security technologies, and worked on numerous strategic security topics.
Frank Bekkers is Director of the Security Program. He studied Applied Mathematics at the University of Amsterdam and spent most of his career at the Netherlands Organisation for Applied Scientific Research (TNO), specializing in the area of Defence, Safety & Security. At TNO, he held a range of positions, including program manager, senior research scientist, group manager and account director. From 1996-1997, he worked as program manager for Call Media and Intelligent Networks for the telecom company KPN. His current position at HCSS combines shaping HCSS’s portfolio concerning defense and security-related projects with hands-on participation in a number of key projects.