Towards Responsible Autonomy - The Ethics of RAS in a Military Context

September 12th 2019 - 10:56

The accelerating arms race in Artificial Intelligence and the diffusion of cheap, technologically advanced military systems among state and non-state actors, compel countries to adopt robotic and autonomous systems (RAS). This is due to not only the prospect of lagging behind allies, but the notion that adversaries use RAS to gain a significant military advantage and escalation dominance. How can countries stay competitive, without violating core national and international ethical principles?

This paper discusses the key ethical challenges spanning human agency, human dignity and responsibility in the operation of RAS. It presents a three-part framework for how to view human control over RAS in an ethical manner:

  • Life cycle perspective
  • Sub-system functionality perspective
  • Observe-orient-decide-act (OODA) loop perspective

Based on the framework above, the paper presents the following concrete recommendations for actors engaging in the design, development, testing, operation and decommissioning of RAS at the strategic and operational levels:

  • The fundamental principle is to work with ‘ethics by design’: ethical considerations are incorporated in not just the operating of RAS, but all throughout a system’s life cycle;
  • Create an institutional culture of shared accountability concerning all actors involved throughout the RAS life cycle;
  • Build a wider understanding of system behavior by involving end users as early as in the design and testing stages. This way, the users within the military can trace, understand and predict a system’s decision-making process;
  • Develop best practice guidelines for (1) outsourcing to external contractors; and (2) interoperability with technologically advanced allied armed forces that co-deploy RAS;
  • Identify within what sub-system functions of RAS increasing automation and autonomy will present benefits to the military without eliciting major ethical concerns, such as movement controls and computer vision;
  • Embed core rules of engagement (ROEs), along with an open architecture that allows for mission-specific ROEs by mission command;
  • Improve transparency on the uses and contexts of use of RAS in the military domain with the general public and inform policymakers of realistic opportunities and limitations of complex system design.

Dowload the report here.


Disclaimer: Responsibility for the content and for the opinions expressed rests solely with the authors. Publication does not constitute an endorsement by the Netherlands Ministry of Defence.

Amit Arkhipov-Goyal
Amit Arkhipov-Goyal is a Strategic Analyst at HCSS focusing on research at the intersection of emerging technologies, foreign policy and international security, primarily for the Netherlands Ministries of Defence and Foreign Affairs. His past work includes studies on the geopolitical implications of adoption of Artificial Intelligence in societies, economies and militaries, as well as the ethical considerations of deploying Robotic and Autonomous Systems within the armed forces. Prior to HCSS, Amit worked at the Institute for Economics and Peace in The Hague, where he focused on building institutional knowledge of conflict prevention and early warning tools. Amit holds a BSc in Economics from Erasmus University Rotterdam and is currently completing an MSc in Crisis and Security Management at Leiden University.
Esther Chavannes is a Strategic Analyst at HCSS, primarily focusing on topics where security, politics and law intersect. Most recently she conducted research into the ethical challenges of using Robotic and Autonomous Systems in a military context. Apart from emerging technologies in the Defense sector, she works on security challenges in Southeast Asia, regulatory frameworks of civil technology in the EU, and right-wing political violence in the digital age.
Patrick Bolder is an officer in the Royal Netherlands Air Force. Since January 2019 he is seconded to The Hague Centre for Strategic Studies. His expertise lies in all Defence matters, with a focus on Military Space, Unmanned systems, European Defence issues and Nuclear Policy.
Bianca Torossian is a Strategic Analyst at HCSS. Her studies at The University of Sydney, Erasmus University Rotterdam, and Leiden University (The Hague), earned her a Bachelor degree in political science, political economy and sociology, and a Master degree in political science and international organization. For her Master Thesis, Bianca analyzed how the institutional legitimacy of the European Union was impacted by Brexit, and hopes later to reopen this line of research and explore how the nationalistic tendencies of states effect the social legitimacy of multilateral institutions. At HCSS, Bianca primarily focusses on security and diplomacy. A specific area of interest is the field of technology and AI in defense contexts, which ties into a HCSS research project that critically analyzes the challenges and opportunities posed by robotic and autonomous systems in the military. She contributes to a range projects commissioned by the Dutch Ministries of Foreign Affairs and Defence.
Michel Rademaker is the deputy Director of HCSS. He has fifteen years of hands-on experience as an officer in The Royal Netherlands Army, where he held various military operational and staff posts and also served a term in former Yugoslavia. He has a masters degree obtained at the University of Tilburg. After leaving the armed forces, Mr. Rademaker went on to work at the Netherlands Organisation for Applied Scientific Research (TNO) as a project and program manager and senior policy advisor on security topics for ten years. Eg. as NATO RTO project leader, he and his team developed serious gaming assessment methods and conducted several assessments of security technologies, and worked on numerous strategic security topics.
Frank Bekkers is Director of the Security Program. He studied Applied Mathematics at the University of Amsterdam and spent most of his career at the Netherlands Organisation for Applied Scientific Research (TNO), specializing in the area of Defence, Safety & Security. At TNO, he held a range of positions, including program manager, senior research scientist, group manager and account director. From 1996-1997, he worked as program manager for Call Media and Intelligent Networks for the telecom company KPN. His current position at HCSS combines shaping HCSS’s portfolio concerning defense and security-related projects with hands-on participation in a number of key projects.