UN conference in Geneva to discuss restrictions and bans on AI-powered autonomous weapons
Nation15 November, 01:39 AM
Lethal autonomous weapons systems (LAWS) "are capable of identifying, selecting and engaging targets without human operator intervention in carrying out these tasks," Jean-Marc Rickli, head of global and emerging risks at the Geneva Centre for Security Policy, told AFP.
The article notes that autonomous weapons, such as anti-personnel mines, have existed for decades. However, with the proliferation of artificial intelligence-equipped drones, such weapons can now "select and engage targets that have not been predetermined, adapting to their environment without human involvement," Rickli said.
A key question for potential regulators is determining how much human control should be required over decisions to use such weapons. The United States calls for an "appropriate level of human judgment" in the use of autonomous weapons, while France insists that humans must also be involved in decision-making. France prohibits lethal autonomous weapons systems (LAWS) but allows the use of partially autonomous weapons systems (PLAWS).
"The human role is always ensured through what is known as appropriate control by a military commander, a concept whose scope varies depending on the criticality of the system and the operational context," said Rear Adm. Vincent Sébastien, deputy director of the French Defense Ministry's Artificial Intelligence Agency for Defense (AMIAD).
"We are now dangerously close to crossing a moral red line — the autonomous selection of human targets by machines," U.N. Secretary-General António Guterres and International Committee of the Red Cross President Mirjana Spoljaric said in a joint statement in August.
Rickli noted that the issues being discussed in Geneva were "very narrow." They do not cover the full range of challenges associated with the use of AI on the battlefield, where its role continues to expand, including in compiling target lists, he explained.
According to Rickli, this was evident during U.S.-Israeli operations against Iran earlier this year, "when the Americans were processing up to 1,000 targets a day" inside Iran.
"There is a whole range of problems and unacceptable risks associated with targeting and decision-support systems. Part of the debate concerns systems that are already widely used on the battlefield," said Laure de Roucy-Rochegonde of the Center for Geopolitics of Technology at the French Institute of International Relations (IFRI).
"What level of human control is actually necessary to ensure that the operator is genuinely exercising independent judgment rather than simply rubber-stamping the machine's recommendations?" she asked.
The autonomous weapons being discussed in Geneva have not yet become widespread, although analysts note that Russia has already used them during its invasion of Ukraine and the United States has deployed them in operations against Iran. AFP reported that their proliferation appears inevitable.
"Autonomous weapons systems are not a distant future. The race to increase autonomy in weapons systems is already well underway," Guterres and Spoljaric said.
According to Rickli, swarms of autonomous drones tasked with neutralizing an enemy by overwhelming its defenses would be extremely effective, and some countries view them as a decisive strategic advantage.
In January, China's Defense Ministry described autonomous drone swarms as "the new face of warfare."
"In an era of intense military competition, the state that first makes fully autonomous lethal strikes routine will reshape perceptions of acceptable norms for everyone else," Gerald Macko, a research fellow at the University of Cambridge, told Small Wars Journal.
Meanwhile, Dario Amodei, CEO of Anthropic, one of the leading AI companies, said earlier this year that serious concerns and risks remain.
"Today's frontier AI systems are simply not reliable enough to control fully autonomous weapons," he said.