Florian Seibel has seen the future of war — and it has robots in it. He now wants to convince Ukraine — and Europe — that they need to embrace autonomous weapons.
A former helicopter pilot in the German army, 44-year-old Seibel is the chief executive of Quantum Systems, a Munich-based company that boasts Silicon Valley wizard Peter Thiel among its investors. The company builds reconnaissance drones used by Ukraine to fend off Russia’s invasion.
The numerical disparity between democratic countries’ armies and those of China and Russia, he said, will make reliance on robots necessary “to cope with our reduced numbers of fighters.”
He has started a new company called Stark Defense to focus on attack drones that can operate without human control. While he said the company was named after Arya Stark, a heroine from the high-fantasy book series “A Song of Ice and Fire” and its TV adaptation “Game of Thrones,” it also harks back to the Marvel comics’ maverick industrialist superhero, Tony Stark.
Stark’s mission, Seibel told POLITICO last month, is to “disrupt the defense industry, and build autonomous systems, on land, in the water, on the surface” all with strike capabilities.
Stark’s launch happens at a crucial juncture in the history of autonomous weapons. With geopolitical tensions reaching a fever pitch, Ukraine’s deft use of drones grabbing headlines, and artificial intelligence technology growing white hot, military powers including the United States, China and Russia are eagerly looking into building up autonomous capabilities.
Drone warfare loses its edge
Some 400 Quantum Systems’ unmanned aerial vehicles are already flying in Ukraine, and 800 more will be delivered in August as per a deal cut during a mission organized in April by Germany’s Vice Chancellor Robert Habeck. The company is also opening a factory in Ukraine.
Russia’s development of jamming techniques capable of preventing unmanned vehicles from orienting themselves or communicating with pilots has convinced Seibel that the remotely-controlled drones Ukraine long relied upon are now losing their edge.
AI, the booming technology able to spot patterns and make decisions, has provided a lifeline. Thanks to AI — and Nvidia chips — Quantum System’s drones can negotiate the battlefield and spot targets without depending on GPS or a human steerer. They can then fly themselves to a jamming-free location, and send back suggested Russian military targets for artillery strikes.
The final decision still rests with Ukrainian soldiers, following a model known as “human-in-the-loop,” meaning that decisions to kill should be taken by humans and not machines.
But Seibel thinks that norm will have to change. “If they cannot communicate back,” he said, “drones will have to make the next step in evolution: Identifying targets for themselves, classifying them as friends or foes, and in case of adversity — hit them.”
In his opinion, the cost of not developing weapons able to do that — to manage the whole kill chain, from target selection to strike — is to start the next war as the underdogs. “If we don’t want our kids to fight Chinese war robots in the future, we have to get going and work on war robots ourselves,” Seibel said.
Banning combat robots
At the same time, diplomats, activists and technologists want to curb the technology, concerned it would precipitate AI-driven war escalation, push-button mass killings, friendly fire catastrophes, and morally repugnant death-by-algorithm.
The Austrian government is spearheading efforts to draft an international treaty that would ban autonomous weapons whose actions cannot be controlled or predicted by their human masters.
“We don’t need weapons manufacturers claiming that unpredictable and unethical weapons will make Europe safer,” Anna Hehir, head of the autonomous weapons program at the Future of Life Institute think tank, told POLITICO.
“There was a period in history when major military powers thought they needed biological weapons to counter their adversaries’ own biological weapons programs. We need to maintain rational foresight when evaluating new technologies.”
Seibel said that autonomous weapons, if correctly designed, can reduce collateral damage by harnessing AI’s superior analytical capabilities: An AI-powered drone would be able to tell friend from foe, or civilian from combatant, with higher accuracy.
“These discussions are similar to those about autonomous driving: human nature is skeptical, and if something is new, it must be bad,” he said. “I think the opposite is true: Autonomous weapons can make much smarter decisions.”
Hehir said that if military AI reached superhuman precision, that would call for even stronger checks on the technology to avoid those capabilities being used for “targeted killings” based on specific characteristics.
Making the case
Stark is still recruiting and seeking venture capital investment, but Seibel said it would be able to deploy its products on short notice if necessary — especially if Ukraine asks for them. In the meantime, he is making his case to the top brass.
“Tonight in Berlin we’re talking again to General [Carsten] Breuer, the chief of Germany’s armed forces. We’re telling them that the next war will be all about drone warfare,” Seibel told POLITICO in May. “They’re starting to act, but very slowly. To be honest, acting has not started yet.”
“These systems are in, the end, capable of operating without a human in the loop. But that’s not my decision: we will prepare the grounds so that it is possible,” he said. “But if the decision of the German government is that we cannot have autonomous weapons with no ‘human-in-the-loop,’ then they won’t be used.”
