New EU regulations on AI seek to ban mass and indiscriminate surveillance. For many, that is the good news. The ‘not so good’ news is that the proposed prohibitions are considered by some as being too vague, with serious loopholes.
Most recently, the European Data Protection Board (EDPB) and European Data Protection Supervisor (EDPS), called for a ban on the use of AI for the automated recognition of human features in “publicly accessible spaces” as well as other uses that might lead to “unfair discrimination”. Broadly speaking, this reflects the response to the EU’s attempt to set a standard on how tech is regulated around the world.
Let’s first push the rewind button. In April, the European Commission unveiled its long-awaited plan to regulate AI, the first endeavour of its kind and one reason why expectations were high, both in Europe and abroad. Regulating the use of such rapidly evolving technologies is no easy task. In fact, it is one that has required years of work, study and consultations with a wide range of stakeholders.
“A general ban on the use of facial recognition in publicly accessible areas is the necessary starting point if we want to preserve our freedoms and create a human-centric legal framework for AI”
European Data Protection Board chair Andrea Jelinek
The EU is looking to set the terms with this, its first ever legislative package on AI and to catch up with the US and China in a sector that spans areas from voice recognition to insurance and law enforcement. The European Parliament has played its part, calling on the Commission to establish a legislative framework on AI as early as 2017. It has also set up its very own Special Committee on Artificial Intelligence, whose work continues.
As well as lawmakers, it is also useful to take into account public opinion on the issue. A survey conducted by YouGov in ten EU countries in March showed that a majority oppose biometric mass surveillance.
Some 55 percent of respondents voiced opposition to the use of facial recognition in public spaces. Younger respondents (aged 18 - 34) oppose the use of these technologies even more strongly (61 percent).
Reaction to the Commission’s plans has been mixed, as might have been expected. The issue that has attracted most attention is the one on real-time remote biometric identification systems in public spaces for law enforcement purposes. The EDPB and the EDPS both “strongly welcome” the aim of addressing the use of AI systems, including their use by EU institutions, bodies or agencies.
At the same time, both have reservations, as outlined by EDPD chair Andrea Jelinek, who says that deploying remote biometric identification in publicly accessible spaces “means the end of anonymity” in those places.
Applications such as live facial recognition interfere with fundamental rights and freedoms to such an extent that they may call into question their very essence, she argues, adding, “A general ban on the use of facial recognition in publicly accessible areas is the necessary starting point if we want to preserve our freedoms and create a human-centric legal framework for AI. The proposed regulation should also prohibit any type of use of AI for social scoring, as it is against the EU’s fundamental values and can lead to discrimination.”
MEPs also have mixed feelings on the proposals. Italian Socialist member Brando Benifei, shadow rapporteur on the special AI Committee, says that compared to the leaked text that was circulated a week prior to the presentation of the proposal - which generated much discussion and concern – the subsequent Commission text “provides for a more detailed and limited framework”.
“We cannot afford to make mistakes, in an era where authoritarian regimes are setting their own, illiberal standards”
Brando Benifei MEP
He says, however, that even if this constitutes a substantial improvement compared to the content of the leaked document, concerns persist as to remaining possible abuses and wide interpretation by some Member States.
Another area to flag up, he believes, concerns labour rights. “Annex III lists high-risk applications, including those that monitor and evaluate workers’ behaviour and performance, those that control the time a worker spends in front of the computer when teleworking, or even assess their mood by detecting emotions when making calls.”
He added “Regulating so loosely a practice (that is) so invasive for workers’ rights can be very dangerous, even more so as we consider these rules will apply to all AI developers targeting the European Union market, including non-EU entities, who might not necessarily share our values.”
The MEP warns, “We cannot afford to make mistakes, in an era where authoritarian regimes are setting their own, illiberal standards. On the contrary, similarly to the case of GDPR, we have a unique opportunity to define a world standard for a human-centric, trustworthy AI, to allow our citizens and businesses to make the most of such a promising technology, whose benefits we already experience in a wide variety of sectors.”
He hopes that the European Parliament will ‘improve’ the text, ensure appropriate safeguards are in place for high-risk applications and “stimulate good innovation and the creation of a true internal market for AI that serves humanity, and not only the interests of the few.”
Other MEPs agree that - while the current proposal is a good start - there is still work to be done to defend fundamental rights when it comes to facial recognition in public spaces, particularly over privacy issues. There is a consensus that the proposal is an excellent base and could open enormous opportunities but, equally, there is potential for abuse and wider interpretation.
“The proposal is an essential first step towards global standards but lacks bite in crucial places”
Kim van Sparrentak MEP
MEPs have also voiced concerns on how to govern the use of AI in the military and public domain. German Socialist deputy Ismail Ertug, who oversees digitalisation for his group, believes AI is the ‘key technology’ for revolutionising the mobility sector. Autonomous vehicles, real-time railway system optimisation and fuel planning all need AI to master the transition to “safe, inclusive environmentally-friendly mobility”.
AI can also reduce traffic, accidents, and pollution, he says. However, he cautions that, “What the Commission proposal lacks is a ban on the harmful use for the military and biometric facial recognition. AI technology should never undermine citizens’ rights to privacy, particularly when it comes to facial recognition rules."
"It should never lead to mass surveillance or discrimination, and we need to ban any applications that would lead to that. We must stop using facial recognition systems until we are sure they are fully compliant with fundamental rights standards.”
The AI regulation can become “another European success story” story like the GDPR but, for that, “we need clear rules that work for all.” Elsewhere, Greens/EFA member Kim van Sparrentak, also a member of the AI committee, applauds the draft proposal as “a worldwide first of its kind.”
It is an “essential first step” towards global standards but “lacks bite in crucial places”. Strong enforcement and options for people to complain to authorities about violation of the regulation is needed.
In April, before the proposal was tabled, 40 cross-party MEPs wrote a letter to the European Commission about their concerns over biometric facial recognition software. They called on the EU Executive to strengthen the proposal to include an outright ban on the use of facial recognition and other forms of biometric surveillance in public places.
Alexandre de Streel, co-director of the Centre on Regulation in Europe (CERRE) thinktank, accepts there is a difficult balance to be struck between protection and innovation. He says the text “sets a relatively open framework and everything will depend on how it is interpreted.”
The EU, with the Regulation, aims to ease public fears of Big Brother-like abuses by imposing checks on technology deemed ‘high-risk’. According to EU competition Commissioner Margrethe Vestager, the European Union is spearheading the development of new global norms “to ensure that AI can be trusted”.
The draft rules have sparked competing complaints from all sides; clearly, this is a debate involving Parliament and Member States that is set to rage on for many months more before a definitive text is in force.