Step 3: Assess for prohibition

The second step in the analysis should be whether the particular AI application or AI System is prohibited per se under the AI Act. There are six prohibited headings as follows outlined in Article 5 of the AI Act.

1. Subliminal techniques

AI Systems which operate beyond a person’s consciousness, and which may include manipulative or deceptive techniques which could distort a person’s decision making in a way which is reasonably likely to cause significant harm.

2. Exploiting human vulnerabilities

Where AI Systems exploit the vulnerabilities of a specific group of persons due to certain characteristics in order to materially distort their behaviour in a way which is reasonably likely to cause significant harm.

3. Social scoring

The deployment of AI to gauge the trustworthiness of natural persons over a period of time based on their social behaviour or known, or predicted personal or personality characteristics where such social scoring has a detrimental effect on the treatment of certain natural persons in contexts which are different to where source data was collected, or where such social scoring has a detrimental effect on the treatment of certain natural persons which is disproportionate to the nature of the source social data.

4. AI System use in Criminality Risk Assessments

This involves AI Systems making risk assessments of natural persons in order to predict criminality.

5. Facial recognition databases

Using AI Systems to create or expand facial recognition databases though untargeted scraping of facial images from internet or CCTV.

6. Emotional inference generation in workplace and education setting

Use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions.

In line items 1 and 2 above, the wording “significant harm” is clearly a higher threshold than mere “harm”. This is a higher threshold than previous drafts of the AI Act, which pinned these line items to harming natural persons in a “material” way. Recital 29 makes clear that the prohibition does not cover AI Systems which use subliminal techniques or exploit vulnerabilities which do not “have sufficiently important adverse impacts on physical, psychological health or financial interests”.