AI Act: How to prepare your company for new AI regulations?

Are you wondering who will pay for mistakes made by algorithms in your company and how this will realistically affect your responsibilities? The upcoming AI Act is a revolution in AI legal regulations, introducing precise rules of the game and establishing who bears liability for artificial intelligence. From this article, you will learn how to classify the AI systems you use, what obligations the new law imposes on you, and how to prepare your organization for the upcoming changes to avoid severe penalties.

Table of contents

Introduction

The AI Act: The foundation of new AI legal regulations in Europe

What is the AI Act and what are its goals?

The main goal of the AI Act is to ensure that artificial intelligence systems introduced and used in the EU market are safe and comply with fundamental rights and EU values. The regulation is intended to harmonize rules across the Union, eliminating legal fragmentation and creating a predictable environment for innovation. The key objectives of the legislators are:

1. Protecting users and society: Ensuring that AI technologies do not violate fundamental rights, such as the right to privacy, non-discrimination, or security.


2. Building trust: Creating conditions where citizens and companies can trust AI solutions, which is essential for their widespread acceptance and market development.


3. Fostering innovation: Establishing clear and proportionate rules is intended to give companies legal certainty, encouraging investment and the development of ethical AI applications.


4. Establishing a global standard: The EU aims for the AI Act to become a benchmark for AI legal regulations worldwide, similar to what happened with GDPR.

A risk-based approach: Four levels of AI system classification

The heart of the AI Act is its risk-based approach. Instead of creating uniform, rigid rules for all AI applications, the regulation categorizes systems according to the potential threat they may pose to health, safety, and fundamental rights. This classification divides AI systems into four main levels:

• Unacceptable Risk: Systems that are considered contrary to EU values and will be completely banned.


• High Risk: Systems that can have a significant impact on people's lives and rights. They are subject to the strictest requirements.


• Limited Risk: Systems requiring compliance with specific transparency obligations.


• Minimal or No Risk: The vast majority of AI systems, which are not subject to special legal obligations.

Risk categories in the AI Act: What this means for your company

Unacceptable risk systems: Red lines for AI

The AI Act introduces a clear list of practices that are considered a threat to fundamental values and human rights. AI systems belonging to this category will be banned in the European Union. These include, among others:

• Social scoring systems run by public authorities that classify citizens based on their social behavior or personality traits.


• Systems that use subliminal techniques to manipulate human behavior in a way that could cause physical or psychological harm.


• Systems that exploit the vulnerabilities of specific vulnerable groups (e.g., due to age, disability) to influence their behavior.


• "Real-time" remote biometric identification systems in public spaces by law enforcement (with very narrow exceptions).

High-risk systems: A key area for CIOs

This is the most important and complex category from a business standpoint. High-risk systems are those that can have a significant, negative impact on the safety, health, or fundamental rights of individuals. The AI Act divides them into two groups:

1. AI systems that are components of products already subject to EU sectoral regulations (e.g., medical devices, toys, machinery, lifts).


2. AI systems listed in Annex III of the regulation, covering key areas of social and economic life.

• Management of critical infrastructure: e.g., systems controlling road traffic, water or energy supplies.


• Education and vocational training: e.g., algorithms that grade exams or assign candidates to educational institutions.


• Employment and workforce management: systems for candidate selection, performance evaluation, or making decisions about promotion or dismissal.


• Access to essential private and public services: e.g., systems for credit scoring.


• Justice and democratic processes.

• Implementing a risk management system throughout the AI lifecycle.


• Ensuring high-quality data used for training models to minimize the risk of discrimination and errors.


• Creating detailed technical documentation to allow for assessment of the system's compliance.


• Ensuring the possibility of human oversight over the system's operation.


• Achieving a high level of accuracy, robustness, and cybersecurity.

Limited and minimal risk systems: Information duties and good practices

Many popular AI applications, such as chatbots, recommendation systems, or deepfakes, have been classified as limited-risk systems. The main requirement for them is transparency. Users must be clearly informed that they are interacting with an AI system (in the case of chatbots) or that the content has been generated or modified by AI (in the case of deepfakes). The goal is to avoid manipulation and deception.

The minimal risk category, on the other hand, includes the vast majority of AI applications, such as spam filters or algorithms in video games. For these systems, the AI Act does not impose any mandatory requirements. However, providers are encouraged to voluntarily adopt codes of conduct that promote ethical and trustworthy practices.

Liability for AI: Who pays for algorithm mistakes?

The new AI Liability Directive: A game-changer

In parallel with the AI Act, the European Union is working on a directive to make it easier for individuals harmed by AI systems to claim compensation. The key change introduced by this draft is the rebuttable presumption of causality. In practice, this means shifting the burden of proof.

Until now, the injured party had to prove that a specific error in the AI system was the direct cause of their damage, which is extremely difficult in the case of complex algorithms. After the directive comes into force, if the injured party demonstrates that they were exposed to a high-risk AI system and suffered damage typical of that type of risk, the court will be able to presume a causal link. It will then be up to the provider or user of the AI system to prove that their technology was not at fault. This is a fundamental change that significantly increases liability for AI on the part of companies.

Who is liable for artificial intelligence errors? The chain of liability

The AI Act precisely defines the roles and responsibilities throughout the AI product lifecycle, creating a clear chain of liability. There is no single, simple answer here – liability can rest with different entities, depending on their role:

• Provider: This is the entity that develops the AI system and places it on the market. They bear the main burden of ensuring compliance with the AI Act, including conducting a risk assessment, preparing technical documentation, and implementing quality management systems. The provider is responsible for "building in" security and ethics into the product.
  
  Check out our guide and find out what to ask a software house before signing a contract to guarantee that the delivered software meets all EU legal requirements:
  Software House – How to choose and what to ask?


• Importer and Distributor: Entities that place AI systems from third-country providers on the EU market. Their duty is to verify that the provider has completed all formalities and that the product has the appropriate markings (e.g., CE).


• User/Deployer: This is the company or institution that uses an AI system in its professional activities (e.g., a bank using a credit scoring algorithm). The user is obliged to use the system in accordance with its intended purpose and instructions, as well as to ensure appropriate human oversight. In the case of high-risk systems, some of the liability for the consequences of their operation may rest with the user.

Legal liability for algorithm decisions in practice

Let's imagine a scenario: a recruitment company uses an AI system to pre-screen candidates. The algorithm, due to errors in the training data, systematically rejects candidates from a specific demographic group. Who bears the legal liability for the algorithm's decisions?

1. The AI system provider can be held liable if they failed to ensure adequate quality of the training data, did not conduct tests for bias, or did not provide clear documentation about the system's limitations.


2. The recruitment company (user) can be held liable if it ignored the provider's warnings, failed to provide adequate human oversight of the recruitment process (e.g., by randomly verifying the algorithm's decisions), or used the system in a manner inconsistent with its intended purpose.

How to prepare your company for the AI Act? A practical guide for IT leaders

Step 1: Audit and classification of existing AI systems

The first and fundamental action is to create a comprehensive inventory of all AI systems used in the organization. You cannot manage what you do not know about. The audit should cover both systems developed internally and those supplied by external vendors. For each system, you need to answer the following questions:

• What is the business purpose of this system?


• What data is used for its training and operation?


• Who is the provider (internal team or external company)?


• What decisions are made or supported by this system?

Step 2: Gap analysis and implementation of requirements for high-risk systems

Once high-risk systems have been identified, a detailed gap analysis should be conducted. This involves comparing the current state of the system and its processes with the stringent requirements of the AI Act. This analysis should answer the questions:

• Do we have a risk management system for this AI system?


• Does our data quality and governance meet the new standards?


• Is the technical documentation complete and up-to-date?


• Are the human oversight mechanisms sufficient and effective?


• Is the level of cybersecurity adequate for the risk?

Step 3: Documentation, transparency, and human oversight

Documentation is becoming one of the pillars of compliance with the AI Act. For high-risk systems, it will be necessary to maintain detailed technical documentation that describes, among other things, the system's architecture, the data used, testing processes, validation results, and foreseeable limitations. This documentation will be crucial not only for audits by supervisory authorities but also in the context of liability for AI in the event of legal disputes.

Read our article and see how reliable technical documentation reduces costs and risks in IT, facilitating potential audits and making the business resilient to employee turnover:
Technical Documentation: How to Lower IT Costs and Risk

Transparency is equally important. For limited-risk systems, mechanisms must be implemented to inform users about their interaction with AI. For high-risk systems, transparency means providing users with clear information about the system's capabilities and limitations, so they can make informed decisions and exercise effective oversight.

What does the AI Act mean for software houses and IT departments?

For development teams, both in software houses and internal IT departments, the AI Act means the need to integrate new requirements into the software development life cycle (SDLC) and Machine Learning Operations (MLOps) processes. In practice, this means:

• Security & Compliance by Design: The principles of the AI Act must be considered from the very beginning of a project, not added on at the end.
  
  Discover proven market practices and learn how to plan data security: strategy and protection procedures to minimize the risk of leaks and vulnerabilities in newly developed algorithms:
  Data Security: Effective strategy and data protection procedures


• Rigorous data management: The processes of acquiring, labeling, validating, and monitoring data must be formalized and documented.


• Versioning and traceability: It will be necessary to version not only code but also models and the data they were trained on to ensure full reproducibility and auditability.


• Test automation: Implementing automated tests that check not only for performance but also for robustness, fairness, and model security.


• New roles on the team: There may be a need for AI ethics specialists, algorithm auditors, or experts in AI Act compliance.

Ethics in AI: More than regulatory compliance

Summary

We will translate the complexity of the AI Act into concrete technical solutions and risk management strategies, ensuring compliance and building trust in your systems.

Talk to our expert about the steps for your company – fill out the form.