High-Risk AI Systems: Practical Guide for SaaS Teams

High-risk AI systems are AI systems that fall into the EU AI Act's stricter compliance route because of their intended purpose, product context, or potential impact on health, safety, or fundamental rights. For SaaS teams, the practical question is not "does this product use AI?" The practical question is whether a specific AI use case is a safety component of a regulated product, is itself a regulated product, or is intended for one of the listed high-risk uses in Annex III.

If the answer may be yes, the team needs a structured workflow. That workflow should identify the system, role, intended purpose, affected people, data, customer configuration, vendor involvement, evidence, controls, owner, and launch gate. A high-risk decision should not live only in a legal memo. It needs to change how product, engineering, security, privacy, compliance, and customer-facing teams build and operate the system.

The European Commission's May 2026 draft guidelines are useful because they explain the Commission's current interpretation of Article 6 and provide practical examples. They are still draft guidance, so teams should treat them as important implementation material while keeping the AI Act itself as the source of legal obligation.

Why this matters in practice

High-risk classification changes the operating burden. The AI Act connects high-risk systems with requirements for risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy, robustness, cybersecurity, quality management, conformity assessment, registration where required, monitoring, and incident handling.

That sounds heavy, and for some systems it is. But the bigger risk for SaaS companies is often ambiguity. A team may ship an AI feature for ranking, scoring, filtering, recommendation, fraud review, employment workflow, education workflow, identity check, or access decision without deciding whether the use case belongs in the high-risk path. By the time procurement, enterprise security, legal review, or a customer audit asks the question, the feature may already be embedded in contracts and release commitments.

Classification helps teams move faster because it routes work. A low-risk drafting assistant should not carry the same process as an AI system that screens job candidates. A vendor AI feature used for internal summarisation should not be treated the same as a customer-facing scoring workflow used in credit, employment, education, healthcare, or essential services. The point is to decide early which route applies and what evidence proves the decision.

The two main high-risk routes

The AI Act's high-risk classification has two main routes that SaaS teams should translate into operational checks.

The first route concerns regulated products. An AI system can be high-risk when it is intended to be used as a safety component of a product, or is itself a product, covered by Union harmonisation legislation listed in Annex I, and the product or AI system is required to undergo third-party conformity assessment. This route is especially relevant for SaaS connected to medical devices, machinery, transport, aviation, toys, radio equipment, or other regulated product environments.

Many ordinary B2B SaaS products will not fall into this route. But teams serving health, industrial, robotics, mobility, infrastructure, or device markets should not dismiss it. If the AI output supports safety-relevant product behavior, diagnostics, control, monitoring, or certification, the review should involve product counsel and the relevant regulatory owner.

The second route concerns Annex III use cases. These include listed areas such as biometrics, critical infrastructure, education and vocational training, employment and worker management, access to essential private and public services, law enforcement, migration and border control, administration of justice, and democratic processes. The analysis depends on intended purpose and concrete use, not just the model category.

For SaaS teams, Annex III review is often the more common concern. A product that supports hiring, candidate filtering, student assessment, worker performance review, creditworthiness evaluation, essential-service access, biometric identification, insurance risk assessment, or legal decision support may need deeper review even if the same model could be low risk in a different workflow.

When SaaS teams should trigger review

Trigger high-risk review whenever an AI system can affect people in a consequential context. Do not wait for a perfect legal conclusion before collecting facts. The first job is to identify whether deeper review is needed.

Common triggers include a new AI feature, a new model or vendor, a new intended purpose, a new customer configuration, use in employment or HR, use in education, use in healthcare or essential services, use in credit or insurance, biometric processing, automated ranking or eligibility support, reduced human review, expansion into the EU market, or a customer question about AI Act exposure.

Vendor AI needs the same trigger discipline. A vendor may describe a capability as insight, intelligence, automation, recommendation, engagement, screening, risk detection, identity, or safety. Those labels do not answer the classification question. The team needs to know what the system does, whose data it uses, who is affected, how the output is used, whether the customer can configure sensitive workflows, and whether the vendor expects the customer to act as deployer.

Customer configuration also matters. A general workflow tool may look ordinary in default settings but become sensitive when a customer uses it to rank applicants, assess workers, route students, score eligibility, or influence access to benefits or essential services. SaaS teams should define configuration triggers that require review before enablement.

A practical workflow

Start with a short intake. Capture system name, owner, business purpose, user journey, affected people, geography, data categories, model or vendor, AI Act role, output, human review, customer configuration, and whether the use case touches regulated products or Annex III areas.

Then assign a route. Clear no-red-flag cases can proceed to ordinary AI classification, privacy, security, and vendor review. Clear high-risk candidates should move into deeper legal and compliance assessment before launch. Ambiguous cases should go to a named reviewer with a deadline rather than sitting in a risk register.

For likely high-risk systems, expand the record. Capture the risk-management approach, data governance checks, training or input data controls, technical documentation owner, logging approach, user instructions, human oversight model, accuracy and robustness testing, cybersecurity measures, post-market monitoring plan, incident escalation, vendor evidence, conformity-assessment route, and customer documentation.

The workflow should connect to product delivery. A missing classification should block release for AI-enabled features in sensitive domains. A likely high-risk classification should trigger controls and approval conditions. A documented no-high-risk conclusion should not create extra delay by itself, but it should remain reviewable when facts change.

Evidence to keep

Evidence should answer four questions: what did the team review, why did it classify the system this way, what controls followed from that decision, and when will the decision be reopened?

Keep the AI inventory entry, intake form, product or vendor description, data-flow note, affected-person analysis, intended-purpose statement, Annex I or Annex III screening result, role analysis, classification conclusion, reviewer, date, rationale, sources consulted, launch decision, controls triggered, and next review trigger.

If the system is built internally, keep architecture notes, model documentation, testing evidence, logging design, human oversight design, and monitoring plan. If the system is vendor-provided, keep the vendor's AI documentation, instructions for use, contractual commitments, security questionnaire answers, audit or certification materials, and any statement about AI Act role allocation.

Evidence should be stored where work happens. Product tickets, vendor records, privacy reviews, security design records, release checklists, and customer trust folders are usually stronger than a standalone spreadsheet that no one checks before launch.

Common mistakes

The first mistake is treating high-risk status as a brand label. The same model family can support a low-risk drafting workflow or a high-risk employment workflow. Classification follows intended purpose and use.

The second mistake is relying only on vendor assurances. Vendor documentation matters, but the SaaS team still needs to understand its own deployment, customers, configuration, affected people, and contractual commitments.

The third mistake is assuming human review solves the classification question. Human oversight may be one required control, but it does not automatically remove a system from the high-risk path.

The fourth mistake is forgetting change management. A previous classification may become stale when the system changes purpose, data, geography, customer segment, output use, model version, vendor, or human review.

The fifth mistake is separating classification from evidence. A conclusion without facts, owner, rationale, and review trigger will be hard to defend during an audit, enterprise customer review, or regulatory question.

Example scenarios

An AI assistant that summarizes internal support notes for agents may be outside the high-risk route if it does not make or recommend consequential decisions, does not operate in an Annex III context, and is reviewed by staff before use. The team should still document the data flow, vendor terms, human review, privacy impact, and customer commitments.

An HR SaaS feature that filters applications, ranks candidates, evaluates performance, or supports termination decisions is different. Employment and worker management are sensitive Annex III areas. The team should route the use case into deeper high-risk review, document human oversight, test data quality and bias risk, define customer instructions, and keep evidence of monitoring.

A healthcare workflow that uses AI to support treatment prioritisation or medical-device functionality may raise both Annex III and regulated-product questions. The team should identify whether the system is a safety component, whether sectoral rules require third-party conformity assessment, and which obligations belong to the SaaS provider, customer, manufacturer, or vendor.

A horizontal workflow platform may not know every customer use case in advance. That does not remove the need for controls. The vendor may need product restrictions, admin controls, customer-use questionnaires, contractual limits, documentation, support escalation, and logging to prevent sensitive configurations from bypassing review.

How to operationalize ownership

Product should own the intended purpose, customer journey, configuration, and release facts. Engineering should own architecture, data flow, model integration, logs, testing, and technical controls. Security should own vendor, access, deployment, and cybersecurity review. Privacy should own data protection impact and data minimisation. Legal and compliance should own AI Act interpretation, role analysis, evidence standards, and approval conditions. Leadership should own risk acceptance for sensitive or ambiguous launches.

Tie the workflow to existing operating points: product intake, architecture review, vendor onboarding, privacy review, security review, sales commitments, customer configuration, and release approval. That keeps high-risk review from becoming a parallel bureaucracy.

This also connects to broader AI governance expectations for SaaS vendors, the EU AI Act overview for SaaS providers, internal AI-tool review, and the controls buyers increasingly ask about for AI-enabled SaaS products.

FAQ

What is the practical purpose of high-risk AI systems?

The practical purpose is to identify AI systems that require a stricter governance path, stronger evidence, and lifecycle controls before they are placed on the market, put into service, or used in sensitive workflows.

When does high-risk AI systems apply to SaaS teams?

It may apply when a SaaS team builds, sells, embeds, configures, or uses AI as a safety component of a regulated product, as a regulated product, or for an Annex III use case such as employment, education, essential services, biometrics, law enforcement, migration, justice, or democratic processes.

What should teams document or change first?

Start with an AI inventory, high-risk intake screen, role analysis, intended-purpose statement, affected-person analysis, classification decision, named owner, evidence location, and launch gate.

Is every AI-enabled SaaS product high risk?

No. Many AI-enabled SaaS features will not be high-risk. The team still needs to document why the use case does or does not enter the high-risk route, especially when people are ranked, scored, filtered, monitored, or given access to important services.

Can a vendor decide classification for us?

No. Vendor documentation is important evidence, but the SaaS team must still assess its own intended use, customer configuration, affected people, market placement, and contractual role.

Sources

• Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence.
• European Commission draft guidelines on the classification of high-risk AI systems.
• European Commission AI Act FAQ on high-risk AI systems.
• AI Act Service Desk guidance on high-risk AI in regulated products.