Artificial Intelligence

AI First Governance and the EU AI Act: What Businesses Need to Know

AI First Governance and the EU AI Act: What Businesses Need to Know

AI Governance and the EU AI Act: What Businesses Need to Know

Correction, 3 October 2026: An earlier version of this article, written before the AI Omnibus, stated that the EU AI Act’s high-risk obligations would apply from August 2026. That is no longer correct. The AI Omnibus, which entered into force on 27 July 2026, moved the high-risk obligations to 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems. From 2 August 2026 the Act became generally applicable, including the Article 50 transparency obligations and the enforcement powers of the AI Office and national authorities. The article’s timeline has been updated to reflect the AI Omnibus, and its descriptions of the Act’s requirements, including who counts as a provider or a deployer, have been corrected throughout.

The EU AI Act is in force and applies in stages. Its prohibitions have applied since February 2025, the general-purpose AI (GPAI) obligations since August 2025, and most of its remaining provisions since 2 August 2026. The high-risk system requirements come later: the AI Omnibus moved them to 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems. For UK businesses, the instinct may be to dismiss this as European regulation that does not apply domestically.

That instinct is wrong. The AI Act applies to providers that place AI systems on the EU market or put them into service in the EU, wherever those providers are established, and to providers and deployers outside the EU where the output of their AI system is used in the EU. Where your servers are located or where your company is incorporated does not take you out of scope.

Artificial intelligence was established as an academic discipline in 1956, rooted in computer science, and has since experienced cycles of optimism and disappointment as well as rapid recent advancements. The field draws on computer science to advance technologies such as artificial neural networks and deep neural networks, which are foundational to modern AI.

AI algorithms now power a wide range of products and services, enabling automation, data analysis, and personalized experiences. Deep neural networks, with their multiple hidden layers, and artificial neural networks are crucial for modeling complex data relationships and underpinning deep learning applications. Generative AI can produce not only text and images but also other forms such as videos, audio, and software code.

AI systems excel at specific tasks, such as playing games, operating in industry-specific applications, or executing short, goal-oriented actions. In transportation, AI enables autonomous vehicles to perceive their environment and make complex decisions. In marketing and branding, AI empowers creatively engaging brands to develop innovative and captivating brand experiences.

The knowledge gained by AI models through learning from data improves their performance and ability to transfer skills to new problems. AI is also used for solving math problems, with specialized models and training approaches designed for this purpose. The Turing test remains a historical benchmark for evaluating machine intelligence. AI systems can simulate emotions to mimic human feelings, though they do not possess genuine consciousness.

This new reality requires organizations to adapt quickly to the rapid advancements in AI and the societal shifts they bring. Becoming an AI first organization means fundamentally redesigning strategies, workflows, and culture around AI capabilities, rethinking jobs, developing new skills, and achieving early wins. Practitioners bold within organizations are actively experimenting with AI and driving innovation, and leaders can encourage AI experimentation across the whole workforce to drive organizational change.

The chief digital officer plays a key role in leading digital transformation initiatives and leveraging emerging technologies, including by integrating new customer-facing technology with brand strategy. Legendary tech visionaries and former chief digital officers have shaped digital transformation and technology strategy, inspiring companies to embrace AI-driven change.

This article is not legal advice. It is a practical guide for development teams and technical leaders who need to understand what the EU AI Act requires, how the UK regulatory approach differs, and what concrete steps your engineering organisation should be taking now. At McKenna Consultants, we help businesses implement AI systems with appropriate governance built in from the architecture level, and the patterns described here are the ones we recommend.

Introduction to Artificial Intelligence

Artificial intelligence (AI) is transforming the way businesses operate, enabling computer systems to perform tasks that once required human intelligence—such as problem solving, decision making, and learning from experience. At its core, artificial intelligence leverages advanced algorithms and vast amounts of data to analyze information, recognize complex patterns, and make predictions, often without being explicitly programmed for every scenario.

AI systems come in many forms, from machine learning models that learn to identify trends in data, to natural language processing tools that understand and generate human language, to computer vision systems that interpret high quality images and video.

AI researchers are continually pushing the boundaries of what these systems can achieve, drawing inspiration from the human brain and the intricacies of human intelligence to develop artificial neural networks and deep learning architectures with multiple layers capable of tackling a broad range of real world applications.

One of the most exciting developments in recent years is the rise of generative AI. Generative AI tools, such as large language models, can create brand new content—text, images, music, and even computer code—at almost no cost.

These generative AI applications are already changing marketing forever, enabling creative professionals and marketers to launch campaigns, analyze data, and engage customers in ways that were once the stuff of science fiction. AI agents, including virtual assistants, are now able to perform tasks like scheduling, customer support, and data analysis, using natural language processing and machine learning to interact with users in a human-like manner.

The rapid rise of AI has been fueled by advances in computing power, the availability of massive training data, and breakthroughs in deep learning and neural networks. Technology leaders regularly speak about the potential of artificial intelligence to reshape industries and society.

From self driving cars that use computer vision and deep neural networks to navigate roads, to personalised retail apps and loyalty schemes, the impact of AI is already visible in our daily lives.

However, the deployment of AI systems is not without challenges. AI models can inadvertently perpetuate algorithmic bias if trained on skewed or incomplete data, leading to unfair or discriminatory outcomes.

There are also concerns about job displacement, as AI tools and autonomous agents take on repetitive tasks and even some creative or problem solving roles previously reserved for humans. As AI systems become more capable—approaching the realm of artificial general intelligence—businesses must grapple with the implications for their workforce, brand strategy, and long-term competitiveness.

Despite these challenges, the benefits of adopting an AI first strategy are significant. AI can help organizations future proof business operations, unlock new opportunities for innovation, and solve complex problems in fields ranging from healthcare and finance to eCommerce and education. Companies that embrace an AI first world—investing in AI research, integrating AI tools into their platforms, and reimagining their approach to problem solving—are well positioned to achieve early wins and thrive in the brand new world of digital transformation.

For both large enterprises and smaller businesses, the adoption of AI is no longer optional. Whether deploying AI-powered chatbots to enhance customer service, using machine learning to analyze new data and optimize marketing campaigns, or building agentic AI systems to automate business processes, the possibilities are vast. By understanding the fundamentals of artificial intelligence and its potential applications, organizations can begin to creatively engage brands, change brand strategy, and lead in the AI first arena.

As the rest of this article will explore, the rapid evolution of AI brings new regulatory and governance challenges. Understanding the basics of AI is the first step toward building responsible, scalable, and secure AI systems that deliver real value—while navigating the complex landscape of compliance, risk, and opportunity.

The EU AI Act: Structure and Scope

The AI Act takes a risk-based approach to regulation. It classifies AI systems into four tiers, each with different obligations.

Unacceptable Risk (Prohibited)

Certain AI practices are banned outright within the EU, and most of the bans have applied since 2 February 2025. They include harmful manipulation and deception, harmful exploitation of vulnerabilities due to age, disability or a social or economic situation, social scoring, predicting an individual’s risk of committing a crime based solely on profiling, untargeted scraping of facial images to build facial recognition databases, emotion recognition in workplaces and education institutions, biometric categorisation to infer sensitive characteristics, and real-time remote biometric identification in publicly accessible spaces for law enforcement purposes (with narrow exceptions). A further prohibition added by the AI Omnibus, covering AI systems that generate non-consensual intimate content or child sexual abuse material, applies from 2 December 2026. In our view, most UK B2B software companies will not encounter these prohibitions — although HR and productivity tools should check the workplace emotion recognition ban — but it is worth understanding the boundary.

High-Risk AI Systems

This is the category with the most substantial obligations for UK technology companies selling into the EU. There are two routes into it. An AI system that is a safety component of a product covered by the EU product legislation listed in Annex I to the Act (machinery, toys and lifts, for example), or is itself such a product, is high-risk where that product must undergo third-party conformity assessment (Article 6(1)). The use cases listed in Annex III are high-risk under Article 6(2). Annex III includes AI systems used in:

  • Employment and workers’ management: Recruitment and selection (including filtering job applications and evaluating candidates), decisions on promotion or termination, task allocation based on individual behaviour or personal traits, performance monitoring

  • Access to essential services: Creditworthiness assessment and credit scoring, risk assessment and pricing for life and health insurance, public authorities’ decisions on eligibility for public assistance benefits

  • Education: Admission decisions, evaluating learning outcomes (including steering a learner’s learning process), monitoring students during tests

  • Critical infrastructure: Safety components in the management and operation of critical digital infrastructure, road traffic, and the supply of water, gas, heating or electricity

  • Law enforcement and justice: Evaluating the reliability of evidence, assessing the risk of offending (where not based solely on profiling, which is prohibited), assisting judicial authorities in researching and interpreting facts and the law

Annex III also covers biometrics, migration, asylum and border control, and AI intended to influence elections. An Annex III system is not high-risk if it does not pose a significant risk of harm — for example, where it only performs a narrow procedural task or a preparatory task — but one that profiles individuals is always high-risk (Article 6(3)). A provider relying on that exception must document its assessment before placing the system on the market.

If your product includes AI capabilities in one of these areas and you place it on the EU market, or its output is used in the EU, the high-risk obligations apply from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems — the dates set by the AI Omnibus. The requirements are substantial: risk management systems, data governance, technical documentation, record-keeping, human oversight provisions, accuracy and robustness, a quality management system, conformity assessment before the system is placed on the market, and post-market monitoring.

Limited Risk (Transparency Obligations)

The Article 50 transparency obligations have applied since 2 August 2026. Providers must design AI systems that interact directly with people so that those people are told they are interacting with an AI system, unless that is obvious. Providers of systems that generate synthetic audio, image, video or text must mark the output in a machine-readable format so that it is detectable as AI-generated. Deployers of emotion recognition and biometric categorisation systems must inform the people exposed to them, and deployers must disclose deep fakes and AI-generated text published to inform the public on matters of public interest (unless the text has undergone human review or editorial control). This is not purely a disclosure exercise: the marking obligation is a technical requirement. Its only grace period covers generative systems placed on the market before 2 August 2026, which must comply with the marking obligation from 2 December 2026.

Minimal Risk

AI systems that do not fall into the above categories — the Commission’s examples are AI-enabled video games and spam filters — face no specific rules under the AI Act beyond the AI literacy duty in Article 4, which applies to providers and deployers of all AI systems. General product safety and data protection rules still apply.

When the Obligations Apply

The AI Act entered into force on 1 August 2024 and applies in stages. As amended by the AI Omnibus, the key dates are:

  • 2 February 2025: Prohibited practices and the AI literacy duty

  • 2 August 2025: Governance rules and the obligations for providers of general-purpose AI models

  • 2 August 2026: General application of the Act, including the Article 50 transparency obligations and the enforcement powers of the AI Office and national authorities

  • 2 December 2026: The new prohibition on AI-generated non-consensual intimate material and child sexual abuse material, and the end of the marking grace period for generative systems already on the market

  • 2 August 2027: Deadline for providers of general-purpose AI models placed on the market before 2 August 2025

  • 2 December 2027: High-risk obligations for Annex III systems

  • 2 August 2028: High-risk obligations for Annex I systems

Fines for breaching the prohibitions can reach €35 million or 7% of total worldwide annual turnover, whichever is higher. Breaches of the high-risk, transparency and GPAI model obligations can reach €15 million or 3%. For SMEs, including start-ups, fines under Article 99 are capped at whichever of the two figures is lower.

Generative AI and GPAI Obligations: What Changed in August 2025

The general-purpose AI model obligations are distinct from the risk-based classification above. They apply to the providers of foundation models and large language models — the companies that train and distribute the base models. However, they have downstream implications for every business that builds on top of those models.

For GPAI Model Providers

Providers of general-purpose AI models must now:

  • Draw up and keep up to date technical documentation of the model, including its training and testing process and evaluation results, for the AI Office and national authorities on request

  • Provide information and documentation to the businesses that integrate the model into their own AI systems, so that they understand its capabilities and limitations and can comply with their own obligations

  • Put in place a policy to comply with EU copyright law

  • Publish a sufficiently detailed summary of the content used to train the model

Providers of models placed on the market before 2 August 2025 have until 2 August 2027 to comply. Since 2 August 2026, the AI Office has been able to enforce these obligations, including by issuing fines.

Models classified as posing “systemic risk” (a model is presumed to qualify when the cumulative compute used to train it exceeds 10^25 FLOPs) face additional obligations: model evaluation including adversarial testing, assessing and mitigating systemic risks, reporting serious incidents, and ensuring adequate cybersecurity protection.

What This Means If You Build on These Models

If you are building AI features using OpenAI’s GPT models, Anthropic’s Claude, Google’s Gemini, or similar foundation models, the GPAI model obligations above sit with the model provider, not with you. You are not exempt, though. Under the Act’s definitions, a business that develops an AI system and places it on the market or puts it into service under its own name is the “provider” of that system — the Act calls a provider that integrates an AI model, including someone else’s, a “downstream provider” — while a business that uses an AI system under its authority is a “deployer.” Business customers who use your AI features are deployers; if you only use such a tool internally, you are a deployer of it.

What you must do depends on how your AI system is classified, not on which model sits underneath it:

  • All AI systems: Providers and deployers must take measures to support the AI literacy of staff who operate and use AI systems on their behalf (Article 4)

  • Systems that interact with people or generate content: The Article 50 transparency obligations described above, which have applied since 2 August 2026

  • High-risk systems: The full provider obligations apply from 2 December 2027 (Annex III) or 2 August 2028 (Annex I). Deployers of high-risk systems must use them in accordance with the provider’s instructions, assign human oversight to people with the necessary competence and authority, monitor their operation and keep the logs they generate (Article 26)

If you take a general-purpose AI system that has not been classified as high-risk and put it to a high-risk use, the Act treats you as the provider of a high-risk AI system (Article 25). The model provider, for its part, must give you the information you need to understand the model and meet your own obligations.

In our view, the practical implication is that your documentation and governance processes should capture how your AI features work end-to-end: which models you use, what prompts and system instructions shape their behaviour, how outputs are validated, and what human review occurs before AI-generated outputs affect real decisions.

How the UK Regulatory Approach Differs

The UK has taken a different route from the EU. The government’s 2023 white paper on AI regulation set out a “pro-innovation” framework based on five cross-sectoral principles: safety, security, and robustness; appropriate transparency and explainability; fairness; accountability and governance; and contestability and redress.

The white paper said these principles would be issued on a non-statutory basis and implemented by existing regulators, using their domain-specific expertise — the FCA for financial services and the ICO for data protection, for example. Each regulator interprets and applies the principles within its own domain.

What This Means in Practice

In our view, for UK businesses operating purely domestically, the regulatory burden is currently lighter than the EU’s prescriptive requirements. However, there are several reasons not to treat this as a free pass.

UK requirements are moving too. Regulations made in 2026 require the UK data protection regulator to prepare a code of practice on developing and using AI and on automated decision-making. Our view is that UK expectations of AI governance will continue to rise, and that building governance structures now means you will not be scrambling to retrofit them later.

The ICO is already active. UK data protection law already applies to AI systems that process personal data. Since 5 February 2026, the UK GDPR’s automated decision-making rules — Articles 22A to 22D, which replaced Article 22 under section 80 of the Data (Use and Access) Act 2025 — have required safeguards whenever a significant decision about a person is based solely on automated processing: giving them information about the decision and enabling them to make representations, obtain human intervention and contest it. Data protection impact assessments are still required for processing that is likely to result in a high risk to individuals. The ICO’s stated areas of focus for 2026/27 include its AI code of practice and dedicated guidance on agentic AI.

Procurement requirements. Enterprise customers, particularly in financial services, healthcare, and the public sector, may ask for AI governance documentation as part of procurement. Even without prescriptive regulation, customer expectations can drive governance requirements.

EU market access. If you place AI systems on the EU market — or plan to — or their output is used in the EU, the AI Act can apply to you. In our view, building governance to the EU standard from the outset is significantly cheaper than retrofitting it later.

Practical Steps for Technical Teams

This is where the article moves from regulatory overview to engineering practice. The following steps represent the governance infrastructure that development teams should be implementing now.

1. Create an AI System Inventory

You cannot govern what you do not know about. Start by cataloguing every AI system, feature, or component in your product portfolio. For each entry, document:

  • What it does. A plain-language description of the AI capability.

  • Which models it uses. Foundation model provider, model version, fine-tuning status.

  • What data it processes. Input data types, sources, whether it includes personal data.

  • What decisions it influences. Does the output inform a human decision, or does it trigger an automated action?

  • Who it affects. End users, employees, third parties.

  • Risk classification. Based on the EU AI Act categories above, what risk tier does this system fall into?

This inventory becomes the foundation for all subsequent governance activities.

2. Implement Human-in-the-Loop Governance

Human-in-the-loop AI governance is not a checkbox exercise. It requires architectural decisions that shape how your AI features are built.

For high-stakes decisions, the architecture must ensure that a qualified human reviews AI outputs before they take effect. This means:

  • Confidence thresholds. AI outputs below a defined confidence score are automatically routed for human review. Outputs above the threshold may proceed, but are subject to sampling and audit.

  • Explanation infrastructure. The human reviewer must be able to understand why the AI produced a particular output. This requires logging the inputs, the model’s reasoning chain (where available), and the key factors that influenced the output.

  • Override mechanisms. Humans must be able to override AI decisions and have those overrides recorded and fed back into system improvement.

  • Escalation paths. When a human reviewer is uncertain, there must be a clear escalation route to a more senior decision-maker.

For lower-stakes applications — content recommendations, search ranking, formatting suggestions — the governance model can be lighter, but transparency requirements still apply. Users should know when AI is influencing what they see.

3. Build Technical Documentation

The EU AI Act’s documentation requirements are specific. Even if you are not yet obligated to produce them, building this documentation practice now creates a durable governance asset.

For each AI system, maintain:

  • System architecture documentation. How the AI component fits into the broader product architecture. Data flows, API boundaries, deployment infrastructure.

  • Training and evaluation data documentation. For fine-tuned models, document the training data sources, preprocessing steps, and evaluation metrics. For prompt-engineered systems, document the system prompts, few-shot examples, and evaluation benchmarks.

  • Risk assessment. A structured assessment of potential harms, including bias, accuracy failures, adversarial manipulation, and unintended use cases. Include mitigation measures for each identified risk.

  • Performance metrics. Ongoing measurement of accuracy, precision, recall, fairness metrics, and failure rates. These metrics should be monitored in production, not just evaluated at launch.

  • Change log. A record of every material change to the AI system — model upgrades, prompt changes, training data updates, threshold adjustments — with the rationale for each change.

4. Implement Bias and Fairness Testing

AI systems can produce discriminatory outcomes even when they are not explicitly designed to consider protected characteristics. Bias testing must be a standard part of your development and deployment pipeline.

  • Pre-deployment testing. Evaluate model outputs across demographic groups (where the use case involves decisions about people) to identify disparate impact.

  • Ongoing monitoring. Bias can emerge over time as input data distributions shift. Implement automated monitoring that flags statistical anomalies in outcomes across relevant groups.

  • Remediation process. When bias is detected, have a documented process for investigating the root cause, implementing corrections, and validating that the fix is effective.

5. Establish an AI Governance Board

Technical governance must be connected to organisational governance. An AI governance board — which may be a standing committee or a function within an existing risk and compliance structure — provides the decision-making authority for:

  • Approving new AI deployments

  • Reviewing risk assessments

  • Setting policies on acceptable AI use cases

  • Responding to incidents and near-misses

  • Liaising with external regulators and auditors

The board should include technical, legal, and business representatives. Governance that lives exclusively within the engineering team will lack the business context to make proportionate decisions. Governance that lives exclusively within legal will lack the technical understanding to be practical.

AI Governance for AI Agents and Agentic Systems

The rise of agentic AI enterprise automation — autonomous agents that plan and execute multi-step tasks — introduces new governance challenges that, in our view, the EU AI Act does not specifically address but that responsible businesses must consider.

AI agents for business process automation are fundamentally different from single-prompt AI features. They make sequential decisions, use tools, access external systems, and can take actions with real-world consequences. Governance for agentic systems requires:

  • Action boundaries. Define what the agent is permitted to do. Can it send emails? Can it modify database records? Can it authorise expenditure? These boundaries must be enforced at the infrastructure level, not just through prompt instructions.

  • Audit trails. Every action an agent takes must be logged with sufficient detail to reconstruct its reasoning chain. This is essential for both regulatory compliance and debugging.

  • Breakpoints. For high-consequence actions, the agent should pause and request human approval before proceeding. The definition of “high-consequence” should be configurable and regularly reviewed.

  • Rollback capability. Where possible, agent actions should be reversible. Design your integration architecture so that an agent’s mistakes can be undone without manual data surgery.

Auditing Existing AI Deployments

If your organisation has already deployed AI features, whether or not they have been formally catalogued, an audit is the essential first step.

Step 1: Discovery

Identify every AI capability in your product and internal tools. Check for AI features that may have been introduced informally — a developer who added GPT-powered summarisation to an internal tool, a data team that built a classification model for customer support tickets, a marketing team using AI to generate content.

Step 2: Classification

Map each discovered AI capability to the EU AI Act risk categories. Be conservative in your classification. If a system is borderline between limited risk and high risk, classify it as high risk and build governance accordingly.

Step 3: Gap Analysis

For each AI system, compare your current governance posture against the requirements for its risk classification. Document the gaps. Gaps to check for include: no technical documentation, no bias testing, no human oversight mechanism, no incident response process, insufficient transparency to end users.

Step 4: Remediation Planning

Prioritise gap remediation based on risk classification and exposure. High-risk systems used by EU customers should be addressed first. Build a realistic timeline — governance remediation is not a weekend project — and allocate engineering resources accordingly.

Building Governance Into the Development Lifecycle

The most effective approach to AI governance is not to bolt it on after deployment but to integrate it into your development lifecycle from the start.

  • Requirements phase. Include governance requirements alongside functional requirements. What risk classification does this feature fall into? What documentation is required? What human oversight is needed?

  • Design phase. Architect the AI feature with governance hooks: logging, confidence scoring, human review workflows, and override mechanisms.

  • Testing phase. Include bias testing, accuracy benchmarking, and adversarial testing alongside functional and performance tests.

  • Deployment phase. Ensure documentation is complete and approved by the governance board before the feature reaches production.

  • Operations phase. Monitor accuracy, fairness, and usage metrics in production. Review and update documentation as the system evolves.

The EU AI Act creates enforceable obligations for businesses that place AI systems on the EU market or whose AI output is used in the EU, with the detail depending on their role and on how each system is classified. The UK’s lighter-touch approach does not eliminate governance requirements — it distributes them across existing regulators and across customer procurement expectations.

For development teams, the practical response is to build governance infrastructure now: system inventories, documentation practices, human-in-the-loop architectures, bias testing pipelines, and organisational governance structures. This investment protects your EU market access, positions you well for future UK requirements, and — most importantly — ensures that your AI systems are reliable, fair, and trustworthy.

McKenna Consultants helps businesses design and implement AI systems with enterprise AI governance built in from the architecture level. Whether you are auditing existing deployments, building new agentic AI capabilities, or preparing for the EU AI Act’s high-risk obligations, which apply from December 2027 for Annex III systems and August 2028 for Annex I systems, we bring the technical governance expertise that development teams need. Get in touch to discuss your requirements.

Sources

Related Articles

Related Case Studies

View all case studies

Have a question about this topic?

Our team would be happy to discuss this further with you.

Get in Touch