AI Governance Glossary

A document — published by an AI vendor — specifying what users may and may not do with the vendor's AI product. AUPs are separate from and additional to your own internal AI policy: they impose vendor-defined restrictions that you must follow as a condition of using the product, and violating them can result in account suspension. Common AUP restrictions include: no use for generating illegal content, no use in certain high-risk applications without prior approval, no systematic scraping, and no use in weapons development. Before deploying an AI tool at scale, your governance process should include reviewing the vendor's AUP for restrictions that conflict with your intended use case.

A legal tech company plans to use an LLM API to automate contract review for their clients. Reading the vendor's AUP reveals that the terms prohibit 'providing legal advice' via the API — a restriction that conflicts with the intended use case, requiring the team to either find a different vendor or restructure the product.

A formal decision by the European Commission determining that a third country, territory, or sector outside the EU provides a level of personal data protection essentially equivalent to the EU standard. Countries with adequacy decisions — such as the UK, Japan, and (with conditions) the US under the EU-US Data Privacy Framework — can receive EU personal data without additional legal safeguards. Without an adequacy decision, EU-to-third-country data transfers require alternative mechanisms such as Standard Contractual Clauses (SCCs) or Binding Corporate Rules (BCRs).

A European startup uses a US-based AI API that processes EU customer data on US servers. Because the US has a partial adequacy arrangement (EU-US Data Privacy Framework), the startup checks that the vendor is DPF-certified before proceeding without SCCs.

AI systems that autonomously plan and execute sequences of actions to complete a goal — browsing the web, writing and running code, sending emails, calling APIs — without a human approving each step. Unlike a chatbot that responds to single prompts, an agentic AI operates over extended time horizons with access to tools and external systems. The EU AI Act has specific provisions for AI systems that operate with 'general purpose' autonomy. The core governance challenge: the fewer human checkpoints in the loop, the harder it is to catch errors before they propagate. Agentic AI deployments require explicit decision boundaries — what actions can the agent take without human approval, and what requires a sign-off?

An agentic AI assistant is given the goal 'book the cheapest flight to London next week' and autonomously searches travel sites, compares prices, and completes the booking — without the user approving each step.

A structured review of an AI system or AI governance program to assess whether it meets defined standards, policies, or legal requirements. AI audits can be internal (conducted by the organization itself) or external (conducted by a third party). A basic internal AI audit covers: which AI tools are in use, what data they access, what controls are in place, whether the AI policy is being followed, and whether any incidents have occurred. External AI audits are increasingly required by regulation and enterprise procurement requirements.

A startup conducts a quarterly AI audit: reviews its tool inventory for changes, checks that each vendor's DPA is still current, reviews the incident log for patterns, and updates the AI policy if new tools have been adopted.

A structured inventory of the components that make up an AI system, analogous to a software bill of materials (SBOM). An AIBOM typically lists: the foundation model(s) used, training datasets and their sources, pre-processing libraries, inference frameworks, third-party APIs, and any post-processing components. AIBOMs help organizations understand what they have built on and where supply chain risks lie. They are emerging as a requirement in enterprise AI procurement, particularly in regulated industries and government contracting.

The branch of ethics concerned with how AI systems should be designed, deployed, and governed to align with human values. AI ethics covers questions like: is it acceptable to use AI for this decision? Are the affected people aware that AI is involved? Is the system fair across different groups? Who is responsible when it goes wrong? AI ethics is distinct from AI law — ethical questions often precede legal ones. Many practices required by regulation today (bias testing, transparency, human oversight) were first articulated as ethical principles years before they became law.

The set of policies, processes, roles, and controls an organization puts in place to ensure its AI systems are used responsibly, legally, and in line with its values. AI governance covers the full lifecycle: how AI tools are evaluated and approved, how data is handled, how outputs are reviewed, how incidents are tracked, and how the program is updated over time. For small teams, AI governance is not a compliance checkbox — it is operational infrastructure that reduces risk, builds trust with customers, and prepares the organization for regulation that is coming whether or not it has arrived yet.

A 20-person company implements AI governance by: maintaining an AI tool inventory, requiring approval for new tools, running quarterly vendor reviews, and keeping an incident log — all without a dedicated compliance team.

An event in which an AI system produces outputs or takes actions that cause harm, near-miss harm, or significant unexpected behavior. AI incidents range from minor (a chatbot gives an unhelpful answer) to severe (an AI medical system recommends a dangerous drug interaction). Under the EU AI Act, providers of high-risk AI systems must report serious incidents to national market surveillance authorities within 15 days of becoming aware. Maintaining an internal AI incident log — even before legal requirements apply — enables pattern identification, demonstrates due diligence, and prepares organizations for regulatory reporting obligations.

An AI scheduling system double-books a critical team member for two client calls simultaneously due to a timezone parsing error. This is an AI incident — logged with the date, tool, what happened, business impact, and what was done to prevent recurrence.

A public repository (incidentdatabase.ai) that catalogs real-world cases of AI systems causing harm or behaving unexpectedly. Maintained by the Responsible AI Collaborative, it contains thousands of documented incidents — from self-driving car crashes to hiring algorithm bias to chatbot harassment — each tagged by harm type, AI system involved, and affected population. The database is a practical resource for governance teams: browsing incidents in your sector reveals the failure modes most likely to occur, informing your risk assessment and pre-deployment testing priorities. The EU AI Act's incident reporting obligations will generate additional structured incident data over time.

Before deploying an AI hiring tool, a team searches the AI Incident Database for 'recruitment' incidents, finds 15 documented cases of algorithmic bias in hiring, and uses the failure patterns to design targeted bias tests for their own system.

A documented list of all AI tools and systems in use within an organization. An effective AI inventory records: tool name, vendor, what the tool does, who uses it, what data it accesses, the vendor's data handling terms, and who is responsible for governance. The AI inventory is the foundation of any AI governance program — you cannot govern what you have not catalogued. It should be updated whenever new tools are adopted and reviewed quarterly. Many AI regulations, including the EU AI Act, require organizations to maintain documentation of the AI systems they deploy.

A company's AI inventory shows it uses 12 AI tools across 8 teams. Three of these tools are used only by individual employees and were never formally approved — these become shadow AI risk items for the governance lead to address.

The legal question of who bears responsibility when an AI system causes harm — financial loss, physical injury, discrimination, privacy violations, or other damages. AI liability is unsettled law in most jurisdictions. The EU's AI Liability Directive (proposed) aims to make it easier for victims of AI harm to claim compensation. The EU Product Liability Directive revision extends product liability to software and AI. In practice, liability questions for small teams usually arise in contracts: does your vendor indemnify you if their AI causes a breach? Do your customer contracts limit your liability for AI-generated errors?

A law firm's AI research assistant cites a non-existent case in a court filing. The question of whether the law firm, the AI vendor, or both bear liability for the resulting sanctions is an AI liability question.

The ability to understand how AI systems work at a level sufficient to use them effectively, evaluate their outputs critically, and recognize their limitations. The EU AI Act requires providers and deployers of certain AI systems to ensure users have adequate AI literacy to use the system safely. For small teams, AI literacy means every employee who uses AI tools should understand: that outputs can be wrong, that AI does not 'know' anything, that human review is their responsibility, and that the organization has policies governing appropriate use.

A company runs a 30-minute AI literacy session with all new hires, covering what LLMs can and cannot do, what data is off-limits, and how to report unexpected outputs.

A pause or suspension of AI development or deployment — either voluntary or legally mandated — to allow time for safety evaluation, regulatory design, or societal deliberation. Calls for AI moratoriums have come from both safety researchers (concerned about near-term misuse) and civil society groups (concerned about AI's effect on labor markets, privacy, and democratic processes). No major jurisdiction has enacted a binding AI moratorium, though some legislatures have proposed temporary pauses on specific high-risk applications like facial recognition in public spaces.

The end-to-end sequence of steps that transforms raw inputs into final AI-generated outputs, including data ingestion, preprocessing, model inference, post-processing, and delivery to end users. In production systems, an AI pipeline may involve multiple models, tools, APIs, and human review checkpoints working in sequence. Governing an AI pipeline means mapping each step: where does data come from, which models touch it, what can go wrong at each stage, and where is the human-in-the-loop?

A content team's AI pipeline: web scraping → topic classification model → draft generation (LLM) → human editor review → publish. Each step is a governance checkpoint.

A written document that defines how an organization's employees may and may not use AI tools. An effective AI policy specifies: which tools are approved, which are prohibited, what data each tool can access, where human review is required before acting on AI output, and what to do when something goes wrong. AI policies exist at two levels: organizational (internal rules for a company) and governmental (laws and regulations set by governments). Most governance discussions for small teams focus on organizational AI policy — the internal document that governs day-to-day AI use.

A startup writes a one-page AI policy specifying that ChatGPT may be used for internal drafts but not for external client communications, and that no customer data may be pasted into any AI tool not covered by a signed DPA.

The policies, processes, and contractual standards an organization uses when buying AI products or AI-containing services. AI procurement is distinct from general software procurement because AI products raise additional questions: what data will the vendor use for training? What bias testing has been done? Who is liable if the AI makes a discriminatory decision? Government AI procurement is a rapidly evolving area — the US Office of Management and Budget and the EU have both issued AI procurement guidance requiring agencies to assess AI risks before purchase. For private organizations, AI procurement governance means having a defined evaluation checklist, contract protections, and approval process before any new AI tool is adopted.

A local government's AI procurement policy requires any AI vendor to complete a security questionnaire, provide a model card, sign a DPA, and agree that the government's data will not be used for training before the contract is signed.

An assessment of how prepared an organization is to adopt, govern, and benefit from AI responsibly. AI readiness considers: whether leadership has defined an AI strategy, whether there are clear policies governing AI use, whether the team has sufficient AI literacy, whether data infrastructure can support AI workloads, whether governance processes (approval, monitoring, incident response) exist, and whether the organization understands its regulatory obligations. AI readiness frameworks are used by consultancies, regulators, and industry bodies to help organizations self-assess before AI adoption. The NIST AI RMF's 'Govern' function maps closely to readiness — governance infrastructure is what makes AI adoption sustainable rather than reactive.

Before piloting an AI customer support tool, a team completes an AI readiness assessment and discovers it lacks an incident response process and has no signed DPA with its current AI vendor — two gaps it closes before expanding AI use.

The process of identifying, evaluating, and prioritizing risks associated with an AI system or use case. A structured AI risk assessment asks: what could go wrong with this AI system? How likely is each failure mode? How severe would the harm be? What controls are in place to mitigate the risk? Risk assessments are required by the EU AI Act for high-risk AI systems, by the Colorado AI Act for consequential decisions, and recommended by NIST AI RMF for any AI deployment. For small teams, a one-page risk assessment for each high-stakes AI use case is a practical standard.

The field concerned with ensuring AI systems behave as intended and do not cause unintended harm. AI safety spans two distinct concerns: near-term safety (preventing current AI systems from making errors, being misused, or causing harm in deployment) and long-term safety (preventing advanced AI systems from developing goals misaligned with human values). For small teams, near-term AI safety is the relevant concern: testing outputs before deployment, implementing human review for high-stakes decisions, maintaining incident logs, and updating systems when problems emerge.

The chain of third-party providers, models, datasets, APIs, and infrastructure components that an AI system depends on to function. A typical AI product sits on layers of foundation models, fine-tuning services, embedding providers, data pipelines, and cloud infrastructure — each governed by different terms of service, carrying different security and privacy risks, and subject to unilateral changes by their providers. Supply chain risk in AI includes model update surprises, provider shutdowns, API deprecation, data source contamination, and cascading compliance failures when an upstream component changes its privacy practices.

An AI startup's customer-facing product depends on: OpenAI's API (foundation model), a vector database hosted by a European provider, a fine-tuning service from a US startup, and cloud infrastructure on AWS. A change to any one of these can affect the product's behavior, data handling, or compliance posture.

Systematic and repeatable errors in an AI system's outputs that create unfair outcomes for particular groups — based on race, gender, age, disability, or other characteristics. Algorithmic bias typically originates in training data that underrepresents or misrepresents certain groups, but can also emerge from problem framing, feature selection, or optimization objectives. Regulators in the US (EEOC, FTC), EU (AI Act), and several states (Colorado, Illinois) have flagged algorithmic bias as a primary AI risk. Testing for disparate impact across demographic groups before deployment is the standard mitigation.

An AI loan approval system approves applications from Black applicants at a 40% lower rate than white applicants with the same financial profile. This is algorithmic bias — likely caused by training on historical loan data reflecting past discriminatory practices.

A structured process for evaluating the potential harms, benefits, and societal effects of an AI or algorithmic system before and during deployment. AIAs are required or recommended by a growing number of AI regulations and frameworks, including Canada's Directive on Automated Decision-Making and the Colorado AI Act (for consequential decisions). An AIA typically covers: system description, intended use and affected populations, potential risks and mitigation measures, governance structures, and ongoing monitoring plans. For small teams, a lightweight AIA for each high-stakes AI use case is a practical starting point.

Before deploying an AI system to screen job applications, a company conducts an algorithmic impact assessment covering potential bias against protected groups, the accuracy of the screening criteria, and the appeals process for rejected candidates.

Processing that uses personal data to evaluate, analyze, or predict aspects of an individual's circumstances, behavior, or attributes, with no meaningful human involvement in reaching or implementing the decision. GDPR Article 22 grants data subjects the right to opt out of purely automated decisions that produce legal or similarly significant effects — such as credit decisions, employment screening, or benefits eligibility. The EU AI Act reinforces these protections by classifying automated decision-making in high-stakes domains as high-risk AI, requiring human oversight mechanisms.

A lending platform uses an AI model to approve or reject loan applications automatically. Because this constitutes automated decision-making with significant legal effects, GDPR Article 22 requires that applicants be able to request human review of any rejection.

A program — analogous to a bug bounty in cybersecurity — that rewards external researchers, affected community members, or the general public for discovering and reporting algorithmic bias or harmful behavior in an AI system. Bias bounties operationalize the idea that internal testing cannot anticipate every failure mode, especially for harms affecting groups not represented in the testing team. Twitter/X and Lensa AI have run bias bounty programs. The practice is emerging as a voluntary governance mechanism and may eventually be referenced in regulation. For small teams, participating in an external red-team or bias testing exercise with diverse testers serves a similar function at lower cost.

An AI hiring company launches a bias bounty program, offering $500–$5,000 to external researchers who can demonstrate that its resume screening model systematically disadvantages applicants from specific universities or demographic groups.

Systematic testing of what an AI model can and cannot reliably do across the range of tasks and inputs it will encounter in production. Capability evaluations measure accuracy, reasoning quality, language abilities, handling of edge cases, and performance degradation under distribution shift. Distinct from safety evaluation — which assesses what the model should not do — capability evaluation focuses on whether the model is competent enough for its intended use. Published benchmarks rarely reflect real-world task distributions, making task-specific capability evaluation essential before deployment.

Before deploying a contract analysis AI, a legal team runs capability evaluations using 50 real contract samples, measuring the model's accuracy on clause identification and flagging — finding that performance drops 30% on contracts longer than 20 pages.

The California Consumer Privacy Act — the most comprehensive US state privacy law, covering residents of California. It gives consumers rights to know what personal data is collected, request deletion, opt out of sale, and non-discrimination for exercising these rights. The California Privacy Rights Act (CPRA) amended CCPA in 2023 to add rights related to automated decision-making and sensitive personal information. Businesses subject to CCPA must disclose AI-driven profiling and provide opt-out mechanisms for certain automated decisions.

A California consumer can request that a company disclose whether it uses AI to evaluate their credit application and can opt out of certain AI-driven profiling under CCPA.

A certification mark required on products sold in the European Economic Area, indicating that the product meets EU safety, health, and environmental standards. Under the EU AI Act, high-risk AI systems must bear the CE marking before they can be sold or deployed in the EU. Affixing the CE mark requires completing a conformity assessment, preparing technical documentation, and registering in the EU database for high-risk AI systems. The CE mark signals to buyers that the provider has taken legal responsibility for the system's compliance.

The process by which an AI system is evaluated to confirm it meets the requirements of the EU AI Act before being placed on the market. For most high-risk AI systems, providers can self-certify via an internal conformity assessment. For certain categories (biometric systems, safety components of critical infrastructure), third-party assessment by a 'notified body' is required. The assessment must verify that the system meets requirements for risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy, and cybersecurity.

A core GDPR principle (Article 5(1)(c)) requiring that personal data collected and processed be limited to what is strictly necessary for the specified, explicit, and legitimate purpose. For AI systems, data minimization means not feeding models unnecessary personal attributes, not logging full personal records in inference pipelines indefinitely, anonymizing or pseudonymizing data at the earliest feasible stage, and using synthetic data for development and testing where full personal records are not required. Data minimization is both a legal obligation and an effective risk reduction strategy — data you do not collect cannot be breached, leaked, or misused.

A churn prediction model is retrained to use product usage patterns and support ticket frequency — removing age, location, and job title from the feature set after analysis showed they added marginal predictive value but created unnecessary GDPR data minimization risk.

A contract between a data controller (the organization that determines how data is used) and a data processor (a vendor that processes data on the controller's behalf). Under GDPR, a DPA is legally required whenever a vendor processes personal data on your behalf. For AI tools, the DPA governs what data the vendor can access, how long they retain it, whether they can use it for training, and what security standards they must meet. Reviewing your AI vendors' DPAs is a core vendor due diligence step.

Before deploying an AI writing tool that processes customer emails, a team reviews the vendor's DPA to confirm: the vendor will not use customer data for model training, data is deleted within 30 days, and EU data stays in the EU.

AI systems or techniques designed — intentionally or by inadvertent design — to manipulate, mislead, or exploit users through means that bypass rational agency. This includes subliminal messaging, false urgency, exploitation of emotional states, dark patterns, synthetic media impersonating real people (deepfakes), and AI-generated disinformation. The EU AI Act explicitly prohibits AI systems that deploy subliminal techniques, exploit psychological or physical vulnerabilities, or manipulate behavior in ways that cause harm. Generative AI significantly lowers the cost of producing deceptive content at scale.

An AI-powered notification system is tuned to maximize engagement by detecting user emotional states and sending messages during moments of anxiety — a form of deceptive AI exploitation that is explicitly prohibited under the EU AI Act.

A formal document in which the provider of a high-risk AI system declares that the system complies with the EU AI Act. By signing the Declaration of Conformity, the provider takes legal responsibility for compliance. The declaration must be kept on file and made available to authorities. It is distinct from a conformity assessment (the process of checking compliance) — the declaration is the formal statement that the check has been completed and the system complies. CE marking can only be affixed after the declaration is signed.

A mathematical framework for adding carefully calibrated statistical noise to data or query results, so that no individual record can be reliably inferred from the output. A system satisfies differential privacy if its outputs are nearly identical whether or not any single person's data is included — providing a formal, quantifiable privacy guarantee. Differential privacy is used in AI to allow training on sensitive datasets (health, financial, location) while limiting what can be learned about any individual contributor to that dataset.

A company trains a salary prediction model using differential privacy, so that no individual employee's salary can be reverse-engineered from the model's outputs, even by someone with access to most of the training data.

A virtual replica of a physical object, process, or system — kept synchronized with the real-world counterpart via sensors, data feeds, or simulation — used for monitoring, analysis, and optimization. AI is increasingly embedded in digital twins to predict failures, optimize operations, and run 'what if' simulations without risk to the real system. Digital twins appear on the EU AI Act's list of high-risk AI applications when used in critical infrastructure (energy grids, water systems, transport). From a governance perspective, digital twins that inform real-world decisions need the same documentation, accuracy assurance, and human oversight as any high-stakes AI system — data quality in the twin directly determines the quality of decisions made from it.

A manufacturing company runs a digital twin of its production line, using AI to predict equipment failures 48 hours in advance. The twin's predictions trigger maintenance work orders — making the AI's accuracy and the quality of sensor data governance-critical.

A numerical representation of text, images, or other data as a dense vector in high-dimensional space. Embeddings allow AI systems to measure semantic similarity — two pieces of text with similar meaning will have similar embeddings, even if the words differ. Embeddings are widely used in search, recommendation systems, and RAG architectures. From a governance perspective, embedding databases may contain encoded representations of sensitive data (customer records, private documents) and should be treated as data stores requiring the same access controls as the original data.

The European Union's comprehensive AI regulation (Regulation 2024/1689/EU), the world's first binding legal framework specifically for artificial intelligence. The Act classifies AI systems by risk level — prohibited, high-risk, limited-risk, and minimal-risk — and imposes obligations that scale with risk level. High-risk AI systems must undergo conformity assessment, maintain technical documentation, implement human oversight, and register in an EU database before deployment. Full enforcement begins August 2, 2026. The Act applies to any AI system deployed in the EU, regardless of where the developer is based.

A US startup selling an AI-powered recruitment tool to European companies must comply with the EU AI Act's high-risk requirements, even though the company is based in the United States.

The ability to describe how an AI system arrived at a particular output or decision in terms that humans can understand. Explainability is legally required in some contexts (GDPR Article 22 gives individuals the right to an explanation for automated decisions that significantly affect them) and operationally important for debugging, auditing, and building user trust. There is a spectrum from full transparency (the model's reasoning is completely legible) to post-hoc explanation (a separate model generates an explanation after the fact). Most large language models are not inherently explainable — they produce outputs without exposing their reasoning process.

A legislative provision in which a federal AI law supersedes state AI laws, preventing states from enacting their own AI regulations. Preemption is a central debate in US AI policy: federal preemption would create a single national standard (simplifying compliance for multi-state businesses) but may set a lower floor than some states would prefer. Weak or no preemption allows states to experiment and fill federal gaps, but creates a patchwork of requirements that increases compliance complexity. Several AI industry groups have lobbied for strong federal preemption as part of their opposition to state-level AI legislation.

A company operating in all 50 states must currently track AI laws in Colorado, California, Illinois, New York, Texas, and Virginia separately. A federal preemption provision would replace all of these with a single compliance standard.

A machine learning technique in which a model is trained across multiple devices or servers without the raw training data ever leaving those devices. Instead of centralizing data, federated learning sends the model to the data, collects local model updates (gradients), and aggregates them centrally — preserving the privacy of individual data sources. It is widely used in healthcare (where patient data cannot be shared across hospitals), finance (where transaction data is sensitive), and mobile devices (where user data stays on-device). From a governance perspective, federated learning reduces data privacy risk but creates new model governance challenges: the training process is distributed and harder to audit.

A healthcare consortium trains a disease detection model using federated learning: each hospital trains on its own patient data locally, and only the model improvements — not patient records — are shared with the central coordinator.

The process of further training a pre-trained foundation model on a smaller, task-specific dataset to adapt its behavior for a particular use case. Fine-tuning is cheaper than training from scratch and often produces better results for specialized tasks. From a governance perspective, fine-tuning raises questions about data provenance (what data was used?), liability (if the fine-tuned model produces harmful outputs, who is responsible?), and EU AI Act classification (fine-tuning can change a model's risk category).

A legal tech startup fine-tunes an open-source language model on a corpus of case law. Under the EU AI Act, this may constitute providing a GPAI model, triggering documentation obligations.

A large AI model trained on broad data at scale, designed to be adapted for a wide range of downstream tasks. Foundation models — such as GPT-4, Claude, or Gemini — learn general capabilities during training, then get specialized through fine-tuning or prompting. Most AI products you buy or build today sit on top of a foundation model developed by a third party. This creates a dependency risk: your product's behavior is partly determined by a model you do not control.

A startup building an AI contract review tool uses Claude as its foundation model, meaning it inherits both Claude's capabilities and Anthropic's data handling practices.

A structured evaluation required by the EU AI Act (Article 27) for deployers of high-risk AI systems, assessing the potential impact of the AI system on fundamental rights — including privacy, non-discrimination, freedom of expression, equal treatment, access to justice, and other rights protected under the EU Charter of Fundamental Rights. The assessment must be conducted before deployment and must be documented, stored, and made available to market surveillance authorities on request. It overlaps with but extends beyond a GDPR Data Protection Impact Assessment (DPIA), covering rights that are not purely data-related.

A local government deploying an AI tool to prioritize social benefit payments conducts a fundamental rights impact assessment, identifying that the system may disadvantage non-native language speakers — and modifying the input data pipeline to correct this before launch.

The General Data Protection Regulation (EU 2016/679) — the EU's core data privacy law. GDPR governs how organizations collect, process, store, and share personal data of EU residents. It requires a lawful basis for processing, data minimization, purpose limitation, and individuals' rights to access and erasure. AI systems that process personal data are subject to GDPR, including AI used for profiling, automated decision-making, or training on personal data. Article 22 specifically restricts purely automated decision-making that significantly affects individuals.

A company using an AI tool to analyze employee productivity data must comply with GDPR, including notifying employees, establishing a lawful basis for processing, and limiting data retention.

General Purpose AI Model — the EU AI Act's formal term for a foundation model trained on large amounts of data and capable of performing a wide variety of tasks. The EU AI Act imposes specific obligations on GPAI model providers, including documentation requirements and cooperation with downstream deployers. If you build products on top of an API like OpenAI or Anthropic, you are a 'deployer' of a GPAI model, not a GPAI model provider — a distinction that significantly affects your compliance obligations.

OpenAI's GPT-4o is a GPAI model under the EU AI Act. A company that builds a customer service chatbot using the GPT-4o API is a deployer, not a GPAI model provider.

When an AI language model generates plausible-sounding but factually incorrect or entirely fabricated information — with no reliable signal to the user that it is wrong. The model does not 'know' it is hallucinating; it generates the most statistically likely next token given its training. Hallucinations are a core risk in any AI deployment involving factual claims, legal analysis, medical information, financial guidance, or customer-facing communications. Governance controls for hallucination typically include human review requirements, RAG-based grounding, and output verification steps.

A chatbot confidently cites a legal case that does not exist. This is a hallucination — the model generated a plausible-sounding citation with no factual basis.

A category defined by the EU AI Act for AI systems that pose significant risks to health, safety, or fundamental rights. High-risk systems include: AI used in critical infrastructure, education, employment (hiring, promotion, performance evaluation), essential services (credit, insurance, emergency services), law enforcement, migration and asylum decisions, and administration of justice. High-risk AI systems face the Act's strictest requirements: conformity assessment, technical documentation, risk management systems, data governance measures, logging, transparency, human oversight, accuracy, and cybersecurity. Annex III of the EU AI Act lists the specific categories.

An AI system used by a company to rank job applicants falls under the EU AI Act's high-risk category (employment decisions). The company must conduct a conformity assessment, maintain technical documentation, and implement human oversight before deploying it in the EU.

The requirement that a human being can monitor, review, intervene in, or override an AI system's outputs or decisions. Human oversight is a core requirement of the EU AI Act for high-risk AI systems and a central principle of responsible AI frameworks globally. In practice, human oversight means designing AI workflows so that a person reviews outputs before they are acted on, can detect errors, and has the authority to correct or reject AI-generated decisions. 'Human in the loop' is the term for workflow designs where human review happens before action; 'human on the loop' refers to review after action with the ability to intervene.

A company's AI-generated contract summary is always reviewed by a lawyer before being sent to the client. The lawyer is the human oversight step — they can catch errors before they become contractual commitments.

The process of documenting and reporting AI system failures, errors, unexpected outputs, or harms. At the organizational level, incident reporting means maintaining a log of AI failures and near-misses to identify patterns and improve governance. At the regulatory level, the EU AI Act requires providers of high-risk AI systems to report 'serious incidents' — those causing death, serious injury, significant property damage, or grave fundamental rights violations — to national authorities. Building an incident log before it is legally required is one of the highest-value AI governance investments a small team can make.

A team's AI customer service bot gives a refund amount that contradicts the company's policy. The customer service manager logs it in the incident log: date, tool, what happened, how it was corrected, and whether the model configuration needs updating.

The process of running a trained AI model to generate predictions or outputs from new inputs. In contrast to training (which adjusts model weights), inference uses a fixed model to process live requests. Most AI governance concerns arise at inference time: what data is being sent to the model, what outputs are being generated, who is seeing those outputs, and how are they being acted on? Inference costs (compute, API calls) and inference latency also affect which models are practical to deploy.

One of six lawful bases under GDPR Article 6 for processing personal data without requiring explicit consent. Processing on the basis of legitimate interest must satisfy a three-part test: the interest must be genuine and specific; the processing must be necessary for that interest; and the controller's interest must not be overridden by the data subject's rights and freedoms. AI use cases commonly relying on legitimate interest include fraud detection, network security, internal analytics, and product improvement. The legitimate interest basis requires a documented balancing test and cannot be applied retroactively if challenged.

A B2B SaaS company uses customer usage data to train a churn-prediction model, relying on legitimate interest as the legal basis — after documenting that the business interest in reducing churn outweighs the minimal privacy impact on users who have opted into the service.

A category in the EU AI Act for AI systems that pose limited risk and face lighter compliance obligations — primarily transparency requirements. Limited-risk systems include chatbots (users must be told they are interacting with AI), AI that generates synthetic content (must be labeled as AI-generated), and AI that manipulates images or video (deepfakes must be disclosed). Most commercial AI applications — customer service bots, AI writing tools, AI image generators — fall in this category. Limited-risk obligations are achievable without significant compliance overhead.

A company's AI customer service chatbot is a limited-risk AI system. The EU AI Act requires that users be clearly informed they are talking to an AI — a disclosure notice at the start of the conversation satisfies this obligation.

A short document accompanying a trained AI model that describes its intended use, evaluation results, performance characteristics, limitations, and ethical considerations. Model cards were introduced by Google researchers in 2018 and have since become an industry standard for AI transparency. Major AI providers (Hugging Face, Google, Anthropic) publish model cards for their models. For teams deploying AI, reading the model card helps identify known failure modes, bias characteristics, and use cases the model was not designed for — all relevant to risk assessment.

The policies, processes, and controls that manage AI models across their full operational lifecycle — from initial selection and approval through deployment, monitoring, versioning, and retirement. Model governance defines who can authorize a model for a given use case, how updates and new model versions are evaluated and promoted, what performance thresholds trigger a review or rollback, and who is accountable for model outputs in production. Without model governance, organizations accumulate unapproved models, lose track of which models are in use, and lack a process for handling degraded or harmful model behavior.

A fintech company implements model governance by requiring that any model touching credit decisions be approved by the risk committee, version-controlled in a model registry, and reviewed quarterly for performance drift.

A third-party conformity assessment organization accredited by an EU member state to audit, test, and certify AI systems before they may be placed on the EU market as high-risk AI. Notified bodies review technical documentation, conduct audits, test system performance, and issue EU type-examination certificates confirming that a system meets the EU AI Act's requirements. Not all high-risk AI systems require third-party notified body assessment — some allow self-declaration of conformity — but those in regulated sectors (medical devices, critical infrastructure, biometrics) typically do.

A medical device company embedding AI-based diagnostics into its product must engage an EU notified body to audit its technical documentation and test the AI component before applying CE marking and launching in the EU.

The tendency for users or operators to defer to AI outputs without sufficient critical evaluation, even when the system is operating outside its area of competence, hallucinating, or producing subtly incorrect guidance. Over-reliance is exacerbated by high-confidence AI presentations, authoritative language, and speed — when an AI responds instantly with apparent certainty, users naturally discount the need to verify. It is a governance failure mode distinct from the AI producing errors: the AI may be behaving exactly as designed while the human fails to apply appropriate skepticism.

A customer support team using AI-generated response drafts stops checking citations after a few weeks of good performance. When the model's underlying knowledge becomes stale, support agents continue trusting and sending incorrect information — not because the AI suddenly changed, but because the team stopped reviewing.

The ongoing collection, analysis, and reporting of real-world performance data after an AI system has been deployed. Required by the EU AI Act (Article 72) for providers of high-risk AI systems, post-market monitoring aims to detect performance degradation, emerging biases, unexpected behaviors, and safety incidents that were not apparent during pre-deployment testing. Providers must maintain a post-market monitoring plan and report serious incidents to national authorities within defined timeframes (24 hours for serious incidents involving safety risks). The broader AI governance practice refers to this as model monitoring or model drift detection.

An HR software vendor deploys an AI shortlisting tool and implements post-market monitoring, tracking weekly accuracy metrics and demographic pass-rate parity. Six months post-launch, the monitoring detects a 15% drop in accuracy for a specific job category — triggering a model review before a client notices.

AI systems that are outright banned by the EU AI Act because their risks are deemed unacceptable regardless of any benefits. Prohibited uses include: subliminal manipulation systems that bypass conscious decision-making, AI exploiting vulnerabilities of specific groups, social scoring by public authorities, real-time remote biometric identification in public spaces (with narrow exceptions), biometric categorization to infer sensitive attributes, and predictive policing based solely on profiling. These prohibitions applied from August 2024. Organizations deploying AI should review use cases against the prohibited list as a first governance step.

A structured adversarial testing process in which a team deliberately attempts to make an AI system fail — by crafting inputs that cause harmful, biased, or unintended outputs. Red-teaming borrows from cybersecurity practice (where a 'red team' simulates attackers). In AI, red-teamers probe for harmful content generation, jailbreaks, prompt injection, bias amplification, and factual failures. The EU-US voluntary AI commitments and the NIST AI RMF both reference red-teaming as a baseline safety practice. Major AI labs (Anthropic, OpenAI, Google) conduct red-teaming before model releases. For small teams deploying AI, a lightweight red-teaming exercise — trying to break your own system before users do — is a practical pre-deployment step.

Before launching an AI customer service bot, a team spends two hours trying to make it give incorrect refund amounts, reveal internal pricing, or produce offensive responses. This red-teaming exercise surfaces a prompt injection vulnerability that is fixed before launch.

A term used by critics to describe a political strategy in which technology companies invoke the scale and novelty of AI to argue that existing regulations are inadequate, while simultaneously lobbying against new AI-specific legislation — producing a regulatory vacuum. The argument pattern: 'This is so transformative that old rules don't fit, and new rules need careful study before enactment.' Critics argue this leaves affected parties with no protection while the technology accelerates. The term gained prominence in AI policy debates around 2024-2026.

Critics described OpenAI's 2026 'Industrial Policy for the Intelligence Age' paper as regulatory nihilism: proposing sweeping hypothetical federal solutions while lobbying against near-term enforceable state laws.

A controlled environment established by a regulator that allows organizations to test innovative products or AI systems under real conditions with relaxed regulatory requirements. The EU AI Act requires each EU member state to establish at least one AI regulatory sandbox by August 2026. Sandboxes are particularly useful for startups and SMEs developing AI systems that might otherwise face regulatory barriers before they have the resources to achieve full compliance. Participation typically involves close regulator supervision and does not guarantee a compliance pass.

A health tech startup developing an AI diagnostic tool applies to participate in the Netherlands' AI regulatory sandbox, allowing them to test the system with real patient data under regulatory guidance before full compliance is required.

A framework or set of principles guiding the development and deployment of AI in ways that are ethical, fair, accountable, and beneficial. Common responsible AI principles include fairness (no unjustified discrimination), transparency (explainable decisions), accountability (clear ownership of outcomes), privacy (minimal data use), reliability (consistent performance), and safety (avoiding harm). Many organizations publish responsible AI principles as public commitments. For small teams, responsible AI is most useful as a practical checklist: for each AI deployment, ask whether the use meets each principle before proceeding.

A technique that combines a language model with a search system. When a user asks a question, the system first retrieves relevant documents from a database, then feeds those documents to the language model to generate an answer grounded in the retrieved content. RAG reduces hallucinations and allows AI systems to work with private or up-to-date information without retraining the model. From a governance perspective, RAG systems require clear data access policies — the retrieval database determines what information the AI can see and use.

A company deploys a RAG system that lets employees query their internal HR policy documents using natural language. The AI can answer questions accurately because it retrieves relevant policy sections before generating a response.

A person's legal right to receive a meaningful, human-understandable explanation of an automated decision that significantly affects them — covering the logic involved, its significance, and its likely consequences. Established under GDPR Articles 13–15 and 22, and reinforced by the EU AI Act's transparency requirements for high-risk AI systems. The right applies to any automated decision that produces legal effects or similarly significantly affects the data subject, such as credit refusals, job application rejections, insurance denials, and content moderation actions.

A job candidate whose application was rejected by an AI screening tool submits a GDPR access request asking for an explanation. The company must provide a meaningful description of the AI's decision logic — 'the algorithm scored you low' is not sufficient.

The ability of an AI system to perform reliably across a wide range of conditions — including inputs the system was not specifically trained on, adversarial inputs designed to cause failures, edge cases, and environmental variations. A robust AI system degrades gracefully when faced with unexpected inputs rather than failing catastrophically. The EU AI Act requires high-risk AI systems to be accurate, robust, and cybersecure. For small teams, robustness testing means evaluating AI tools on realistic edge cases before deployment, not just on the ideal-case scenarios shown in vendor demos.

A legal provision that limits liability for organizations that follow specific procedures or meet defined standards. In AI governance, safe harbor provisions are emerging in several US state AI laws, offering protection to companies that conduct risk assessments, implement oversight mechanisms, or follow recognized industry standards. The EU AI Act's conformity with harmonized standards creates a form of safe harbor — following the standard creates a presumption of compliance. Safe harbor provisions are important for small teams because they often provide a practical compliance path without requiring expensive legal analysis.

AI tools adopted by employees without official organizational approval, review, or knowledge. Shadow AI is the AI equivalent of shadow IT — it proliferates when employees find AI tools useful but the official approval process is slow or unclear. Shadow AI creates governance gaps: sensitive data may be sent to unapproved vendors, organizational AI policies are bypassed, and the AI inventory is incomplete. Preventing shadow AI requires a combination of clear AI policies, fast approval processes for new tools, and regular audits of actual tool usage versus approved tools.

A marketing team starts using an AI image generator without telling IT or legal. The tool's terms of service include a clause allowing it to use uploaded images for model training. This is shadow AI creating a data governance gap.

Artificially generated data that mimics the statistical properties of real-world datasets without containing actual personal or sensitive records. Synthetic data is produced using generative models, statistical sampling, or rule-based simulation. It is used to train and test AI systems in privacy-sensitive domains — such as healthcare, finance, and HR — where using real personal data would create legal or ethical risk. While synthetic data reduces privacy exposure, it can introduce or amplify biases from the original data it was generated from, requiring careful validation before use.

A health startup uses synthetic patient records to train a diagnostic AI model, avoiding the need to access real patient data during development — while still validating the model against a small real-world test set before deployment.

Under the EU AI Act, a mandatory set of records that high-risk AI system providers must compile before market deployment. Required content includes: a detailed description of the system and its intended purpose, development methodology and training data, performance metrics and testing results, risk management measures, human oversight capabilities, and instructions for use. Technical documentation must be kept up to date throughout the system's lifecycle and made available to market surveillance authorities on request. For small teams building on third-party models, this means documenting how you configured and deployed the model, not just the model itself.

An independent assessment of an AI system or organization's AI governance practices conducted by an external party. Third-party audits are more credible than self-assessments because the auditor has no conflict of interest. They are required for certain high-risk AI systems under the EU AI Act, increasingly requested in enterprise AI contracts, and used by regulators to verify compliance. For small teams, conducting a third-party audit may be cost-prohibitive in the short term — but being auditable (maintaining the documentation an auditor would need) is achievable and builds toward audit-readiness.

The process of breaking text into smaller units called tokens, which language models use as their basic unit of processing. A token is typically a word fragment — 'governance' might be split into 'govern' and 'ance'. Token counts determine AI API pricing, context window limits, and processing speed. Understanding tokenization helps teams estimate costs, set rate limits, and ensure that long documents are handled correctly by AI systems that have maximum context lengths.

A team's AI document analysis tool hits the model's 128,000-token context limit when processing long contracts, causing truncation errors. Understanding tokenization helps them design a chunking strategy.

The dataset used to train an AI model. The quality, composition, and provenance of training data fundamentally shape what a model can do and how it behaves. Training data issues — including biases in the dataset, inclusion of copyrighted material, or presence of personal data — can create downstream legal and ethical liability for AI developers. The EU AI Act requires GPAI model providers to document their training datasets, including sources and any copyright opt-out processes used.

A company discovers its AI hiring tool was trained partly on historical hiring data that underrepresented women in technical roles. The training data bias translates directly into discriminatory recommendations.

In AI governance, transparency means being open about how an AI system works, what data it uses, what it can and cannot do, and when it is being used. Transparency operates at multiple levels: system transparency (how the AI works, disclosed to regulators and auditors), interaction transparency (users are told when they are interacting with AI), and output transparency (AI-generated content is labeled as such). The EU AI Act and multiple US state laws impose transparency requirements. Internally, transparency means your team understands what AI tools are in use and what decisions they influence.

The EU's framework for AI that is lawful, ethical, and robust. Published by the High-Level Expert Group on AI in 2019, the Trustworthy AI framework established seven key requirements: human agency and oversight, technical robustness and safety, privacy and data governance, transparency, diversity and non-discrimination, societal and environmental wellbeing, and accountability. These principles formed the foundation for the EU AI Act's design. Many corporate AI governance frameworks are structured around the same seven dimensions.

The process of evaluating an AI vendor before adopting their product and on an ongoing basis, to assess security, privacy, legal compliance, and reliability. AI vendor due diligence covers: data handling and retention policies, security certifications, subprocessor lists, incident response procedures, model training policies (is your data used?), contractual protections, and financial stability. The EU AI Act requires deployers of high-risk AI systems to verify that the AI they purchase complies with the Act's requirements. Even outside regulated contexts, vendor due diligence protects against supply chain risk.

Techniques for embedding invisible signals in AI-generated content — text, images, audio, video — that identify it as machine-generated and trace it to a specific model or organization. The EU AI Act requires that synthetic content (deepfakes, AI-generated media) be labeled as AI-generated. The Coalition for Content Provenance and Authenticity (C2PA) standard is the emerging industry protocol for embedding cryptographic provenance metadata in media files. For marketing and content teams, content provenance matters both as a compliance requirement (synthetic content must be disclosed) and as a trust signal (authentic content can be verified as human-created).

A media company uses C2PA watermarking on all human-produced photos to distinguish them from AI-generated images. The watermark is invisible to viewers but readable by detection tools, proving the image's authentic provenance.