Governance & security before AI touches production.

1 Data Protection

How do we prevent data leaks when using AI features in ITSM?

The leak most leadership worries about, vendor exfiltrating customer data, is real but well-mitigated by contracts. The leak most organisations actually experience is more mundane and harder to prevent: agents pasting ticket content into public LLM tools to get help with a problem, copying confidential customer information into systems with no enterprise protection.

Three layered controls produce defensible leak prevention:

The questions to ask vendors before signing:

• Where is inference performed geographically? EU data may need to stay in the EU. UK data residency is its own conversation post-Brexit.
• Is the data used to train your models? Default for many vendors is yes. Negotiate a no, or accept that your data shapes the vendor's product.
• What's the data retention policy after deletion request? "We delete within 30 days" usually means logs persist longer than the active data.
• Can we audit the AI's data handling? Audit rights are standard in enterprise contracts but rarely volunteered.

2 Hallucinations & Transparency

How do we manage AI hallucinations and ensure transparency in ticket handling?

Hallucination, the AI confidently producing plausible-sounding but incorrect content, is an inherent property of language models, not a bug that gets fixed. The question for ITSM is not how to eliminate hallucination but how to contain it so that hallucinated content cannot cause harm.

Three categories of hallucination that matter in ITSM, ranked by severity:

Transparency mechanisms that materially help:

• Source citation on every suggestion. When the AI suggests a reply or surfaces a KB article, show which past tickets or articles it drew from. Agents check the source when stakes are high.
• Confidence scoring. Where vendors expose it, surface the AI's own confidence. Suggestions below a threshold should be flagged, not hidden, agents need to know the system was unsure.
• Visible AI labelling on outputs. Customer-facing AI responses must be identifiable as AI-generated, both for trust and for emerging regulatory requirements.
• Reasoning chain access. One-click "why did you suggest this?" gives agents an audit trail for unusual recommendations.

3 Ownership

Who owns the AI model, the inputs, and the outputs?

Ownership in AI contracts is more complex than ownership in conventional SaaS contracts, and the differences matter. Three layers need explicit clarity:

Five contract clauses worth negotiating before signing any AI-feature ITSM agreement:

• Non-training commitment. Your data is not used to train models that benefit other customers. Default vendor terms often allow this; explicit negotiation is required.
• Deletion timelines. When you ask for data to be deleted, what actually gets deleted, and when? Logs, caches, training-set inclusions all need explicit treatment.
• Output ownership. AI-generated content is unambiguously yours, with no vendor reuse rights.
• Sub-processor disclosure. Which AI providers does the vendor use? Changes in sub-processors need notification and approval.
• Audit rights. Right to inspect AI handling of your data, including reasonable visibility into training and inference processes.

4 Human Oversight

What does "human-in-the-loop" actually mean for ITSM AI in practice?

"Human in the loop" has become a marketing phrase that vendors use to imply oversight without committing to specifics. The practical version requires defining which actions require what kind of human approval, by whom, in what time frame.

A useful taxonomy of AI actions in ITSM, with the appropriate level of human oversight for each:

The mistake to avoid: treating HITL as a slider that gets dialled to "more" or "less." It's not a single setting. It's a policy that varies by action class, with explicit approvals, named owners, and consequence-proportional rigour. A single HITL configuration applied uniformly is either too restrictive (where automation could safely run) or too permissive (where it shouldn't).

Three common HITL failures to watch for:

• Approval fatigue. Asking humans to approve too many AI actions produces rubber-stamping. Approval becomes ceremonial within weeks.
• Out-of-band approvers. The named human isn't actually available in the time window the action requires. Approvals time out and default behaviour kicks in.
• Logging without review. "We log every AI action" is not oversight. Logs that aren't reviewed are evidence in a future investigation, not control in the present.

5 Regulation

How do we comply with the EU AI Act and similar regulations for ITSM AI?

The EU AI Act, in force from 2024 with phased application running through 2026 and beyond, classifies AI systems by risk. Most ITSM AI features fall into the limited-risk category, which is significantly less onerous than the high-risk category that has dominated public discussion. But "limited-risk" still carries real obligations, and the boundary between limited and high-risk in ITSM is closer than most teams realise.

How ITSM AI typically maps to EU AI Act risk classes:

Three principles to apply regardless of regulatory regime:

• Transparency to end users. If a customer or employee is interacting with AI-generated content, they should know. This is becoming universal, EU AI Act, UK guidance, US sectoral rules all converge on this.
• Documentation as compliance. Even limited-risk AI should have documented purpose, training data sources (where vendor-disclosed), and known limitations. This costs little to maintain and saves significant effort if regulatory scope expands.
• Bias monitoring. AI features deployed in service contexts can develop subtle biases over time. Regular monitoring of outcome distribution by customer group, agent group, and ticket type is good practice and increasingly expected by regulators.