Checklist · 25 Controls

AI Security & Governance Checklist

Practical controls for organizations adopting AI and LLMs safely. Aligned to NIST AI RMF 1.0, ISO/IEC 42001, and the OWASP Top 10 for LLM Applications.

Governance & Inventory

  • AI use policy approved by exec and communicated; covers permitted tools, data classes, and prohibited use.
  • Inventory of approved AI services (SaaS, embedded features, internal models) maintained and reviewed quarterly.
  • Named accountable owner for AI risk; reports into existing risk/security committee.
  • AI-related risks added to the enterprise risk register with assigned treatments.

Data Protection

  • Classification rules explicitly state what data may and may not be sent to public LLMs.
  • Enterprise-tier AI services configured with data-use opt-out and no model training on customer data.
  • DLP / browser controls block paste of regulated data into unapproved AI tools.
  • Retention configured: prompts and outputs aligned to existing records-retention policy.
  • Data residency requirements verified per jurisdiction (GDPR, sector regulators).

Model & Vendor Risk

  • Pre-adoption review of each AI vendor: SOC 2 / ISO 27001, model card, training-data disclosures.
  • Third-party model providers contractually bound on confidentiality, breach SLAs, and IP indemnity.
  • Open-source / self-hosted models scanned for known vulnerabilities and malicious weights.
  • Change-management process for model upgrades; behavior delta tested before rollout.

Application Security (OWASP LLM Top 10)

  • Prompt-injection defenses: input sanitization, system-prompt isolation, output filtering.
  • Tool/function-calling limited to least privilege; destructive actions require human confirmation.
  • Retrieval-augmented generation (RAG) sources access-controlled to the user's permissions.
  • Output handling treats LLM responses as untrusted input (no direct eval, SQL, or shell).
  • Rate-limits, abuse detection, and cost caps on agentic / autonomous workflows.

Identity & Access

  • AI tools behind SSO with MFA; no shared / personal-tier accounts for business use.
  • Per-user logging of prompts, outputs, and tool calls retained for audit.
  • Role-based access to sensitive AI capabilities (e.g., code execution, customer-data RAG).

Oversight & Assurance

  • Human-in-the-loop required for AI decisions with legal, financial, or safety impact.
  • Bias, accuracy, and hallucination testing performed on high-impact use cases before launch.
  • Incident response plan covers AI-specific scenarios (prompt-injection, data leakage, model abuse).
  • Annual independent review against NIST AI RMF or ISO/IEC 42001 control objectives.
  • End-user training on safe AI use, prompt hygiene, and how to report concerns.

Building an AI governance program?

Dephiant helps organizations stand up AI governance aligned to NIST AI RMF and ISO/IEC 42001 — policy, tooling, and ongoing assurance.

Book a consultBack to resources