NCSA AI Security Guidelines: lifecycle and governance
A summary of NCSA's AI Security Guidelines, covering the 7-phase secure AI lifecycle, governance, risk management, and practical testing focus areas.
- Target Audience
- Executives, AI development and operations teams, legal and DPO functions, and security teams using the NCSA guidance
- Mandatory Frequency
- Lifecycle-aligned (Phases 0 to 6, Concept through Disposal)
- Required Scope
- Prompts, models, APIs, training data, software supply chain, and surrounding software
- Non-Compliance Risk
- Prompt injection, jailbreaking, data poisoning, confidential-data leakage, and go-live without verification evidence
Purpose and intended audiences
NCSA presents the document as guidance for developing, managing, and using AI systems securely and trustworthily.
Each stakeholder has a distinct role linking risk, safeguards, and evidence across the AI lifecycle.
- Executives — Set direction, risk ownership, and accountability
- AI development and operations — Design, build, operate, and monitor the system
- Legal, privacy, and security — Govern requirements, data, and safeguards
- Auditors — Independently examine processes and evidence
- Users and suppliers — Use and support the system within defined boundaries
- It spans policy-level decision making for executives and technical practice for AI development and operations teams.
- Its intended stakeholders include legal and data-protection functions, cybersecurity teams, employees, customers, data subjects, and supply-chain vendors.
- The guidance draws on international standards and technical material including ISO/IEC, ENISA, and OWASP alongside Thai laws and practices.
The secure AI lifecycle framework
The guidelines structure security controls across the full lifecycle, from pre-investment planning to decommissioning.
Assign owners and evidence at each phase so security continues from concept to retirement.
- Concept — Define needs and risks from the start
- Secure Design — Identify threats and design safeguards
- Secure Development — Protect data, code, and components
- Secure Verification — Verify and test before deployment
- Secure Deployment — Control release and configuration
- Secure Operation and Maintenance — Monitor, maintain, and respond
- Disposal — Retire the system and manage remaining data
- Phase 0: Concept - Identify business context, legal requirements, assets, threats, vulnerabilities, and risks before development.
- Phases 1-3: Secure Design, Secure Development, and Secure Verification.
- Phases 4-6: Secure Deployment, Secure Operations and Maintenance, and Proper Disposal.
Governance, risk, and accountability
Governance uses the GRC framework and stresses that risk must be evaluated in each organization's specific context.
AI risk needs accountable owners, decisions, evidence, and review.
- Assign roles — Define accountability and decision authority
- Integrate with ERM — Bring AI risks into enterprise risk management
- Select safeguards — Assign controls, owners, and evidence
- Audit and assurance — Examine conformity and effectiveness
- Communicate and train — Enable each role to perform its duties
- Document security-related roles, authority, and accountability for the AI system.
- Integrate AI risk into enterprise risk management and track relevant Thai laws and regulations.
- Establish auditing, certification, communication, and training for controllable, auditable operations.
Practical implementation focus
The document recommends key measures to support auditing and retrospective review.
Selected safeguards need evidence that they work in the real AI system.
- Data provenance and quality — Verify data origin and suitability
- Component vulnerabilities — Examine software, libraries, APIs, and integrations
- Prompt injection — Test both direct and indirect paths
- AI red teaming — Simulate adversarial behavior within an authorized scope
- Logging and monitoring — Retain events, test results, and anomaly signals
- Establish Data Provenance and vulnerability scanning for processing software before launch.
- Test Direct and Indirect Prompt Injection, including jailbreaking, as described in the verification phase.
- Conduct AI Red Teaming and penetration testing across models, APIs, and surrounding software before go-live.
Requirements and Testing Scope Matrix
Summary of the referenced clauses, the testing scope they cover, and the expected evaluation cycle.
| Reference | Mandate Title | Scope Required | Testing Cycle |
|---|---|---|---|
| Phases 0 to 3 (NCSA AI) | Concept, Secure Design, Secure Development, and Secure Verification | Threat models, training data, software supply chain, prompt injection, and AI red teaming | Before investment, development, and go-live decisions |
| Phases 4 to 6 (NCSA AI) | Secure Deployment, Operations and Maintenance, and Disposal | Secure baseline, prompt monitoring, APIs, and surrounding software | Pre-launch and throughout operations until disposal |
Compliance Readiness Self-Assessment
Select items your organization has completed to evaluate your readiness score.
STH AI SECURITY TESTING
LLM-integrated App Pentest and AI Red Teaming
An LLM-integrated application has a wider attack surface than the model alone. The application, input paths, RAG or vector store, agent tools, identity, data, and event logging must be assessed together.
Distinguish LLM-integrated application testing from AI red teaming and use evidence suited to each objective.
- Map the system — Identify prompts, models, retrieval, tools, and trust boundaries
- Define scenarios — Set risks, success criteria, and stop conditions
- Test within controls — Use synthetic data and authorized scope
- Collect evidence — Record reproducible results and impact
- Remediate and retest — Assign owners and verify the outcome
LLM-integrated application pentest
Validate controls at the system integration points, rather than limiting the assessment to model behaviour.
- User authorization, tenant and data separation, and data exposure through responses or retrieved documents.
- RAG, vector store, model-provider, API, and third-party tool integrations, including token and secret boundaries.
- Approval and authorization enforcement before an agent invokes a tool or performs an impactful action.
AI Red Teaming
Simulate adversarial behaviour within an authorized scope to demonstrate whether guardrails and protective workflows work in practice.
- Prompt injection and indirect prompt injection through documents or data used as context.
- Jailbreaking or attempts to move the system outside its intended purpose, with validation of unsafe-output handling.
- Agentic-AI risks where a prompt or tool invocation could affect data, accounts, or downstream systems.
Expected testing evidence
- An authorized scope, test environment and data, stop criteria, and emergency contacts.
- Evidence, severity, impact conditions, and remediation guidance that can be independently retested.
- A retest after remediation and a record of controls or risks that need production monitoring.
NCSA's Secure Verification phase gives active testing, LLM penetration testing for prompt injection and jailbreaking, data-leakage risks, and dangerous agent actions as examples of evidence to verify before release.
Talk to STH about LLM-integrated App Pentest and AI Red Teaming
Frequently Asked Questions (FAQ)
Key answers and practical guidance addressing common compliance questions.
Which AI implementations fall within the scope of NCSA AI Security Guidelines?
The guidelines encompass traditional Machine Learning models as well as Generative AI and LLM systems deployed within enterprise environments or public services.
How does security testing for LLM systems differ from standard web or API testing?
In addition to web and API vulnerabilities, the guidance calls for AI-specific verification such as prompt injection, jailbreaking, and AI red teaming across models, APIs, and surrounding software.
