Azure AI Foundry inherits Azure's enterprise security posture, but production AI deployments require deliberate configuration across six distinct security layers. The platform gives you the controls — this guide tells you exactly how to configure each one for enterprise compliance. Skipping any layer leaves gaps that will surface in a security audit.
Written by Kovil AI engineers · Updated May 2026
The most impactful security decision in any Azure AI deployment is authentication method. API keys stored in environment variables or Key Vault references are auditable but still represent a credential that can be leaked, rotated incorrectly, or shared across environments. Managed Identity eliminates the credential entirely — the Azure runtime handles identity, and access is controlled through Entra ID RBAC.
API key approach (avoid in production)
Managed Identity (correct approach)
The RBAC role assignments that matter for a typical Azure AI Foundry deployment are precise. Assigning too broad a role (Contributor at subscription scope) is a common shortcut that fails security audits. Assign the minimum required role at the minimum required scope:
Azure OpenAI
Resource level (not resource group)
Cognitive Services OpenAI User
Allows inference calls; does not allow model deployment or resource configuration.
Azure AI Search
Resource level
Search Index Data Reader
Read-only access to indexes. Use Search Index Data Contributor only for indexer identity.
Azure Storage
Container level
Storage Blob Data Reader
Scoped to the specific container holding your documents, not the entire storage account.
Azure Key Vault
Secret level where possible
Key Vault Secrets User
Allows reading specific secrets. Never assign Key Vault Administrator to an application identity.
Azure ML Workspace
Workspace level
AzureML Data Scientist
For Prompt Flow evaluation runs. Does not allow compute provisioning or workspace configuration changes.
Private endpoints are the network-layer control that prevents any AI traffic from traversing the public internet. Every Azure service in your AI stack — Azure OpenAI, Azure AI Search, Azure Storage, Azure Key Vault — should be deployed with a private endpoint for production workloads handling sensitive data. This is a non-negotiable control for HIPAA, PCI DSS, and most enterprise security frameworks.
Architecture principle
Every AI service should be deployed with “Public network access: Disabled”. Traffic flows: Application (ACA/AKS/VM) → Private Endpoint → Azure Service. No traffic takes a public path. DNS resolution for service endpoints is handled by a Private DNS Zone linked to your VNet — not the public DNS records that resolve to public IP addresses.
Azure OpenAI
Azure AI Search
Azure Storage (documents)
Azure Key Vault
Understanding exactly what Azure AI services log — and where those logs go — is a prerequisite for data governance compliance. The default logging configuration is not always appropriate for regulated data, and several logging capabilities require explicit opt-in or opt-out decisions.
| Service | What is logged by default | PII risk | Action required |
|---|---|---|---|
| Azure OpenAI | Request metadata (timestamp, token counts, model, latency) | Low — prompt content NOT logged by default | Enable diagnostic logs to Log Analytics; verify content logging is off |
| Prompt Flow | Flow run metadata, step-level latency, evaluation scores | Medium — flow inputs/outputs can be captured | Review trace settings; mask PII fields before trace capture |
| Azure AI Search | Query text, result count, latency | High — query text may contain PII | Enable slow query logging to Log Analytics with appropriate retention |
| Azure Monitor | Aggregated metrics only by default | Low | Configure resource-level diagnostic settings explicitly |
Azure OpenAI content logging opt-in
Microsoft offers an opt-in “abuse monitoring” feature that logs prompt and completion content for safety review. This is OFF by default for most enterprise agreements but should be verified explicitly — especially in HIPAA or data residency-sensitive contexts. Check your Azure OpenAI resource under “Abuse monitoring” in Azure AI Foundry.
Azure AI Content Safety filters inputs and outputs for four harm categories. Default thresholds are generic — production deployments for regulated industries require explicit threshold configuration per deployment, and the configuration must be documented and defensible to auditors.
Hate & Fairness
For customer-facing agents, keep at Medium. Internal tools may use High threshold if content policy permits.
Violence
Adjust to High for general business use. Health/insurance deployments may require Medium to handle clinical content.
Sexual Content
Enterprise business applications should use High threshold. Adult content platforms require Azure content filtering waiver.
Self-Harm
For any consumer-facing deployment, configure Low threshold and route flagged content to appropriate resources.
Beyond the four harm categories, two additional controls matter for enterprise deployments. Jailbreak detection (Prompt Shield) blocks adversarial prompt injection attempts — this should be enabled on all production deployments. Groundedness detection flags AI responses that make claims not supported by the retrieved context — critical for any compliance or regulated-content use case where hallucination is a business risk.
Prompt Shield (jailbreak detection)
Enable on all deployments. Adds ~30ms latency.
Custom blocklist
Define for domain-specific prohibited terms (competitor names, restricted topics, internal product codes).
Groundedness detection
Enable for any RAG-based agent where factual accuracy is a compliance requirement.
PII entity detection
Enable for customer-facing agents. Configure redaction mode (substitute vs. mask) based on downstream processing needs.
Azure AI Foundry inherits compliance from the underlying Azure services. Each certification has different implications for what you need to configure versus what Microsoft has already done. Understanding these boundaries is critical for audit preparation.
HIPAA
BAA availableWhat's covered: Azure OpenAI, Azure AI Search, Azure Storage, Azure Monitor are all covered under Microsoft's HIPAA BAA when deployed in a compliant Azure subscription.
Your responsibility: You must sign the BAA (available in Azure portal under Compliance), configure all services with private endpoints (no public access), and ensure PHI is not captured in diagnostic logs. The BAA covers the platform — your application-level PHI handling (logging, caching, display) is your responsibility.
SOC 2 Type II
CoveredWhat's covered: All Azure AI services are SOC 2 Type II audited. Microsoft publishes the audit report under NDA via the Service Trust Portal.
Your responsibility: Your application code and deployment processes are not in scope of Microsoft's SOC 2. You need your own controls for application-layer access management, change management, and incident response.
ISO 27001
CoveredWhat's covered: Azure OpenAI and Azure AI Search are ISO 27001 certified. Certificates are available on the Service Trust Portal.
Your responsibility: ISO 27001 requires a complete ISMS covering your organisation. The Azure certification covers the infrastructure layer — you need to demonstrate controls for the application, process, and people layers.
PCI DSS
QualifiedWhat's covered: Azure is a PCI DSS Level 1 service provider. Payment card data can be processed within Azure boundaries when configured correctly.
Your responsibility: Cardholder data must never enter Azure OpenAI prompts or AI Search indexes. Architect your solution so payment data is tokenised or detokenised outside the AI layer. The Azure PCI attestation does not cover AI workloads that process raw card data.
FedRAMP Moderate
Authorized (Azure Government)What's covered: Azure Government regions have FedRAMP Moderate authorization. Azure OpenAI is available in Azure Government with FedRAMP coverage.
Your responsibility: Standard Azure commercial regions are not FedRAMP authorized. If your use case requires FedRAMP, you must deploy to Azure Government — verify service availability as not all AI Foundry features are available in Azure Government regions.
This 12-point checklist represents the production security baseline we verify on every Azure AI deployment before go-live. Each item is either a direct audit finding or a prerequisite for passing one.
All service-to-service authentication uses Managed Identity — zero API keys in application code or environment variables.
RBAC assignments follow least privilege: minimum required role, at minimum required scope (resource level, not subscription).
Private endpoints deployed for all AI services (OpenAI, AI Search, Storage, Key Vault). Public network access disabled on each resource.
Private DNS Zones configured for each service and linked to the application VNet. Public DNS records resolve to private IPs within the VNet.
Azure AI Content Safety thresholds configured explicitly per deployment — not relying on defaults.
Prompt Shield (jailbreak detection) enabled on all customer-facing and internal deployments.
Azure Monitor diagnostic settings configured for all AI services. Logs route to a dedicated Log Analytics workspace with appropriate retention (90 days minimum; 1 year for regulated industries).
PII is identified in the data flow and either masked before it enters LLM context or explicitly confirmed to be compliant with retention requirements.
Azure OpenAI content logging (abuse monitoring) opt-in status verified and documented.
Prompt Flow evaluation pipeline with a representative test dataset deployed and running on every code deployment. Evaluation scores tracked and alerted on regression.
Incident response runbook documented: who is alerted on content safety violations, what the escalation path is, how the agent is disabled if needed.
HIPAA BAA (or relevant compliance agreement) signed for the Azure subscription if regulated data is processed. Verified in Azure portal Compliance section.
Key takeaways
Continue Reading
Azure OpenAI vs OpenAI API: What actually changes when you deploy in Azure?
PlaybookAzure AI Foundry Pricing Guide 2026: What enterprise AI actually costs
Implementation GuideHow to architect your first Azure AI Foundry agent: A practitioner's checklist
ServiceAI Agent Design & Build — end-to-end agent engineering on Azure
Azure AI Practice
By Industry
How We Compare
Integrations