Azure Landing Zones for AI Workloads

If your AI project is stuck waiting for GPU quota approvals while your data scientists spin up shadow IT resources, you do not have an AI problem — you have a landing zone problem.

Azure Landing Zones are well-documented for traditional enterprise workloads. But AI workloads break the standard patterns: they need GPUs, large ephemeral storage, model registries, feature stores, and data pipelines that move petabytes. The default Enterprise-Scale Landing Zone assumes steady-state compute. AI is anything but steady-state.

This post covers the practical decisions I have made when designing Azure infrastructure for AI workloads — GPU quota strategy, model lifecycle management, and the data plumbing that actually matters.

Why Standard Landing Zones Fall Short for AI

The Azure Landing Zone Accelerator is built around Hub-Spoke networking, centralized identity, and predictable resource topologies. That works for web apps and APIs. AI workloads introduce three problems the standard pattern does not address well:

1. GPU Quota Is a Bottleneck, Not a Line Item

Azure GPU quotas (NC, ND, NV series) are per-subscription and per-region. If your landing zone uses a single subscription for AI compute, you will hit quota limits fast — and quota increase requests take days. The fix: dedicate a subscription specifically for AI compute, separate from general workloads. This gives you a clean quota ceiling and prevents GPU requests from competing with standard VM reservations.

resource "azurerm_subscription" "ai_compute" {
  alias           = "ai_compute"
  subscription_id = var.ai_subscription_id
  display_name    = "AI Compute Sub"
  billing_account_id = var.billing_account_id
}

2. Model Registries Need Their Own Lifecycle

Azure Machine Learning has a built-in model registry, but it sits inside the ML workspace — which means it shares lifecycle with experiments, endpoints, and data assets. For production, I treat the model registry as infrastructure:

• Azure ML Model Registry for versioned model artifacts with lineage
• Azure Container Registry (ACR) for inference container images
• Blob Storage for raw model weights (especially for large open-source models)

The registry is the contract between training and inference. If you do not version your models, you cannot roll back a bad deployment. Period.

3. Data Pipelines Are the Hidden Cost

AI workloads consume data at scale — training data, feature stores, embedding caches, evaluation datasets. The standard landing zone provides shared services networking, but AI needs high-throughput data movement:

• Azure Data Lake Storage Gen2 for raw and processed data
• Azure Data Factory or Apache Spark on Fabric for ETL
• Private Link endpoints for data access without crossing the public internet

The Architecture I Would Build Today

Here is the hub-spoke topology I would design for an organization deploying AI workloads at scale on Azure.

Hub Subscription: Shared Services

• Firewall (Azure Firewall Premium for TLS inspection)
• Bastion host for secure VM access
• DNS Private Zones
• Azure Monitor and Log Analytics workspace
• Key Vault (HSM-backed) for secrets and model API keys

Spoke 1: AI Training Compute

• Dedicated subscription with GPU quota (NC A100 v4, ND H100 v5)
• Azure ML workspace with managed VNet
• CycleCloud or AML Compute Clusters for auto-scaling GPU pools
• Private endpoints to ADLS Gen2 and ACR

Spoke 2: AI Inference and Serving

• Azure Container Apps or AKS for model serving endpoints
• AKS with GPU node pools for large model inference
• Application Gateway with WAF for public-facing inference APIs
• Cosmos DB for request/response logging and feature cache

Spoke 3: Data Platform

• Azure Data Lake Storage Gen2 (Bronze, Silver, Gold layers)
• Microsoft Fabric for analytics and data science notebooks
• Azure Event Hubs for real-time feature streaming
• Private Link to all storage accounts

GPU Quota Strategy: The Part Nobody Talks About

GPU quota is the single biggest operational blocker for AI teams on Azure. Here is what I have learned:

• Request early. GPU quota increases take 2-5 business days. Do not wait until you need them.
• Use multiple subscriptions. Each subscription has its own quota ceiling. Spread training and inference across subscriptions if you need more aggregate GPU capacity.
• Reservation vs. on-demand. Azure Reservations can lock in GPU capacity for predictable workloads, but they are 1-3 year commitments. Use them for production inference, not experimentation.
• Spot VMs for training. NCasT4 v3 and NDm A100 v4 spot instances can cut training costs by 60-80%. Use AML managed spot clusters with automatic checkpointing.

Model Registry Pattern

I use a three-tier registry approach that keeps things clean:

The deployment pipeline promotes models through these tiers: train, register in ML, build container, push to ACR, deploy to AKS or Container Apps. If a model fails validation, it never reaches ACR. That is the quality gate.

Networking: Private Link Everywhere

AI workloads move large datasets. The default Azure networking model (public endpoints with NSGs) works but introduces latency and security risk. For production AI, I enforce:

• Private Link for all storage accounts — ADLS, ACR, Key Vault, Cosmos DB
• Service endpoints for Azure ML — keeps training traffic off the public internet
• Azure Firewall rules — explicit egress control for PyPI, Hugging Face, and model download endpoints
• Hub-Spoke VNet peering — each spoke connects to hub, never directly to each other

Private Link is not optional for production AI. It is the difference between we hope nothing leaks and nothing can leak.

Cost Controls That Actually Work

AI infrastructure costs can spiral. Here are the controls I have seen make the biggest difference:

• Dedicated cost allocation tags: Tag every resource with cost-center, project, and environment. Azure Cost Management can then attribute GPU spend to specific teams.
• Auto-shutdown for training: AML Compute Clusters auto-scale to zero. Do not leave GPU VMs running 24/7 for weekly training jobs.
• Budget alerts: Set Azure Budgets with 80% and 100% thresholds on the AI subscription. Alert the team, not just finance.
• Spot for training, reserved for inference: Training is interruptible; inference is not. Match the purchase model to the workload.

Common Mistakes

Three patterns I see repeated across organizations:

Mistake 1: Putting everything in one subscription. AI compute, data, and serving in the same subscription means quota contention, billing confusion, and blast radius problems. Split them.

Mistake 2: Skipping Private Link. We will add it later always means we will never add it. Design for private endpoints from day one.

Mistake 3: No model versioning. If your team cannot tell you which model version is running in production right now, you have a problem. The ML Model Registry is free to use — there is no excuse.

Disaster Recovery for AI Workloads

AI infrastructure has unique DR considerations. Your training data in ADLS Gen2 needs geo-redundancy (GRS or ZRS). Your model registry in Azure ML is region-locked, so back up model artifacts to a secondary region via ACR geo-replication. For inference endpoints, deploy to a secondary region with traffic manager failover. The key insight: DR for AI is not just about VMs and databases — it is about preserving the entire training pipeline state, including data lineage, model versions, and container images.

Putting It All Together

The landing zone is not a one-time project. It evolves as your AI workloads mature. Start with the hub-spoke topology and GPU-dedicated subscription. Add the model registry and data platform spokes as your team scales. Enforce Private Link from day one. Tag everything for cost attribution. And most importantly — treat your AI infrastructure with the same discipline as your production application infrastructure. The GPU does not get a pass on operational excellence.

Wrapping Up

Azure Landing Zones for AI are not a different framework — they are the same hub-spoke topology with GPU-specific considerations, dedicated subscriptions for quota management, a model registry lifecycle, and private networking for data movement. The fundamentals do not change; the operational details do.

If you are starting an AI initiative on Azure, invest the time in the landing zone first. It is cheaper than fixing it after your data scientists have already built a shadow IT empire.