Document Intelligence Copilot:
Complete Implementation Guide

Table of Contents

1. Architecture Evolution: Why Application Gateway for Containers
2. Complete AGC Setup Guide (5 Steps)
3. Cost Analysis: $1K to $19K/Month Scenarios
4. 7-Week Migration Guide (NGINX → AGC)
5. Monitoring & Observability
6. Troubleshooting Runbooks
7. Testing Strategies
8. Complete CI/CD Pipeline
9. YouTube Videos & Microsoft Learn References

1. Architecture Evolution: Why Application Gateway for Containers

In late 2024, the Kubernetes community announced that the ingress-nginx controller will be retired in March 2026. This affects millions of production deployments worldwide.

Timeline

2024 ────────── 2025 ────────── 2026 ────────── 2027+
  │                │                │                │
  ├─ AGC GA        ├─ NGINX         ├─ Gateway API  ├─ Full
  │  (Available)   │  Retirement    │  (Istio)      │  Migration
  │                │  (March)       │  Available    │  Complete

Microsoft's Strategic Direction

Microsoft is investing in two future-proof solutions:

This guide uses Application Gateway for Containers (AGC) as the long-term, production-ready solution.

2. Complete AGC Setup Guide

Follow these 5 steps to deploy Application Gateway for Containers for your RAG Copilot.

Prerequisites

# Azure CLI version 2.50.0 or later
az version

# AKS cluster with managed identity
az aks show -g myResourceGroup -n myAKSCluster --query identity

# Required permissions
az role assignment list --assignee <your-identity> --scope <aks-resource-id>

Required Roles:

• Contributor on the AKS cluster
• Network Contributor on the VNET
• Managed Identity Operator on the AKS managed identity

Step 1: Enable the AGC Add-on

# Register the feature (if not already registered)
az feature register \
  --namespace Microsoft.ContainerService \
  --name AKS-ExtensionManager

# Wait for registration (check status)
az feature show \
  --namespace Microsoft.ContainerService \
  --name AKS-ExtensionManager

# Enable the add-on
az aks enable-addons \
  --resource-group myResourceGroup \
  --name myAKSCluster \
  --addons azure-application-gateway-for-containers

# Verify installation
kubectl get pods -n azure-alb-system

Expected Output:

NAME                                  READY   STATUS    RESTARTS   AGE
alb-controller-7d4b8c9f5d-x7k2m      1/1     Running   0          2m
alb-controller-bootstrap-xyz123      0/1     Completed 0          3m

Step 2: Deploy the Traffic Controller (Bicep)

Create the AGC infrastructure using Bicep:

@description('Name of the Application Gateway for Containers')
param agcName string

@description('Location for all resources')
param location string = resourceGroup().location

@description('Subnet ID for AGC')
param subnetId string

// Application Gateway for Containers (Traffic Controller)
resource trafficController 'Microsoft.ServiceNetworking/trafficControllers@2023-11-01' = {
  name: agcName
  location: location
  properties: {
    associations: [
      {
        subnet: {
          id: subnetId
        }
      }
    ]
  }
}

// Frontend configuration
resource frontend 'Microsoft.ServiceNetworking/trafficControllers/frontends@2023-11-01' = {
  parent: trafficController
  name: '${agcName}-frontend'
  location: location
  properties: {
    fqdn: 'api.mycompany.com'
  }
}

output trafficControllerId string = trafficController.id
output frontendFqdn string = frontend.properties.fqdn

Deploy:

az deployment group create \
  --resource-group myResourceGroup \
  --template-file infra/modules/agc.bicep \
  --parameters agcName=agc-rag-copilot \
               subnetId=/subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.Network/virtualNetworks/{vnet}/subnets/agc-subnet

Step 3: Configure Gateway API Resources

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: rag-gateway
  namespace: ai-workloads
spec:
  gatewayClassName: azure-alb-external
  listeners:
  - name: https-listener
    protocol: HTTPS
    port: 443
    hostname: "api.mycompany.com"
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: tls-cert-secret

Step 4: Configure HTTPRoute for Path-Based Routing

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: chat-route
  namespace: ai-workloads
spec:
  parentRefs:
  - name: rag-gateway
  hostnames:
  - "api.mycompany.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /v1/chat
      method: POST
    backendRefs:
    - name: rag-orchestrator-service
      port: 8000

{
  "name": "rag-index-v1",
  "fields": [
    { "name": "id", "type": "Edm.String", "key": true },
    { "name": "content", "type": "Edm.String", "searchable": true },
    { "name": "embedding", "type": "Collection(Edm.Single)", "dimensions": 1536, "vectorSearchProfile": "my-profile" },
    { "name": "metadata_storage_path", "type": "Edm.String" },
    { "name": "page_number", "type": "Edm.Int32" },
    { "name": "group_ids", "type": "Collection(Edm.String)", "filterable": true },
    { "name": "classification", "type": "Edm.String", "filterable": true }
  ]
}

// The filter every query MUST have
{
  "search": "user question",
  "filter": "group_ids/any(g: search.in(g, 'user_department_id'))"
}

Step 5: The Ingestion Pipeline (The Missing Link)

Most guides skip this, but it's 50% of the work. Networking is plumbing; Data is gold. Here is the robust path from PDF to Index:

1. Extract (Layout Model): Do not use simple OCR. Use the prebuilt-layout model in Document Intelligence to identify paragraphs, tables, and headlines. Keep the page_number metadata for every span!
2. Chunk Strategy: Split by "Semantic Paragraph". Do not break sentences. Overlap by 50 tokens.
3. Embed: Pass chunks to text-embedding-ada-002 to get the 1536-dimensional vector.
4. Index: Push the JSON payload (Content + Vector + group_ids) to Azure AI Search.

Dev Tip: Run this pipeline efficiently using an Azure Function with an Event Grid trigger on the ADLS `landing/` container.

Step 6: Enable WAF Protection

apiVersion: alb.networking.azure.io/v1
kind: ApplicationLoadBalancerPolicy
metadata:
  name: waf-policy
  namespace: ai-workloads
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: rag-gateway
  waf:
    enabled: true
    mode: Prevention
    ruleSetType: OWASP
    ruleSetVersion: "3.2"

3. Cost Analysis: What Will This Actually Cost?

*Estimates based on East US pricing (Jan 2026). Costs may vary by region.

Let's break down the total cost of ownership with real numbers across three deployment scenarios.

Scenario 1: Small Deployment (100 users, 10k queries/month)

Scenario 2: Medium Deployment (1,000 users, 100k queries/month)

Scenario 3: Enterprise Deployment (10,000 users, 1M queries/month)

4. 7-Week Migration Guide (NGINX → AGC)

This parallel deployment strategy ensures zero downtime during migration.

Week 1: Preparation

• Audit current NGINX configuration
• Map Ingress resources to HTTPRoute
• Create conversion scripts

Week 2: Deploy AGC

• Deploy AGC infrastructure (Bicep)
• Deploy Gateway resources in test namespace
• Run smoke tests

Week 3: Canary Deployment

• Deploy production Gateway
• Configure traffic splitting
• Route 10% of traffic to AGC

Weeks 4-6: Gradual Migration

• Week 4: 25% traffic to AGC
• Week 5: 50% traffic to AGC
• Week 6: 100% traffic to AGC

Week 7: Decommission NGINX

• Verify zero traffic on NGINX
• Backup and remove NGINX
• Clean up DNS records

5. Monitoring & Observability

Key Metrics to Monitor

// AGC Capacity Utilization
AzureMetrics
| where ResourceProvider == "MICROSOFT.SERVICENETWORKING"
| where MetricName == "CapacityUnits"
| summarize avg(Average), max(Maximum) by bin(TimeGenerated, 5m)

// OpenAI Token Usage
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| extend tokens = toint(parse_json(properties_s).usage.total_tokens)
| summarize sum(tokens) by bin(TimeGenerated, 1h)

Critical Alerts

• High Error Rate: >5% for 5 minutes
• AGC Capacity: >85% utilization
• OpenAI Quota: 429 errors detected
• Low Citation Rate: <80% of responses

6. Troubleshooting Runbooks

Runbook 1: 403 Forbidden Errors (Private DNS Issue)

Root Cause: Private DNS zones not linked to AKS VNET

Solution:

az network private-dns link vnet create \
  --resource-group myResourceGroup \
  --zone-name privatelink.openai.azure.com \
  --name aks-vnet-link \
  --virtual-network /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Network/virtualNetworks/aks-vnet

Runbook 2: 429 Too Many Requests (Quota Exhaustion)

Immediate Fix: Increase TPM quota if available

Long-term Solution: Implement APIM retry policy and semantic caching

Runbook 3: Slow Query Performance

Solution: Reduce AI Search top results from 50 to 10, implement parallel processing

7. Testing Strategies

Unit Testing

def test_citation_extraction(orchestrator):
    mock_response = {
        "choices": [{
            "message": {
                "content": "The refund policy is... [doc1.pdf, page 5]"
            }
        }]
    }
    citations = orchestrator.extract_citations(mock_response)
    assert len(citations) == 1
    assert citations[0].document == "doc1.pdf"

Load Testing

# Test with 100 concurrent users
locust -f tests/load/locustfile.py \
  --host https://api.mycompany.com \
  --users 100 \
  --spawn-rate 10 \
  --run-time 10m

8. Complete CI/CD Pipeline

7-stage GitHub Actions workflow for production deployment:

1. Validate Infrastructure: Bicep validation and what-if analysis
2. Security Scanning: Trivy, Checkov, APIM policy validation
3. Build and Test: Unit tests, Docker build, image scanning
4. Deploy Infrastructure: Bicep deployment to Azure
5. Deploy Application: Push to ACR, deploy to AKS
6. Smoke Tests: Health checks, chat endpoint validation
7. Rollback: Automatic rollback on failure

9. YouTube Videos & Microsoft Learn References

Essential YouTube Videos

RAG at scale with Azure AI Search

Watch on YouTube →

Azure AI Document Intelligence: extraction patterns

Watch on YouTube →

Azure API Management: token control and guardrails

Watch on YouTube →

APIM as AI Gateway: securing and scaling AI APIs

Watch on YouTube →

Microsoft Learn References

Summary

This implementation guide provides everything you need to deploy a production-grade Document Intelligence Copilot using Application Gateway for Containers. You now have:

• ✅ Complete AGC setup guide (5 steps)
• ✅ Detailed cost analysis (3 scenarios)
• ✅ 7-week migration strategy
• ✅ Monitoring and troubleshooting runbooks
• ✅ Testing strategies and CI/CD pipeline
• ✅ Curated resources for deep learning