Document Intelligence Copilot on Azure: The Production-Ready Blueprint

The Cast

Upendra

Lead Architect

Trinity

Cloud Engineer

Morpheus

Sec Architect

Project Manager

Plans, Progress, Risks & Issues

The Story

1) TL;DR (30 seconds)

UKLifeLabs needs employees to upload large document packs and ask questions.

But answers must be:

• provable (citations)
• traceable (audit trail)
• access-controlled (no leakage)
• cost-controlled (no runaway spend)

This blog gives you a buildable reference architecture:

UI → APIM (AI Gateway) → RAG Orchestrator → AI Search → Azure OpenAI + Document Intelligence → Citations + Audit

2) Design principles (the Microsoft review-board version)

1. APIM-first boundary: no direct calls from UI to Search, models, or ingestion.
2. Two-lane architecture: chat stays fast, ingestion stays heavy and async.
3. Evidence-or-refusal: answers must include citations (or the system refuses).
4. Least privilege by default: retrieval filters enforce the user's access.
5. Cost is a control plane concern: token budgets and rate limits are enforced at APIM.
6. Operationally boring: everything is observable, alertable, and repeatable as code.

3) The Airport + Research Library analogy

We use the "Airport" analogy to explain the security model to stakeholders, but here is the strict mapping to technical components.

Rule: Nobody reaches the experts without passing security.

3.5) Architecture Evolution: The Path to 2026+

4) The architecture (practical reference stack)

UI layer (pick ONE)

UI is replaceable. The boundary is not.

• Option A: Open WebUI – Fastest route. Treat it like a product.
• Option B: Microsoft-native UI – Copilot Studio or Teams (best for adoption).
• Option C: Custom portal – Best control for upload flows and audit viewing.

APIM is the AI Gateway (non-negotiable)

APIM is the platform control plane. It enforces:

• JWT validation (identity gate)
• Rate limits (traffic control)
• Token budgets (cost control)
• Audit logs (traceability)

{
  "timestamp": "2026-01-18T10:00:00Z",
  "user_id": "upendra@company.com",
  "department": "IT_Architecture",
  "request_id": "req-guid-123",
  "action": "rag_query",
  "input_tokens": 150,
  "output_tokens": 300,
  "retrieved_documents": [
    { 
      "doc_id": "HR_Policy.pdf", 
      "index_version": "v1.2", 
      "classification": "Internal" 
    }
  ],
  "model_deployment": "gpt-4-turbo-0125"
}

Runtime layer (AKS vs ACA)

Recommendation: use AKS for production. It fits regulated enterprise patterns (network segmentation, control). Use ACA only for pilots.

Deep Dive: Unsure which to pick? Read AI Hosting Decision Tree: AKS vs. ACA vs. Web Apps.

Data + RAG layer

• ADLS Gen2: Raw docs, extracted JSON, audit artifacts.
• Document Intelligence: Extracts text/fields from PDFs.
• Azure AI Search: Chunk index + vector search (Standard S1+).

5) The two flows you must separate

This prevents production pain.

Flow A: Chat (user-facing, low latency)

1. UI → APIM `/v1/private/chat`
2. APIM validates JWT + limits
3. Orchestrator queries AI Search with access filters
4. Orchestrator calls Azure OpenAI with retrieved chunks
5. Response returns with citations

{
  "answer": "The standard policy allows 15 days of PTO...",
  "citations": [
    {
      "id": "doc-123",
      "filename": "HR_Policies_2025.pdf",
      "page_number": 5,
      "text_snippet": "...employees accrue 1.25 days per month...",
      "relevance_score": 0.89
    }
  ],
  "refusal_reason": null
}

Flow B: Ingestion (async, heavy workload)

1. Document lands in storage
2. Event triggers ingestion worker
3. Document Intelligence extracts text
4. Chunking + embeddings → Index into AI Search

6) Security trimming (do not skip this)

RAG must enforce permissions at retrieval time. Every chunk should carry `department`, `classification`, and `allowedGroups`.

Every query should filter by the user's Entra groups. If you skip this, you will leak data across teams.

7) Quota planning (TPM/RPM)

Quota is not a mystery. It’s a budget. Think of Azure OpenAI quota like a Family Data Plan:

• Subscription Quota: The total allowance for the whole family (e.g., 2M TPM per region).
• Deployments: The individual devices. Each needs its own cap so the teenager (Batch Job) doesn't starve the parents (Chat App).
• TPM (Tokens Per Minute): The speed limit. This is your primary throttle.
• 429 Errors: This isn't a bug; it's a guardrail. It means the plan works.

8) The Architecture Board

Use these prompts to generate your official documentation diagrams. Standard notation matters.

Shared Hub

Prod AI Sub

User

Cloudflare

Tunnel

App Gateway

WAF v2

Azure Firewall

IDPS

APIM

Gateway

AKS Cluster

Orchestrator

Search

OpenAI

HTTPS

TLS

INSP

mTLS/JWT

9) The Build Room (Copy-Paste)

Real production means infrastructure as code. Here are the core modules you need.

A) APIM Policy: The "Airport Security" Check

Don't rely on code for security. Enforce it at the gateway.

<!-- Validate JWT before anything else -->
<validate-jwt header-name="Authorization" failed-validation-httpcode="401">
    <openid-config url="https://login.microsoftonline.com/{tenantId}/v2.0/.well-known/openid-configuration" />
    <required-claims>
        <claim name="aud">
            <value>{audience}</value>
        </claim>
    </required-claims>
</validate-jwt>

<!-- Enforce Token Budget (Cost Control) -->
<rate-limit-by-key calls="500" renewal-period="60" counter-key="@(context.Request.IpAddress)" />
<azure-openai-token-limit tokens-per-minute="10000" counter-key="@(context.Subscription.Id)" />
        

B) Repo Structure (The "Standard")

/src
  /orchestrator (Python/FastAPI)
  /ingestion (Dotnet Worker)
/infra (Bicep)
  /modules
    /apim
    /ai-search
    /openai
  main.bicep
/policies (APIM XML)
  /fragments
    /security.xml
    /audit.xml
/tests
  /load-testing (Locust)
        

C) The Pipeline (GitHub Actions Snippet)

Friends don't let friends deploy broken XML. Validate it before the merge.

name: Validate APIM Policies
on: [pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      # Don't deploy. Just check if the XML is valid.
      - name: Check XML Syntax
        run: |
          find ./policies -name "*.xml" -print0 | xargs -0 -I {} xmllint --noout {}
          
      - name: Run OPA Policy Check
        run: ./scripts/check-compliance.sh # e.g., ensure no "star" allows
        

10) Go-live checklist

• UI calls APIM only
• JWT validation enabled
• APIM Products split (internal vs external)
• Token budgets enforced
• Citations required rule enabled
• Audit logs include chunk IDs + model deployment ID

Essential Resources

Watch these sessions to understand the "Security Sandwich" architecture in depth.

RAG in Azure AI Search

Azure Friday - Intro to RAG with Azure OpenAI

Build your own Copilot

Microsoft Mechanics - Copilot extensibility & Plugins

Vector Search Deep Dive

Microsoft Mechanics - RAG at Scale & Vector Search

Ready to operationalize your Azure journey?

I help organizations turn stalled cloud initiatives into execution engines.