Azure Service Bus: queues, topics, subscriptions, and reliable AI messaging

radio_button_uncheckedshort_text1. The AI messaging scenario and learning goals

Consider a document-analysis API whose inference takes anywhere from a few seconds to half a minute. Traffic arrives in bursts, only a limited number of processors can use the model concurrently, and notification, audit, metrics, and quality systems need the result. A synchronous chain makes the caller wait, couples every component to the same availability window, and lets one slow or malformed request block useful capacity.

Azure Service Bus introduces a durable broker between those components. The API can acknowledge accepted work promptly, inference workers can drain it at a controlled rate, and downstream consumers can receive independent result copies. Failed input remains inspectable instead of disappearing into an application log.

• Apply load leveling, competing consumers, temporal decoupling, and publish/subscribe to AI workloads.
• Choose a queue or a topic with subscriptions, including sessions and broker-side filters.
• Design JSON bodies, application and system properties, correlation, batching, expiration, and claim checks.
• Process with Peek-Lock, explicit settlement, idempotency, lock renewal, and dead-letter recovery.
• Build and test a Python flow with the current azure-servicebus library and Microsoft Entra ID.

Topic summary

A broker separates fast request acceptance from variable AI processing and gives every delivery, retry, and failure an explicit operational path.

radio_button_uncheckedshort_text2. Understand namespaces, entities, protocols, and identity

Service Bus is a fully managed enterprise message broker. A namespace is the administrative and network boundary that contains queues, topics, and subscriptions and exposes an endpoint such as <namespace>.servicebus.windows.net. AMQP 1.0 is the primary protocol used by the modern SDKs and supports rich broker semantics such as settlement, transactions, ordering controls, and duplicate detection.

Clients can authenticate with Shared Access Signature credentials or Microsoft Entra ID. For workloads hosted in Azure, prefer a managed identity and the narrowest built-in data role: Azure Service Bus Data Sender, Data Receiver, or Data Owner. This avoids embedded connection strings and key rotation. Local SAS authentication can be disabled after every client has migrated.

Topic summary

Design the namespace as a security and capacity boundary, place the correct entities inside it, and use managed identity with least-privilege Service Bus data roles.

radio_button_uncheckedshort_text3. Decouple request intake from inference

In an asynchronous request-reply flow, the API validates the envelope, writes a status record, sends a work message, and returns an operation identifier. A worker later receives the message, runs inference, persists the result, and either emits a completion event or lets the client poll. Producer and consumer no longer have to be online, deployed, or scaled at the same time.

This architecture does not make latency disappear; it makes latency controllable and visible. The product must communicate accepted, running, completed, failed, and expired states, protect status endpoints, set realistic timeouts, and decide how clients are notified. The queue is not the result database and should not be queried as one.

Topic summary

Asynchronous messaging turns long inference into a tracked operation: acknowledge acceptance, persist status and results, and let producers and workers evolve independently.

radio_button_uncheckedshort_text4. Level load with a durable queue

Load leveling absorbs a short traffic spike in a queue while a stable worker fleet consumes at sustainable throughput. The design protects GPU, high-memory, or rate-limited model endpoints from a sudden concurrency surge and avoids paying for enough compute to handle a rare peak instantly.

The queue must still be sized and monitored. A growing backlog increases completion time and can outlive its business value. Set capacity, TTL, alerts, and scaling thresholds from arrival rate, processing rate, acceptable age, and failure behavior. Backpressure should be visible to callers through admission controls or status, not hidden until storage fills.

Topic summary

Use the queue as a controlled buffer between bursty demand and finite inference capacity, with explicit limits for backlog size and age.

radio_button_uncheckedshort_text5. Scale with competing consumers and temporal decoupling

Multiple worker instances can receive from the same queue. Service Bus locks a message for one receiver at a time, so workers compete for available work without a central dispatcher. Azure Functions, Azure Container Apps, or AKS can add instances when backlog grows and remove them when demand falls.

If a worker crashes before settlement, the lock eventually expires and the message becomes available again. That resilience also creates possible duplicates, so each worker must make downstream effects idempotent. Temporal decoupling means the API can continue accepting within configured limits during a worker deployment or brief outage because durable messages wait for a receiver.

Topic summary

Competing consumers provide horizontal work distribution, while durable storage separates component availability; idempotency makes redelivery safe.

radio_button_uncheckedshort_text6. Treat queue depth as backpressure and a scaling signal

Active message count, oldest-message age, incoming and completed rates, processing latency, failure rate, and dead-letter count describe the health of the pipeline better than CPU alone. A consistently rising queue means arrival exceeds completion. A permanently empty queue can mean correct low latency, but it may also reveal overprovisioning.

Azure Monitor can collect Service Bus metrics and drive alerts. KEDA-based scaling in Azure Container Apps or AKS and Service Bus triggers in Azure Functions can translate backlog into worker replicas. Set minimum and maximum capacity, cooldown, concurrency, and model endpoint limits together; blindly adding receivers can move the bottleneck downstream and increase throttling.

Topic summary

Scale from backlog, age, throughput, and downstream limits together; queue depth is a pressure signal, not a complete capacity plan.

radio_button_uncheckedshort_text7. Select Standard or Premium and respect size limits

The quota includes body and properties and can change by tier, protocol, and configuration, so verify current documentation before deployment. Larger broker messages consume bandwidth and broker resources even when technically allowed. AI documents, images, audio, and model artifacts usually belong in object storage with a claim check rather than in the broker.

Topic summary

Choose the tier for isolation, networking, availability, and measured throughput; treat current size quotas as hard validation boundaries, not payload design targets.

radio_button_uncheckedshort_text8. Choose a queue or a topic with subscriptions

A topic is not received directly; receivers consume from subscriptions. A subscription behaves like a virtual queue and can itself have competing consumers. Do not create three consumers on one queue when all three must see the message—the broker intentionally gives each queued message to only one competing receiver.

Topic summary

Use a queue for one processing responsibility and a topic with subscriptions when several independent responsibilities need matching copies.

radio_button_uncheckedshort_text9. Preserve per-workflow order with sessions

Normal queue order does not guarantee that concurrent workers finish in arrival order. Sessions group related messages by session_id and grant one receiver an exclusive session lock, enabling FIFO processing within that group while different sessions run in parallel. A document pipeline can put extract, classify, and summarize commands under the same document identifier.

Sessions require the entity to be created with session support and every message to carry a valid session identifier. They reduce concurrency for hot sessions and add state and lock management, so use them only when ordered handling is a correctness requirement. If steps are better represented as an orchestrated workflow, a workflow engine may be clearer than encoding every dependency in messages.

Topic summary

Sessions provide ordered, exclusive processing per business key while preserving parallelism across keys; enable them deliberately because they change entity and receiver behavior.

radio_button_uncheckedshort_text10. Fan out results with subscription filters

Each topic subscription starts with a TrueFilter rule that accepts every publication. Replace or supplement it with SQL filters over system and application properties when a consumer needs only a subset. Correlation filters perform efficient exact matching. A FalseFilter accepts nothing and can be useful while rules are built explicitly.

Properties such as priority, model_name, document_type, tenant, or review_required let the broker route without parsing the JSON body. Keep their types and names stable, test overlapping rules, and remove the default true rule if it would defeat selective delivery. Filters control delivery; they do not replace authorization in the consumer.

from azure.identity import DefaultAzureCredential
from azure.servicebus.management import (
    ServiceBusAdministrationClient,
    SqlRuleFilter,
)

admin = ServiceBusAdministrationClient(
    "<namespace>.servicebus.windows.net",
    DefaultAzureCredential(),
)

admin.create_rule(
    topic_name="inference-results",
    subscription_name="priority-notifications",
    rule_name="high-priority-only",
    filter=SqlRuleFilter("priority = 'high'"),
)

Topic summary

Route topic publications with stable application properties and tested rules, while each subscription keeps independent receivers, backlog, retries, and failures.

radio_button_uncheckedshort_text11. Use the Python SDK sender and receiver lifecycle

The azure-servicebus package exposes ServiceBusClient plus queue and topic senders and queue or subscription receivers. Context managers close AMQP links and clients predictably. Reuse long-lived clients and links where practical instead of reconnecting for every message, and follow the current SDK reference because APIs and supported Python versions evolve.

from azure.identity import DefaultAzureCredential
from azure.servicebus import ServiceBusClient, ServiceBusMessage

namespace = "<namespace>.servicebus.windows.net"
credential = DefaultAzureCredential()

with ServiceBusClient(namespace, credential) as client:
    with client.get_queue_sender("inference-requests") as sender:
        sender.send_messages(ServiceBusMessage(
            '{"request_id":"req-917","model":"document-analyzer"}',
            content_type="application/json",
            message_id="req-917",
            correlation_id="trace-6d13",
            application_properties={"priority": "high"},
        ))

DefaultAzureCredential supports local developer sign-in and managed identity in Azure without changing business code. Assign the sender and receiver roles separately where different components have different duties. A topic sender is created with get_topic_sender(); a subscription receiver uses get_subscription_receiver(topic_name, subscription_name).

Topic summary

Use current azure-servicebus clients as managed resources, reuse connections appropriately, and authenticate each producer or consumer with a narrowly scoped Microsoft Entra identity.

radio_button_uncheckedshort_text12. Structure AI message bodies and properties

A message has a body, application properties, and broker-defined system properties. JSON is a common body format for a request containing request_id, model, parameters such as temperature and max_tokens, and references to input or context. Set content_type to application/json and version the contract so consumers can reject or transform incompatible payloads intentionally.

Validate schema, ranges, allowed models, URI ownership, and authorization before expensive inference. Do not place secrets in either the body or properties. Properties participate in the message quota and may be visible to operators who can inspect the entity.

Topic summary

Keep the versioned AI contract in the body, lightweight routing values in application properties, and delivery semantics in system properties.

radio_button_uncheckedshort_text13. Correlate, trace, deduplicate, and process idempotently

message_id identifies one logical message and is the key used by Service Bus duplicate detection within the configured history window. correlation_id connects API intake, queue work, inference, result publication, and downstream logs. They often hold related values, but their responsibilities differ. Propagate W3C traceparent and tracestate in application properties when OpenTelemetry spans must cross the broker boundary.

Duplicate detection protects against repeated sends with the same message_id; it does not prevent a locked message from being delivered again after a consumer failure or ambiguous settlement. A receiver therefore records a business operation or message identifier in a durable store and makes writes, notifications, and billing effects safe to repeat. Peek-Lock plus idempotency provides an effectively-once business outcome where designed; the broker delivery guarantee remains at least once.

Topic summary

Use message_id for send-side duplicate detection, correlation and trace context for observability, and durable idempotency for safe receive-side redelivery.

radio_button_uncheckedshort_text14. Apply the claim-check pattern to large payloads

A 500 MB document cannot be a Service Bus message, and even a technically allowed large payload can reduce throughput. Upload the document to private Azure Blob Storage, then publish a small claim check containing an opaque reference, content hash, size, media type, model, and business identifiers. The authorized worker retrieves and verifies the object before processing.

# 1. Upload the large document to private Azure Blob Storage.
blob_uri = upload_with_managed_identity(document_bytes)

# 2. Send only the claim check and routing metadata.
message = ServiceBusMessage(
    json.dumps({
        "request_id": request_id,
        "blob_uri": blob_uri,
        "sha256": payload_hash,
        "model": "document-analyzer",
    }),
    content_type="application/json",
    message_id=request_id,
    correlation_id=correlation_id,
)
sender.send_messages(message)

# 3. The authorized consumer retrieves, validates, and processes the blob.

Prefer managed identity and private networking for blob access; if a shared access signature is unavoidable, scope it narrowly and expire it quickly. Define ownership, retention, retry, and deletion so a failed send does not orphan blobs and a replay does not find prematurely deleted input. The claim-check pattern can also isolate sensitive payload content from intermediate messaging components.

Topic summary

Store large or sensitive input in secured object storage and send a small, verifiable reference whose retention lifecycle is coordinated with message processing.

radio_button_uncheckedshort_text15. Control freshness with TTL and throughput with batches

time_to_live expresses how long work remains useful. A real-time recommendation might expire quickly, while a batch analysis can remain valid longer. Expired messages are removed and can be moved to the dead-letter queue when dead-letter-on-expiration is enabled. Deferred messages have special expiration behavior, so do not use deferral as unattended long-term storage.

ServiceBusMessageBatch packs messages up to the protocol and tier limit and reduces network round trips. add_message raises when the next message does not fit; send the current batch, create another, and retry that message. A single oversized message still needs claim check. Batching improves transport efficiency but does not combine unrelated business operations into one atomic AI transaction.

from azure.servicebus.exceptions import MessageSizeExceededError

batch = sender.create_message_batch()

for payload in payloads:
    message = ServiceBusMessage(json.dumps(payload))
    try:
        batch.add_message(message)
    except MessageSizeExceededError:
        sender.send_messages(batch)
        batch = sender.create_message_batch()
        batch.add_message(message)

if len(batch) > 0:
    sender.send_messages(batch)

Topic summary

Set TTL from business usefulness, monitor expiration, and batch small messages within the SDK-calculated limit without treating a batch as a business transaction.

radio_button_uncheckedshort_text16. Receive with Peek-Lock and settle deliberately

from azure.servicebus import ServiceBusReceiveMode

with client.get_queue_receiver(
    queue_name="inference-requests",
    receive_mode=ServiceBusReceiveMode.PEEK_LOCK,
    max_wait_time=30,
) as receiver:
    for message in receiver:
        try:
            payload = json.loads(str(message))
            validate(payload)
            process_idempotently(payload, str(message.message_id))
            receiver.complete_message(message)
        except PermanentPayloadError as error:
            receiver.dead_letter_message(
                message,
                reason="InvalidPayload",
                error_description=str(error),
            )
        except TransientDependencyError:
            receiver.abandon_message(message)

Settle only after the required durable effects complete. A timeout around settlement is ambiguous: the broker might have applied it even if the client did not receive confirmation. That is another reason the processing effect must be idempotent.

Topic summary

Peek-Lock favors recovery over silent loss: complete success, abandon transient failure, dead-letter permanent failure, and defer only when the retrieval plan is explicit.

radio_button_uncheckedshort_text17. Renew locks and operate the dead-letter queue

The default entity lock duration is one minute and can be configured up to five minutes. If legitimate processing can exceed it, renew the message lock manually or register it with AutoLockRenewer for a bounded period. Avoid receiving or prefetching more messages than the worker can finish before their locks expire. For operations that routinely last far longer, consider recording accepted work durably, completing the broker message quickly, and running the long task through a separate tracked state machine.

from azure.servicebus import AutoLockRenewer

with AutoLockRenewer() as renewer:
    with client.get_queue_receiver("inference-requests") as receiver:
        for message in receiver:
            renewer.register(
                receiver,
                message,
                max_lock_renewal_duration=600,
            )
            run_long_inference(message)
            receiver.complete_message(message)

Each queue and subscription has a dead-letter subqueue. A message that exceeds maxDeliveryCount—10 by default—moves there with MaxDeliveryCountExceeded; an application can also dead-letter with its own reason and description. DLQ messages remain until a receiver explicitly completes them. Alert on count and age, inspect payload and diagnostics, repair the systemic cause, and replay only through an approved idempotent process.

from azure.servicebus import ServiceBusSubQueue

with client.get_queue_receiver(
    queue_name="inference-requests",
    sub_queue=ServiceBusSubQueue.DEAD_LETTER,
    max_wait_time=10,
) as dlq_receiver:
    for message in dlq_receiver:
        inspect(
            message.dead_letter_reason,
            message.dead_letter_error_description,
            message.delivery_count,
            message.correlation_id,
        )
        # Re-submit only after fixing the cause and preserving idempotency.
        replay_if_approved(message)
        dlq_receiver.complete_message(message)

Topic summary

Renew locks for bounded long processing, control delivery attempts, and run the DLQ as an observable repair workflow—not as a forgotten archive.

radio_button_uncheckedshort_text18. Guided lab, assessment review, and production checklist

The source exercise creates a namespace and Python Flask application, sends and receives queue messages in Peek-Lock mode, inspects malformed input in the DLQ, and publishes inference results to filtered subscriptions. Plan about 30 minutes after prerequisites: an Azure subscription, Visual Studio Code, Python 3.12 or later, and the current Azure CLI.

az group create --name ai200-servicebus-rg --location eastus
az servicebus namespace create   --resource-group ai200-servicebus-rg   --name <globally-unique-namespace>   --location eastus   --sku Standard

az servicebus queue create   --resource-group ai200-servicebus-rg   --namespace-name <namespace>   --name inference-requests   --max-delivery-count 5

az servicebus topic create   --resource-group ai200-servicebus-rg   --namespace-name <namespace>   --name inference-results

python -m venv .venv
python -m pip install --upgrade azure-identity azure-servicebus flask

1. Create the namespace, inference request queue, result topic, and subscriptions; configure max delivery count and filters deliberately.
2. Assign managed identities the Sender or Receiver data role and use DefaultAzureCredential.
3. Send valid JSON plus one invalid message; process valid work idempotently under Peek-Lock.
4. Confirm the invalid message reaches the DLQ with a useful reason, then repair and replay it once.
5. Publish one result with priority properties and prove only matching subscriptions receive it.
6. Observe active messages, oldest age, completions, retries, lock loss, and dead-letter count; delete lab resources when finished.

Before production, confirm tier and quotas, entity definitions as code, least-privilege identity, local-auth policy, private networking where required, contract validation, TTL, duplicate window, session and filter tests, idempotency store, lock and prefetch sizing, retry policy, DLQ ownership, monitoring, alert thresholds, recovery runbooks, cost, and load tests against the real model endpoint.

1. Microsoft Learn: Introduction to Azure Service Bus messaging
2. Microsoft Learn: Service Bus queues, topics, and subscriptions
3. Microsoft Learn: Prevent message loss and duplicate processing
4. Microsoft Learn: Service Bus quotas and limits
5. Microsoft Learn: Service Bus dead-letter queues
6. Microsoft Learn: Python client library for Azure Service Bus
7. Azure Architecture Center: Claim-Check pattern
8. Microsoft Learn: Authenticate Service Bus applications with Microsoft Entra ID

Topic summary

The complete design combines the right entity, small versioned messages, identity, filters, correlation, Peek-Lock, idempotency, lock control, monitored DLQ recovery, and evidence from an end-to-end test.