Turn business requirements into reliability targets, resilient and recoverable architecture, observable operations, and deliberately simple solutions.
Suggested study time: 88 minutes • Intermediate • Complete original rewrite with a concise summary for every topic
By João Ricardo Dutra••Complete original content
1. Treat reliability as a workload capability
The Reliability pillar of the Azure Well-Architected Framework asks whether a workload can remain available, withstand faults, and return to a healthy state after disruption. Outages, degraded dependencies, faulty deployments, exhausted capacity, and data corruption are normal risks in a distributed system. A reliable design therefore detects trouble, contains its effect, continues the most important journeys at an agreed quality level, and recovers predictably.
Reliability is not an infrastructure add-on. Application code, data design, Azure resources, delivery practices, monitoring, and incident operations must support the same business promises. Every measure has a cost or complexity trade-off, so architects need explicit requirements rather than an undefined demand for “always on.”
Business requirements, resilience, recovery, operations, and simplicity work as one reliability system.
Topic summary
Design reliability across code, infrastructure, data, and operations, and measure it against business promises rather than an abstract uptime ideal.
2. Use five connected design principles
Reliability principles and their central question.
Principle
Architectural question
Business requirements
Which flows matter, and what measurable service and recovery targets do they require?
Resilience
How will the workload preserve full or reduced functionality when a fault occurs?
Recovery
How will components and state return to a trusted operating condition?
Operations
How will teams see, predict, test, and learn from reliability risks?
Simplicity
Which components, custom code, and processes can be removed or standardized?
These principles are interdependent. Redundancy without monitoring can hide a failing replica; recovery without drills is only a document; autoscale without a defined flow objective can raise cost without protecting the right user journey. Reliability must also be balanced with Performance Efficiency and Cost Optimization.
Topic summary
Use the five principles as one decision framework, and record the cost, performance, and operational trade-offs of every reliability choice.
3. Start the Contoso Insurance design with business requirements
Consider a fictional insurer building a claims-processing web application. The proposed solution combines ,, Azure AI services, , and . Before choosing tiers or replication, the workload team and business stakeholders identify user experience, data, workflows, compliance outcomes, budget, and unique operating constraints.
The team divides the application into user and system flows and ranks them. Submission and approval of a claim receives the highest criticality because doctors and patients depend on it. Supporting components can then inherit appropriate targets instead of every component receiving the most expensive design.
Topic summary
Begin with shared, achievable business expectations and flow criticality; service selection and spending follow those decisions.
4. Define targets for flows, components, and the whole workload
Define service-level objectives and recovery measures for individual components, each important system or user flow, and the workload as a whole. Metrics turn intent into testable acceptance criteria and reveal whether the required architecture fits the budget. Useful measures include availability, latency, error rate, throughput, recovery time objective (RTO), recovery point objective (RPO), mean time to recover, and the time needed to return from a degraded state.
Compliance-sensitive flows need unambiguous outcomes. A monitoring system that observes the workload is a dependency and operational control, but it is not itself one of the workload scopes for which the exercise asks teams to define business-flow reliability targets. Keep the distinction between workload objectives and the tools used to measure them.
Targets remain traceable from stakeholder outcomes to the resources that support each flow.
Topic summary
Attach measurable targets to critical flows, their components, and the end-to-end workload, then test both normal and recovery behavior.
5. Understand platform commitments and constraints
Azure service-level agreements differ by service, tier, region, deployment topology, and feature. Read the Microsoft SLAs for online services to understand guaranteed uptime; a pricing page, FAQ, or general product article is not the authoritative commitment. Also account for service limits, quotas, regional capacity, maintenance behavior, and which features are excluded from an SLA.
Suppose the claims team requires critical data flows to recover within 30 seconds. After validating the applicable commitments and failure model, it can evaluate the business-critical tier of with active geo-replication. The architecture still needs monitoring, tested failover, connection handling, data-loss analysis, and sufficient secondary capacity; purchasing a tier alone does not satisfy the RTO.
Topic summary
Base reliability claims on the applicable SLA, limits, quotas, capacity, and tested topology—not on a service name or marketing description.
6. Map every internal and external dependency
Document infrastructure, APIs, functions, data sources, identity, DNS, certificates, networks, third-party services, and processes owned by other teams. For each dependency, record the flows that need it, its reliability target, failure behavior, timeout, data-freshness tolerance, owner, and escalation path. This exposes cascading failures and downstream operational consequences.
The claims flow depends on a small reference dataset managed by another department. Its availability objective is lower than the claims flow, while the application can accept bounded staleness but not absence. A local cache removes the runtime dependency; a background refresh keeps it sufficiently current. The refresh schedule must match the stated business tolerance—nightly is valid only when that age is acceptable.
Decoupling a dependency changes failure behavior but also introduces freshness and reconciliation decisions.
Topic summary
Include all internal and external dependencies, then use caching, queues, isolation, or fallback only where the resulting consistency trade-off is acceptable.
7. Design the Contoso Air booking flow for resilience
A fictional airline runs a booking solution on ,,,, and . Ticket purchase is the most important user flow. The team focuses its limited improvement effort on the paths whose failure has the greatest business and customer impact.
An external payment gateway is highly available but can still return transient errors during network disruption or bursts of concurrent requests. The original synchronous design forces customers to resubmit rejected payments. The reliability task is to change that failure experience without duplicating a charge or losing an order.
Topic summary
Prioritize resilience work by flow criticality and business impact, even when the weak point is a highly available external service.
8. Perform failure mode analysis before testing faults
For each potential failure, describe the initiating condition, affected component, blast radius, intensity, duration, detection signal, user-visible effect, and recovery path. Include small failures such as a temporarily unavailable process, capacity exhaustion and throttling, and large events such as zone or regional loss. Rate likelihood and impact so the team spends on the highest risks.
A DDoS exercise or other disruptive test should be preceded by failure mode analysis. The team needs an expected failure boundary, protection and abort conditions, owner approvals, telemetry, and a safe recovery plan before introducing stress.
Fault testing is safe and useful only when the expected behavior and boundaries are known.
Topic summary
Use failure mode analysis to rank risks and define controlled experiments before injecting a fault into any environment.
9. Build self-preservation with retry and isolation
Self-preservation prevents one failing dependency from consuming the rest of the system. Modular boundaries, timeouts, retry, circuit breaking, bulkheads, load leveling, idempotency, and graceful degradation address different failure modes. Retry only operations that are safe to repeat and errors classified as transient; use increasing delays, jitter, and a maximum attempt count to avoid amplifying overload.
For the airline, separates the customer interaction from payment processing. A worker retries temporary failures with increasing waits. After the attempt limit, it stops hammering the gateway and retains or transfers the message for controlled later handling. Correlation, deduplication, poison-message handling, visibility to the customer, and reconciliation are essential because asynchronous delivery changes the business workflow.
Queue-based isolation protects the user flow and the dependency while preserving work for recovery.
Topic summary
Self-preservation combines bounded retry, isolation, idempotency, backpressure, and a durable failure path rather than endless automatic attempts.
10. Add redundancy at the right layers
Redundancy can cover power and facilities, availability zones, regions, data replicas, service instances, network paths, functions, operational processes, and people. Choose active-active or active-passive arrangements from targets, consistency, failover time, capacity, cost, and operational maturity. Intermediaries such as queues buffer demand and reduce direct runtime coupling.
The airline can use the premium tier of with availability-zone support so service data is spread across separate facilities in a supported region. Geo-disaster recovery provides a secondary namespace configuration for regional continuity; architects must distinguish metadata replication from message-data replication and validate the current feature behavior. Redundancy is proven by drills, not merely by deployment.
Each redundant layer addresses a different blast radius and has its own recovery behavior.
Topic summary
Enable geo-replication when it fits the data and recovery model, and test the exact zone or regional failover behavior promised by the selected service.
11. Prepare the Contoso analytics platform for recovery
A fictional analytics solution ingests on-premises SQL Server data with , processes it, and uses ,,, and . A legacy Windows process runs on a VM. Because users are internal and the workload is not mission-critical, the team initially deploys everything in one Azure region and accepts rebuilding elsewhere after a regional disaster.
That choice is valid only when the recovery targets, required source data, infrastructure definitions, identities, network configuration, secrets, capacity, and deployment procedure make a rebuild achievable within the promised time.
Topic summary
A single-region architecture can be appropriate when its tested rebuild process and data protections satisfy explicit business recovery targets.
12. Create and drill a complete recovery plan
A disaster recovery plan covers every component and the end-to-end system, with decision authority, detection and declaration criteria, communications, failover, validation, failback, rollback, data integrity, and exit conditions. Store the runbook where it remains reachable during the event. Assign owners and record dependencies, credentials, automation, target capacity, DNS, and third-party actions.
The analytics team wrote a secondary-region rebuild plan but did not rehearse it. During a real outage, missing deployment steps delayed recovery. The correct response is to fix the runbook and advance a full drill. Measure outcomes such as the average time to restore a database from backup because this evidence directly informs a recovery plan.
A plan becomes a recovery capability only after realistic rehearsals close its gaps.
Topic summary
Document, automate, own, and repeatedly rehearse failover and failback; use measured restore time to validate RTO assumptions.
13. Recover stateful data from a trusted point
Every stateful component needs a recovery method that satisfies its RPO and RTO. Backups should be immutable where required, transactionally consistent, protected from the same failure domain, retained according to policy, and routinely restored. Replication improves availability and RPO but can copy corruption or deletion, so it does not replace independent recovery points.
In the scenario, databases larger than 4 TB are placed on Azure running SQL Server 2022. Automated Backup protects all databases. Critical databases also use Managed Instance link to replicate toward . Current link direction, failover, failback, update policy, version, and regional prerequisites must be verified. Moving a single-region deployment to multiple regions is another stateful recovery strategy.
Topic summary
Combine restorable backups and, where justified, replication; validate consistency, immutability, version support, failover, and failback for each state store.
14. Add automated self-healing
Self-healing uses health signals and bounded automation to restart, reimage, replace, or redirect an unhealthy component. It shortens recovery and reduces manual error, but it needs grace periods, attempt limits, safe state handling, escalation, and observability so automation does not create a repair loop.
The legacy Windows process can run in . The Application Health extension reports instance health, and automatic instance repairs can restart, reimage, or replace an unhealthy instance according to the supported configuration. actions can also trigger an approved application restart. evaluates governance compliance and provides recommendations; neither is the direct self-healing action in this example.
Data recovery and component repair solve different parts of the recovery objective.
Topic summary
Drive automated repair from trustworthy health signals, constrain the action, preserve state, and escalate when repeated recovery fails.
15. Operate the Contoso University microservices visibly
A fictional university runs five decoupled services across several tiers with ,,,,,, and . A student request can cross multiple services, so a green status for each isolated resource does not prove that registration or course access works.
Operations must understand the end-to-end journey, the health of each dependency, and ongoing failures. Shared visibility improves triage, root-cause analysis, post-incident learning, and prioritization by site reliability engineers.
Topic summary
Monitor user and system flows across service boundaries, not only the infrastructure health of individual resources.
16. Correlate telemetry into an observable system
Instrument the application to emit telemetry and combine it with platform logs and metrics. Application Insights captures application behavior; relevant signals are sent to an Log Analytics workspace. Consistent correlation identifiers connect the same request across tiers, queues, and databases so teams can locate latency, errors, and bottlenecks.
A useful observability model covers symptoms visible to users, component saturation, dependency health, deployment changes, business transactions, and recovery automation. Telemetry requires retention, access control, cost management, sampling, and protection of sensitive data.
Observability explains when, where, and why an end-to-end flow degraded.
Topic summary
Making a workload observable means emitting and correlating actionable telemetry across the complete journey, with appropriate governance.
17. Predict problems with actionable alerts and capacity
Alerts should indicate a condition that requires an owner and a defined response. Prioritize them by affected flow and urgency, reduce duplicates, route them to a reliable on-call process, and include context and a runbook. Informational signals belong in dashboards or reports when nobody needs immediate action.
The university expects a start-of-term surge. It prepares web and database tiers to scale out and updates the incident plan so alerts for registration, syllabus access, and purchasing receive priority. Scheduled headroom and metric-driven scaling should be load-tested against quotas, database limits, downstream capacity, scale delay, and cost.
Topic summary
Combine tested capacity and autoscale with prioritized, actionable alerts that name an owner and a response.
18. Test reliability in preproduction and production
Exercise component faults, degraded modes, recovery paths, and business thresholds in preproduction and, with safeguards, in production. A tabletop exercise tests decisions and communication; a load test evaluates demand; directly simulating a component failure proves whether the workload continues in a degraded state.
The university uses client certificates for a seasonal registration REST service. Monthly synthetic transactions keep the flow active and alert on failure or approaching certificate expiry. Routine chaos experiments become part of the software development life cycle, with hypotheses, scope, stop conditions, monitoring, and captured results. The evidence validates self-preservation and reveals unknown coupling.
Topic summary
Use synthetic transactions and controlled chaos experiments to prove real user flows, degraded operation, and recovery before an incident does it for you.
19. Simplify the Contoso Travel workload
A fictional travel company acquires a Node.js application running on VMs in a datacenter and AWS. The architecture includes a little-used website feedback feature, while customers usually respond through the company’s social channels. Every extra feature adds code, dependencies, deployment risk, monitoring, and support effort.
For the first branded release, the team removes the feedback component. The smaller codebase costs less to operate and does not weaken an agreed business requirement. Simplicity is not blind removal: oversimplification can introduce a single point of failure, so each reduction must be checked against reliability targets.
Topic summary
Keep only components justified by a business outcome, while ensuring the simpler design does not remove required redundancy.
20. Standardize the software development life cycle
The acquired team uses overlapping libraries, inconsistent code styles, and release pipelines without formal automated gates. Large releases often require rollback or emergency fixes, consume the whole team, damage reputation, and create a poor customer experience.
Standardize supported languages, frameworks, libraries, patterns, naming, coding style, documentation, tests, and deployment controls. Automated formatting, static analysis, security checks, regression tests, and release gates catch deviations early. Document new tools and patterns so reviews and troubleshooting share the same reference.
Topic summary
Standardization across tools, documentation, and testing reduces human variation and makes releases, maintenance, and incident response repeatable.
21. Prefer managed platform capabilities over custom burden
The travel team moves the Node.js application from VMs to . Application Insights autoinstrumentation replaces custom monitoring code, while capabilities provide autoscaling, integration, and zone redundancy where supported. The move reduces code and infrastructure the team must own; it does not automatically make the application serverless or remove the need for architecture and testing.
Managed capabilities remove undifferentiated code and operations while retaining explicit reliability controls.
Topic summary
Use proven platform features when they meet the requirement, and spend custom engineering effort only where it creates distinct business value.
22. Review the complete reliability decision
Rank user and system flows, then define measurable availability and recovery targets.
Validate Azure SLAs, limits, quotas, capacity, and every internal or external dependency.
Perform failure mode analysis and build bounded self-preservation and layered redundancy.
Protect state with tested backups and appropriate replication; rehearse failover and failback.
Correlate telemetry, route actionable alerts, and test degraded behavior through synthetic and chaos exercises.
Remove unjustified components, standardize the SDLC, and prefer suitable managed capabilities.
Exam decisions should connect the requirement to a failure model, an Azure capability, evidence from a test or drill, and an operational owner. A geo-replica, queue, health probe, or alert is useful only when its behavior satisfies the target and its trade-offs are understood.
Reliable Azure architecture is a continuously tested agreement among business targets, resilient design, recoverable state, observable operations, and disciplined simplicity.
