Generative models and agents in Microsoft Foundry

radio_button_uncheckedshort_text1. Generative AI, innovation, and Microsoft Foundry

Generative AI is the branch of artificial intelligence that creates new content. It has quickly changed work and expanded what software can do; because progress moves so fast, even experienced developers benefit from a stable framework for understanding models, tools, and development patterns.

Microsoft Foundry supports a complete workflow: discover, compare, evaluate, and deploy a model; test the deployment in a Playground; and integrate it through code. An agent adds instructions and tools to produce reusable, consistent behavior in both the portal and client applications. This chapter is the detailed reading version; video materials can be an alternative, while text generally supplies additional context.

radio_button_uncheckedshort_text2. Model catalog and foundation models

LLMs underpin many generative and agentic systems, but no single model is best for every workload. A lightweight phone app may favor a small language model, while a government system might require a domain-specialized model. Microsoft Foundry requires an Azure subscription and a Foundry project.

The Foundry model catalog is the central place to search by provider, source, capability, and inference task. It combines models sold directly by Azure with partner and open-community choices.

• Azure-direct models are hosted by Microsoft under Microsoft Product Terms and provide deep Azure integration, enterprise SLAs, preconfigured security, and compliance alignment.
• Partner and community models broaden experimentation and rapid innovation through open, vendor-hosted, and domain-specific options.
• A catalog entry typically documents text generation, reasoning, coding, multimodal, and embedding capabilities; benchmarks; inference tasks; fine-tuning support; model cards; constraints; and responsible-AI caveats.

A foundation model is a large, pretrained model with general language, reasoning, or multimodal capabilities. GPT, Claude, Mistral, and related families can be deployed immediately or customized with fine-tuning and become the base layer of an AI application.

radio_button_uncheckedshort_text3. Common families and task-based selection

A model family shares an architecture or lineage while varying in size, version, specialization, or capability. Several families stand out among the catalog’s thousand-plus choices.

• GPT-5.x emphasizes multi-step reasoning, structured logic, planning, long context, and agent workflows. Adjustable reasoning levels trade speed for accuracy.
• Anthropic Claude Opus 4.5 targets sophisticated agents, complex code reasoning, and computer-use tasks, with large context and output windows for long specifications, multi-file diffs, and extended research.
• Mistral AI Mistral Large 3 is a strong general model that balances quality, cost, latency, and throughput for multilingual drafting, structured reports, and mid-latency agents.
• The GPT-5 family may require registration and have restricted availability. GPT-4.1 remains broadly accessible for real-time chat, support, and high-volume interactive systems that value speed and low latency.

For a tightly defined use case, a Foundry tool can be more appropriate than selecting a raw catalog model. These tools use prebuilt models to deliver predictable behavior, built-in compliance, and faster time to value.

radio_button_uncheckedshort_text4. Benchmarks, comparisons, and evaluators

Selection begins with task type, traffic, latency, cost, safety requirements, and business constraints. Catalog benchmarks use consistent datasets, while leaderboards rank quality, safety, and throughput. Side-by-side views compare accuracy, price, compliance, performance, industry, use case, model type, and license.

A useful workflow is to open a model, select Benchmarks, and try representative prompts with your own data. Classic NLP metrics include accuracy, precision, recall, and F1. AI-assisted quality metrics cover groundedness, relevance, coherence, fluency, and GPT similarity.

Evaluators measure the quality, safety, and effectiveness of model or agent outputs. The Evaluator Library provides reusable components, while safety evaluators scan for harmful content, bias and unfairness, violence, self-harm, and protected-class harms. Evaluators detect and score concerns; by themselves, they do not repair the output.

radio_button_uncheckedshort_text5. Deployment, TPM, quotas, and throttling

Deployment turns a configured model into a stable, secure, scalable service endpoint. It locks a dependable combination of model version, response behavior, and safety settings, supports consistent performance, and helps restrict unauthorized or unsafe access.

• Deployment types such as standard, global batch, and regional provisioned throughput determine where and how inference runs and relate to throughput and data-processing needs.
• Model version selects the exact implementation behind the endpoint.
• Tokens per minute (TPM) allocation controls token traffic capacity and helps define boundaries such as requests per minute (RPM).

A token is the smallest unit of text or data processed by a model, such as a word, subword, character, or punctuation mark. Higher TPM supports more traffic; lower TPM constrains aggregate consumption. High-end reasoning models such as DeepSeek R1, Grok, and large Llama versions may expose high TPM ceilings, while image or specialized models can use capacity units instead.

Throttling temporarily limits compute work near a processing boundary to keep the service stable and responsive. Long prompts, large maximum outputs, and high concurrency consume quotas and can trigger rate-limit errors; reducing max output tokens or concurrent requests can help.

radio_button_uncheckedshort_text6. Playground and runtime parameters

The Foundry model Playground is the simplest way to interact with a deployment. It supports prompt testing, model comparison, and capture of working instructions and settings before application code is written.

• Temperature balances determinism and creativity.
• Max output tokens caps response length and influences token consumption and throttling.
• System instructions establish role, behavior, tone, available tools, guardrails, and output format.
• A user prompt carries the end user’s request, while the system message governs how the assistant handles it. A tutor might explain step by step, cite sources, and decline medical advice.

After representative testing, the same system message, user request, and parameter values can move into code. Code view exposes the operation behind the chat UI and supplies a starting point for the OpenAI-compatible Responses API.

radio_button_uncheckedshort_text7. Lightweight client, Foundry SDK, and Responses API

A lightweight client gathers input, calls a remote API, and displays output without a heavy UI framework, complex backend, or large local dependency set. It may be a CLI, desktop utility, or simple web page. The remote service performs model computation, keeping the prototype small, easy to run, and easy to extend.

For a Foundry project with a deployed chat model such as gpt-4.1, the Foundry SDK exposes a project client for Foundry-native operations and an OpenAI-compatible client for the Responses API. Applications commonly use both.

# Install: pip install azure-ai-projects azure-identity "openai>=1.3.0"
import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['AZURE_OPENAI_ENDPOINT']}/openai",
    api_key=os.environ['AZURE_OPENAI_API_KEY']
)

result = client.responses.create(
    model=os.environ['DEPLOYMENT_NAME'],
    input=[
        {'role': 'system', 'content': 'You are a concise, helpful assistant.'},
        {'role': 'user', 'content': 'Summarize our release notes in three bullets.'}
    ],
    max_output_tokens=300,
    temperature=0.7
)
print(result.output_text)

The sample loads its endpoint, API key, and deployment name from environment variables, authenticates, sends system and user messages, and prints the returned text. The model runs remotely while the client remains minimal.

radio_button_uncheckedshort_text8. Model versus agent

Models and agents operate at different layers. A model supplies raw intelligence: it accepts a prompt and performs inference. It fits direct Responses API calls and Playground experiments where the goal is simply to generate an output from an input.

An agent is an application on top of the model. It packages task-oriented behavior, chooses among actions, and reuses the same rules across apps, experiences, and services.

radio_button_uncheckedshort_text9. Agent anatomy and creation

A reusable Microsoft Foundry agent brings together three essentials: a model for language and reasoning, instructions that establish role, behavior, style, constraints, and output rules, and tools that perform actions. Knowledge is an optional source of external context.

• Agents can call APIs, functions, retrieval systems, and other external services.
• They can decompose goals into steps, maintain working memory during a conversation, choose actions, and produce structured output.
• In the portal, select a model, write system instructions — for example, a scheduling assistant that answers with concise bullets — and add tools and knowledge.

radio_button_uncheckedshort_text10. Tools, knowledge, and RAG

Tools: the ability to act

Tools let an agent search the web, query a database, read or write files, run code, invoke functions and APIs, or use MCP servers. The model inspects available capabilities and selects one when a request requires action. The Foundry Tool Catalog centralizes discovery and management. Code Interpreter, knowledge sources, and custom functions are common examples.

Knowledge: retrievable context

Knowledge connects documents and datasets through retrieval-augmented generation (RAG). Sources can include internal PDFs, SharePoint content, Azure Storage files, and multisource knowledge bases. The retrieval pipeline ingests and indexes material, searches relevant passages, and grounds the answer.

• Question answering can rely on organizational documents.
• Responses gain domain context, accuracy, and traceability.
• When a knowledge source is used, the answer can cite the store that supplied it.
• This supports enterprise-safe retrieval while keeping knowledge separate from actions.

radio_button_uncheckedshort_text11. Save, publish, and invoke an agent

The portal can save the model, instructions, and tools as an agent and continue testing it in the Playground. Publishing creates a managed Azure resource with a stable endpoint that can be shared and integrated without exposing the Foundry project or source code.

A client application uses the Foundry Projects SDK and Project API to connect agents to web apps, bots, and backend workflows, orchestrate multi-step tasks, pass structured inputs or tool calls, and run at scale. The agent ID is available from Code view in the .env variables.

# Install: pip install --pre "azure-ai-projects>=2.0.0b1" azure-identity
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

project = AIProjectClient(
    endpoint='https://YOUR-RESOURCE.services.ai.azure.com/api/projects/YOUR-PROJECT',
    credential=DefaultAzureCredential()
)
agent = project.agents.get(agent_name='learning-agent')
openai_client = project.get_openai_client()

response = openai_client.responses.create(
    input=[{'role': 'user', 'content': 'Tell me which tasks you can help with.'}],
    extra_body={'agent': {'name': agent.name, 'type': 'agent_reference'}}
)
print(response.output_text)

The call retrieves the named agent and references it through extra_body. Without agent_reference, the application would call only a model deployment. The project endpoint, Microsoft Entra ID, and DefaultAzureCredential provide authentication without embedding a secret in source code.

radio_button_uncheckedshort_text12. Guided Microsoft Foundry exercise

The guided exercise deploys and explores a generative model in Microsoft Foundry and then uses that model in an agent with knowledge tools to answer questions. It requires an Azure subscription; a new Azure account may include credits for the first 30 days.

• Start the original Microsoft Learn exercise

radio_button_uncheckedshort_text13. Knowledge check

Reworded questions

1. Is the catalog Microsoft-only, a hub for models from many providers, or a replacement for an Azure subscription?
2. Is a foundation model a small unusable model, a benchmark tool, or a large pretrained model with general capabilities?
3. Why use the Playground before code: to avoid APIs, generate every instruction automatically, or validate reusable prompts and settings?
4. Does publishing an agent make execution free, restrict it to the portal, or create a managed resource with a stable endpoint?
5. In the Python sample, which part invokes the published agent instead of a model deployment alone?

Explained answers

• The catalog discovers, filters, compares, and tests models from multiple providers.
• A foundation model is large, pretrained, immediately useful, and customizable.
• The Playground validates behavior and captures settings that code can reproduce.
• Publishing creates an integrable managed Azure resource; model, token, tool, and data costs still apply.
• responses.create carries extra_body with agent_reference; retrieving the agent and client prepares the call, while the reference selects the agent.

radio_button_uncheckedshort_text14. Chapter summary

• Language models power generative AI, and the Foundry catalog combines Azure-direct, partner, and community choices.
• Task, quality, latency, cost, safety, benchmarks, and tests with your own data guide model selection.
• Deployment allocates compute, creates an endpoint, locks settings, and enables metrics; TPM, RPM, and throttling govern capacity.
• The Playground turns prompt experiments into reproducible Responses API and SDK configuration.
• A model performs inference; an agent combines a model, instructions, tools, memory, and knowledge into a reusable workflow.
• RAG grounds answers in documents, while tools act through files, APIs, databases, and MCP servers.
• The Foundry Projects SDK and Project API bring published agents into web apps, bots, and backend workflows with identity, scale, and governance.

• Explore Microsoft Foundry Models
• Learn about Foundry Agent Service
• Build agents with the Responses API