← All posts

Building a Stateful Multi-Agent System with LangGraph and Microsoft Foundry Agent Service, From Scratch

5 min read3.4k
#Microsoft#Foundry#Azure#LangGraph#AI#AIAgents

LangGraph gives you fine-grained control over agent state and routing logic in code. Microsoft Foundry gives you managed hosting, identity, tracing, and one-click publishing to Teams and M365 Copilot. Increasingly, teams want both: LangGraph's graph-based orchestration for the hard routing logic, and Foundry's operational layer for everything downstream of "it works on my machine."

This is a from-scratch walkthrough of wiring the two together — building a small multi-agent support triage system in LangGraph, then hosting it as a Foundry Hosted Agent.

Architecture

The system has three LangGraph nodes: a triage node that classifies the incoming request, and two specialist nodes (billing, technical) that the triage node routes to conditionally.

Once this graph is compiled, it doesn't run as a standalone script — it gets wrapped and exposed through Foundry's hosting layer, which handles the runtime, session management, and identity so the graph itself stays free of infrastructure concerns.

Step 1: Define the graph state and nodes

python
from typing import TypedDict, Literal
from langgraph.graph import StateGraph, END
from langchain_azure_ai.chat_models import AzureAIChatCompletionsModel
import os

class TicketState(TypedDict):
    user_input: str
    ticket_type: Literal["billing", "technical", ""]
    response: str

model = AzureAIChatCompletionsModel(
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    model=os.environ["FOUNDRY_MODEL_NAME"],
)

def triage_node(state: TicketState) -> TicketState:
    result = model.invoke(
        f"Classify this support ticket as 'billing' or 'technical' only: {state['user_input']}"
    )
    ticket_type = "billing" if "billing" in result.content.lower() else "technical"
    return {**state, "ticket_type": ticket_type}

def billing_node(state: TicketState) -> TicketState:
    result = model.invoke(f"You are a billing specialist. Resolve: {state['user_input']}")
    return {**state, "response": result.content}

def technical_node(state: TicketState) -> TicketState:
    result = model.invoke(f"You are a technical specialist. Resolve: {state['user_input']}")
    return {**state, "response": result.content}

Step 2: Wire the conditional routing

python
def route(state: TicketState) -> str:
    return state["ticket_type"]

graph = StateGraph(TicketState)
graph.add_node("triage", triage_node)
graph.add_node("billing", billing_node)
graph.add_node("technical", technical_node)
graph.set_entry_point("triage")
graph.add_conditional_edges("triage", route, {"billing": "billing", "technical": "technical"})
graph.add_edge("billing", END)
graph.add_edge("technical", END)

compiled_graph = graph.compile()

Notice the model calls point at FOUNDRY_PROJECT_ENDPOINT, not directly at Azure OpenAI or any single provider's SDK. This is what lets the same graph run locally against a Foundry project and later behind the hosted runtime without changing model-calling code.

Step 3: Expose the graph for Foundry hosting

Foundry's langchain_azure_ai.agents.hosting package wraps a compiled LangGraph graph so it can be deployed as a Hosted Agent, speaking either the Responses or Invocations protocol.

python
from langchain_azure_ai.agents.hosting import LangGraphResponsesHost

host = LangGraphResponsesHost(graph=compiled_graph)
app = host.create_app()

That app is what gets deployed — Foundry manages the runtime, session handling, and scaling from there. Locally you can run it with uvicorn app:app --reload and test over HTTP before deploying with azd ai agent deploy or the Foundry Toolkit extension for VS Code.

Local tools vs. built-in tools

One distinction that trips people up: LangGraph tools you attach in code run locally, colocated with your graph. Foundry's own built-in tools (file search, code interpreter, etc.) run server-side inside the Agent Service and don't add a node to your graph at all — they're invoked transparently when the service handles a request.

AspectLocal (LangGraph) toolsBuilt-in Foundry tools
Execution locationColocated with your graph processServer-side in Foundry Agent Service
Graph representationAdds a tool nodeNo node added
Applicable toAny LangGraph/LangChain toolFoundry agents only
Best forDeterministic business logic, custom APIsFile search, code interpreter, managed retrieval

Tracing across two systems

Because the LangGraph portion runs outside Foundry's native agent runtime during local development, its traces don't automatically appear in the Foundry portal. You view LangGraph-level traces through Azure Monitor / Application Insights (Investigate → Agents), while Foundry-native agent traces show up directly in the Foundry portal. Once deployed as a Hosted Agent, LangChain and LangGraph applications can also be registered in the Foundry Control Plane for unified governance and trace visibility.

What this buys you in production

Once deployed, the same graph becomes available as a turnkey Application across Teams, M365 Copilot, and Bot Service channels without additional publishing code — Foundry handles the channel plumbing, while your LangGraph code stays focused purely on routing and reasoning logic.

References