Class 7: Handing the Tools to a Model

Duration: ~85 minutes | Level: Intermediate | Prerequisites: Class 6: Connecting a Client, and either an API key from Anthropic or OpenAI, or Ollama running locally. This is the first class that needs a model.

What We'll Cover

Configuring a provider, and keeping the API key out of the repository
Running a model on your own machine instead, without an account
ChatClient, the discovered tools, a system prompt and conversation memory
What happens when a tool fails, and why our code does not see it by default
Streaming, and the endpoint the browser talks to
A time limit on the model call, so a stalled model becomes an error instead of a hang

Companion code

This class carries on from Class 6. If you followed along, keep working in the project you already have. If you skipped it, clone the class_6 branch to start from the same place:

git clone --branch class_6 https://github.com/the-mcp-guy/spring-ai-mcp-course.git

Set the Key as an Environment Variable

This is the first class that costs money. Both providers bill per token, the unit a model reads and writes text in: roughly a short word, or a piece of one. A support question in this class sends a few thousand tokens and gets a few hundred back, so working through the class should cost only a few cents.

Treat that as a rough guide. Prices change, and what you spend depends on how much you experiment. The Anthropic and OpenAI pricing pages have the current rates.

The three ways to get a model for this class:

Route	Account	Where the key comes from	Environment variable	What it costs
Anthropic	yes, with credit on it	console.anthropic.com	`ANTHROPIC_API_KEY`	$3 per million input tokens and $15 per million output tokens for `claude-sonnet-4-6`
OpenAI	yes, billed separately from ChatGPT	platform.openai.com	`OPENAI_API_KEY`	the rates on the OpenAI pricing page
Ollama, on your own machine	none	nothing to get: `ollama pull qwen3:8b`	none	nothing

The Ollama row is the one to take if you would rather not spend anything, and The Same Agent, on a Local Model runs everything below against it.

Where to get an API key (click to expand)

Anthropic. Sign in at console.anthropic.com and create a key under Settings → API keys. It starts with sk-ant-api03- and is shown once, so copy it before closing the dialog.

OpenAI. Sign in at platform.openai.com and create a secret key. It starts with sk-proj- and is also shown only once.

Neither. Ollama runs a model on your own machine. The last section of this class sets that up.

Whichever you use, the key goes in an environment variable:

You run this
export ANTHROPIC_API_KEY="sk-ant-api03-..."

# or, for OpenAI
export OPENAI_API_KEY="sk-proj-..."

Adding that to ~/.zshrc or ~/.bashrc makes it survive a new terminal.

On Windows: setting the key and checking it (click to expand)

export is not a command on Windows. In PowerShell, $env:ANTHROPIC_API_KEY = "sk-ant-api03-..." sets the variable for the current window only. setx writes it to your user account so later windows have it too:

You run this, in PowerShell
setx ANTHROPIC_API_KEY "sk-ant-api03-..."

The check that the variable reached this window:

You run this, in PowerShell
$env:ANTHROPIC_API_KEY.Substring(0,14) + "... (length " + $env:ANTHROPIC_API_KEY.Length + ")"

You cannot call a method on a null-valued expression is PowerShell saying the variable is not set here.

The check that the key is accepted needs three changes from the version below. Write curl.exe rather than curl, because Windows PowerShell treats curl as an alias for Invoke-WebRequest, which does not take these flags. Use NUL for the null device instead of /dev/null. Read the variable as $env:ANTHROPIC_API_KEY. Line continuations are backticks, and so is the newline in -w:

You run this, in PowerShell
curl.exe -s -o NUL -w "%{http_code}`n" https://api.anthropic.com/v1/models `
  -H "x-api-key: $env:ANTHROPIC_API_KEY" `
  -H "anthropic-version: 2023-06-01"

200 and 401 mean the same as below.

Open a new terminal after setting the variable

~/.zshrc and ~/.bashrc are read once, when a terminal starts, and setx writes a value that only new processes pick up. Either way, a window that is already open keeps the environment it started with, so setting the variable does not change anything in that window. An agent started in that window reports API key is invalid. while your key is perfectly good. That is the most common reason for the error, and the hardest to spot, because everything you check afterwards looks correct.

Check it in the same terminal we will run the agent from, before going any further:

You run this
echo "${ANTHROPIC_API_KEY:0:14}... (length ${#ANTHROPIC_API_KEY})"

sk-ant-api03-6... (length 108)

If nothing appears before the ..., this terminal cannot see the key.

A sensible length confirms the key reached this terminal, though it does not confirm the key still works. Only the API settles that. This call asks it for the list of models the key can use, and listing them does not run one, so nothing is billed:

You run this
curl -s -o /dev/null -w "%{http_code}\n" https://api.anthropic.com/v1/models \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01"

The three flags together say "print the status code and nothing else":

-s is silent. Without it curl prints a progress meter, which is noise for a request this small.
-o /dev/null writes the response body to /dev/null. That is a file the operating system provides which discards everything written to it, so this throws the body away. We do not need the list of models, only whether we were allowed to ask for it.
-w "%{http_code}\n" writes a line of our own once the request finishes. %{http_code} is a curl variable holding the HTTP status code, and \n ends the line so the next shell prompt starts cleanly.

anthropic-version is not a flag but a header the API requires, on every request and every endpoint. Leave it out and the reply is anthropic-version: header is required, even with a valid key. 2023-06-01 is the current value, and a version is a promise about the shape of requests and responses: within one, Anthropic may add optional fields but will not remove or rename what is already there.

This is the only place in the course we send it by hand. The Spring AI starter sends it on every call the agent makes.

200 means the key works. 401 means it was rejected, and the agent will fail the same way, reporting authentication_error with the message API key is invalid. For OpenAI the equivalent is curl -s -o /dev/null -w "%{http_code}\n" https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY". The two checks fit together like this:

The loop back to the echo check is the step people skip: after opening a new terminal, run the check again in that one.

Starting the agent from your IDE

On macOS, an IDE launched from the dock or Spotlight does not read ~/.zshrc, so ${ANTHROPIC_API_KEY} is empty there even when it works in your terminal. On Windows an IDE picks up the user environment when it starts, so a variable written with setx afterwards is invisible until the IDE is restarted.

Either run mvn -pl support-agent spring-boot:run from a terminal that passes the check above, or add the variable to the run configuration's environment in the IDE.

We use the environment rather than application.yaml because that file gets committed. A key written there enters the repository's history and stays there after the line is deleted, and the only real fix then is to revoke the key and issue a new one.

The Provider

Add one starter to support-agent/pom.xml:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-anthropic</artifactId>
</dependency>

Then the provider block in support-agent/src/main/resources/application.yaml:

spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        model: claude-sonnet-4-6
        max-tokens: 4096

${ANTHROPIC_API_KEY} reads the environment variable at startup, so the key itself does not appear in the file.

Property names changed in Spring AI 2.0

Older examples use spring.ai.anthropic.chat.options.model. Those keys still work in 2.0.0 but are deprecated for removal, so this course uses the flattened form, spring.ai.anthropic.chat.model.

For OpenAI instead

Swap the starter for spring-ai-starter-model-openai and the block for:

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        model: gpt-4o

Everything else in this class is the same with either provider. The agent code we write does not name one.

Running the model locally instead

This subsection is optional. If we are staying with Anthropic or OpenAI, skip ahead to The Agent; nothing before The Same Agent, on a Local Model needs Ollama.

Everything from here works against a model on your own machine, which does not need an account or an API key. Ollama runs it and exposes an HTTP API on port 11434:

You run this
ollama pull qwen3:8b

Rather than swapping one starter for the other every time we want to compare them, we can keep both on the classpath and choose between them with a property. Add the Ollama starter alongside the Anthropic one:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>

Then configure both providers, and name the one to use:

spring:
  ai:
    model:
      chat: anthropic          # or: ollama
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        model: claude-sonnet-4-6
        max-tokens: 4096
    ollama:
      chat:
        model: qwen3:8b
        temperature: 0.6
        top-p: 0.95
        top-k: 20

spring.ai.model.chat is what each provider's auto-configuration checks before it creates anything. Set it to anthropic and only AnthropicChatModel is built; set it to ollama and only OllamaChatModel is. The block for the provider that is not selected is ignored, so both can sit in the file permanently. With the OpenAI starter in place instead of the Anthropic one, the value that selects it is openai.

Two starters need this property

Both auto-configurations treat themselves as the default when the property is absent, so adding the second starter without setting spring.ai.model.chat breaks startup:

No qualifying bean of type 'org.springframework.ai.chat.model.ChatModel' available:
    expected single matching bean but found 2: anthropicChatModel,ollamaChatModel
APPLICATION FAILED TO START

Setting the property fixes it, because it makes one of the two conditions false.

The choice of model matters more here than for plain chat, because this class needs tool calling: the model has to pick one of four tools and produce well-formed arguments for it. Qwen 3, Llama 3.1 and later, and Mistral Nemo all support it. Many smaller models do not, and some that list it still get it wrong often enough to be frustrating.

The three settings under the model name control how the model picks each next token while it writes. They come from the Qwen3 model card, which recommends them for this model with thinking enabled, and qwen3:8b ships with them as its defaults, which ollama show qwen3:8b prints. Writing them in the file keeps them visible, next to the warning they come with:

Setting	Value	What it does
`temperature`	`0.6`	scales how much randomness goes into each choice, where `0.0` always takes the single most likely token, which is called greedy decoding
`top-p`	`0.95`	narrows each choice to the smallest set of candidate tokens whose probabilities add up to 95 percent
`top-k`	`20`	caps that set at twenty tokens

Do not set the temperature to 0.0 for this model

A temperature of 0.0 looks like the right choice for an agent: the least random setting, so tool choices are careful and runs repeat. We ran this class that way, and one of the questions later in the class stalled the agent completely. The Qwen3 model card warns about this directly: "DO NOT use greedy decoding, as it can lead to performance degradation and endless repetitions."

Qwen3 is a thinking model: it writes internal reasoning before each answer. With greedy decoding that reasoning can fall into a repetition loop that never produces a stop token. Ollama's server log (~/.ollama/logs/server.log on macOS, %LOCALAPPDATA%\Ollama\server.log on Windows) recorded ours:

slot print_timing: id  0 | task 1780 | eval time =  956288.38 ms / 40006 tokens
slot      release: id  0 | task 1780 | stop processing: n_tokens = 40931, truncated = 0
[GIN] 2026/08/09 - 16:18:23 | 200 |        15m56s |       127.0.0.1 | POST     "/api/chat"

The request generated 40,006 tokens over fifteen minutes and fifty-six seconds. It stopped only when the model's context window filled: the largest number of tokens a model can hold in one exchange, 40,960 for this one. Ollama also serves one request at a time per loaded model, so every question asked while this ran waited in a queue behind it.

The recommended values above avoid the loop, and When the Model Never Answers, at the end of this class, adds a time limit as well, so that a stall from any cause becomes an error instead of an endless wait.

Setting all of this up now means the switch is ready when we want it. The Same Agent, on a Local Model, at the end of this class, runs the agent against qwen3:8b and shows what it answered.

The Agent

Create support-agent/src/main/java/com/themcpguy/supportdesk/agent/service/SupportAgentService.java:

package com.themcpguy.supportdesk.agent.service;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
import org.springframework.stereotype.Service;

@Service
public class SupportAgentService {

    private final ChatClient chatClient;

    SupportAgentService(ChatClient.Builder builder, SyncMcpToolCallbackProvider mcpTools) {
        this.chatClient = builder
                .defaultSystem("""
                    You are a support agent for an online shop. You answer questions
                    about orders using the tools you have been given.

                    Guidelines:
                    - Use a tool to find out anything about an order. Never guess an
                      order ID, a status, a total or a delivery date.
                    - Order IDs look like ORD-10001 and customer IDs like CUST-42. If
                      the user gives you something that is not in that form, ask.
                    - Amounts are in euros. Quote them with two decimal places
                      and the euro sign.
                    - If a tool returns an error, tell the user what it said and what
                      they could try instead.
                    - Keep answers to a few sentences unless asked for detail.
                    """)
                .defaultTools(mcpTools)
                .defaultAdvisors(MessageChatMemoryAdvisor.builder(
                        MessageWindowChatMemory.builder().build()).build())
                .build();
    }

    public String chat(String conversationId, String userMessage) {
        return chatClient.prompt()
                .advisors(advisor -> advisor.param(ChatMemory.CONVERSATION_ID, conversationId))
                .user(userMessage)
                .call()
                .content();
    }
}

What Happens When Someone Asks a Question

None of the earlier classes' work is visible in the code above, so it is worth walking through one question in full. Someone asks "where is order ORD-10001?", and four things happen:

The model is asked twice, with one call to order-service in between.

1. Spring AI builds a request for the model. The request carries the system prompt, the conversation so far, the new question, and a description of every tool the agent has. The system prompt holds the standing instructions we set once and send with every question. Three parts of each tool description do the work here:

Part	Where it comes from	Written in
the name	the `name` on the `@McpTool` annotation in `order-service`	Class 2
the description	the `description` text on the same annotation	Class 3
the input schema	generated by Spring AI from the method's parameter types and names	Class 3, by writing the method signature

All three arrived from order-service when the client connected in Class 6, and for get_order they look like this:

The three parts, for get_order
{
  "name": "get_order",
  "description": "Look up a single order by its ID.\nReturns the status, the customer, the line items, the total and the shipment.\nOrder IDs look like ORD-10001.\n",
  "inputSchema": {
    "type": "object",
    "properties": {
      "orderId": { "type": "string", "description": "The order ID, for example ORD-10001" }
    },
    "required": [ "orderId" ]
  }
}

2. The model chooses a tool. The model cannot run a tool itself. It is a service somewhere else, and it cannot reach our database. So it answers with a name and some arguments:

name      : get_order
arguments : { "orderId": "ORD-10001" }

It chose that tool by reading the descriptions from step 1, which are all it has to go on. That is why Class 3 spent so long on how to write them.

3. Spring AI makes the call the model asked for. Each tool is backed by a ToolCallback, a small object that knows how to run one tool. For an MCP tool that callback sends tools/call to order-service, the same request we made with curl in Class 2. The order comes back as JSON.

4. The model is asked again, with the answer attached. The second request carries everything from the first, plus the tool call and the JSON that came back:

system    : the guidelines from defaultSystem
user      : where is order ORD-10001?
assistant : call get_order with orderId ORD-10001
tool      : {"orderId":"ORD-10001","status":"SHIPPED","customer":{...},"shipment":{"carrier":"DHL", ...}}

Only now does the model have the order in front of it, and it writes the sentence the user reads. The last line is the tool result, and whatever order-service returns becomes part of what the model reads next. That is safe while the server is ours. In Class 9 we connect the agent to a server we did not write, and Class 16 shows what a hostile tool result can do from there.

If the model decides it needs another tool after seeing the first result, the cycle repeats: Spring AI keeps going until the model replies without asking for a tool.

The tool descriptions travel with every request, so they are paid for every time. Class 10 comes back to that once the agent is connected to more servers.
The model does not see our Java code. All it has is the names, the descriptions and the schemas, so writing a good description is as much a part of the work as writing correct code.

The Three Lines

Three lines in the constructor shape the exchange above:

Builder call	Which step it shapes	What it puts there
`defaultTools(mcpTools)`	step 1, building the request	the name, description and schema of every discovered tool
`defaultSystem(...)`	step 2, the model's choice	the rules the model weighs alongside the tool descriptions
`defaultAdvisors(MessageChatMemoryAdvisor...)`	step 1, on every question after the first	the earlier questions and answers, so "it" and "that customer" resolve

defaultTools(mcpTools). SyncMcpToolCallbackProvider turns each tool discovered on each connection into a ToolCallback Spring AI can describe to the model and later invoke. It holds that list rather than a snapshot taken once at startup, so a re-established connection gives a fresh list. defaultTools also accepts plain ToolCallback instances and @Tool-annotated beans, so MCP tools and local ones can be mixed. The older defaultToolCallbacks(...) overloads are deprecated in 2.0 for removal in 3.0.

defaultSystem(...). The instruction not to guess is the important one: without it a model will state a delivery date it did not look up, because producing a plausible sentence is what it does. The instruction about ID formats saves a whole cycle: a model that knows the shape of an order ID asks the user for a valid one instead of calling get_order with a value that can only fail.

MessageChatMemoryAdvisor gives the agent a memory:

user      : Where is order ORD-10001?
assistant : Order ORD-10001 for Ana Ruiz is currently shipped, ...
user      : And when does it arrive?

An advisor in Spring AI sits around each ChatClient call and can change the request on its way out or the response on its way back. This one adds the earlier questions and answers. Without it the second question would arrive on its own, and the model would not know what "it" refers to. Every call to the model is a separate HTTP request that does not carry anything from the call before.

MessageWindowChatMemory decides how much of that conversation to keep: a fixed number of the most recent messages, twenty by default, changed with .maxMessages(n) on the builder. Everything sent is charged for, so without a limit a long conversation would make every request more expensive than the last.

chat takes a conversationId because the advisor stores one history per conversation and needs to know which one to send.

A Command Line to Try It

Before wiring the browser up, a console loop is the quickest way to talk to the agent. Create support-agent/src/main/java/com/themcpguy/supportdesk/agent/cli/SupportCli.java:

package com.themcpguy.supportdesk.agent.cli;

import java.util.Scanner;

import org.springframework.boot.CommandLineRunner;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;

import com.themcpguy.supportdesk.agent.service.SupportAgentService;

@Component
@Profile("cli")   // so it does not read standard input when serving the browser
public class SupportCli implements CommandLineRunner {

    private final SupportAgentService agent;

    SupportCli(SupportAgentService agent) {
        this.agent = agent;
    }

    @Override
    public void run(String... args) {
        System.out.println("Support desk ready. Ask about an order. Type 'quit' to exit.\n");

        try (Scanner scanner = new Scanner(System.in)) {
            while (scanner.hasNextLine()) {
                String input = scanner.nextLine().trim();

                if (input.isBlank()) {
                    continue;
                }
                if (input.equalsIgnoreCase("quit") || input.equalsIgnoreCase("exit")) {
                    break;
                }

                try {
                    System.out.println("Agent: " + agent.chat("cli", input));
                }
                catch (Exception e) {
                    System.err.println("Error: " + e.getMessage());
                }
                System.out.println();
            }
        }
    }
}

@Profile("cli") ties this bean to a Spring profile, a named group of beans and settings switched on by name at startup, so SupportCli is created only when we ask for it. CommandLineRunner is the other half: Spring Boot calls run once the application context has started, which is what drives the loop. The try/catch is deliberate, because without it the first failure ends the session.

Delete the hard-coded tool call at the end of McpInspector.run, the one that fetches ORD-10001 and prints the result. In Class 6 we used it to prove the connection could carry a real request. The model does that now, and left in place it fires on every startup and prints a raw TextContent block into the middle of the log. Keep the rest of McpInspector: the list of tools, resources and prompts is still useful at a glance.

Run It

Before starting anything, turn on two logging levels. One reports which tool Spring AI is about to run, the other reports the MCP request it sends to run it. With these on we can watch each decision as it happens, next to the answer it produced. Add them to support-agent/src/main/resources/application.yaml:

logging:
  level:
    root: WARN
    com.themcpguy: INFO
    org.springframework.ai.model.tool: DEBUG
    io.modelcontextprotocol.client.transport: DEBUG

The transport logger also reports connections, so expect lines about Server-Sent Events (SSE) streams between the ones that matter. We come back to reading this output once the questions below have produced some.

Now start order-service, then the agent with the cli profile:

You run this, in another terminal
mvn -pl support-agent spring-boot:run -Dspring-boot.run.profiles=cli

On Windows: quoting the profile flag (click to expand)

The mvn commands so far have worked unchanged in PowerShell. This one does not, because the -D property contains dots, and PowerShell splits the unquoted token at the first one. Maven then reads the pieces as extra goals and stops with Unknown lifecycle phase ".run.profiles=cli". Quoting the whole property keeps it in one piece:

You run this, in PowerShell
mvn -pl support-agent spring-boot:run "-Dspring-boot.run.profiles=cli"

The same quoting applies to every dotted -D property from here on.

order-service has to be up first. The MCP client opens its connection while the agent is starting, so an agent started on its own does not come up at all.

The agent exits with Client failed to initialize by explicit API call (click to expand)

order-service is not answering on http://localhost:8080/mcp. Spring AI builds the mcpSyncClients bean during startup, and building it means connecting. If the server is down, the bean cannot be created and the application context fails to start. The message names the symptom rather than the cause, which is further down the stack trace:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate
    [java.util.List]: Factory method 'mcpSyncClients' threw exception with message:
    Client failed to initialize by explicit API call
Caused by: java.net.ConnectException: Connection refused

Start order-service, wait for it to log Started OrderServiceApplication, and then start the agent.

Without -Dspring-boot.run.profiles=cli the agent starts and serves HTTP without reading standard input. That is what we want once the browser is driving it, which it can do from the end of this class, when we add the endpoint it sends each message to.

Your answers will not match these word for word

The model writes every answer fresh, so the wording, the ordering and the formatting change from run to run, and between models. The replies below are real, captured from one session against claude-sonnet-4-6, and they are here to show the shape of what comes back. Treat the facts in them as fixed, since those come from the database, and everything around the facts as one model's phrasing on one day.

Start with an order:

Where is order ORD-10001?

Order **ORD-10001** for Ana Ruiz is currently **shipped**. Here are the key details:

- **Carrier:** DHL
- **Tracking Number:** DHL-88213
- **Shipped On:** 2026-05-12
- **Estimated Delivery:** 2026-05-14

You can use the tracking number DHL-88213 on the DHL website to get the latest
location update. Let me know if you need anything else!

That took one tools/call, to get_order. Now a follow-up that leaves the order and the customer unnamed:

And what did they order?

Ana ordered the following items:

| Product          | Quantity | Unit Price |
|------------------|----------|------------|
| Wireless Keyboard| 1        | €89.99     |
| USB-C Hub        | 2        | €45.00     |

**Order Total: €179.99**

"They" resolves to Ana Ruiz because the memory advisor sent the previous exchange along with the question. The model understood who was meant, and then called get_order a second time rather than reading the line items out of the result it already had. It was free to do either, because the number of tool calls is the model's choice.

Now something that needs a different tool:

Has that customer ordered anything else?

Yes, Ana has 6 other orders in addition to ORD-10001! Here's a summary:

| Order     | Date       | Status    | Total     |
|-----------|------------|-----------|-----------|
| ORD-10198 | 2026-05-27 | SHIPPED   | €1,322.99 |
| ORD-10003 | 2026-05-01 | DELIVERED | €599.00   |
| ORD-10030 | 2026-04-03 | DELIVERED | €318.78   |
| ORD-10072 | 2026-03-10 | DELIVERED | €1,710.50 |
| ORD-10114 | 2026-02-14 | DELIVERED | €304.79   |
| ORD-10156 | 2026-01-21 | SHIPPED   | €596.23   |

She's a Gold tier customer with quite a history!

The customer ID was not typed at all. "That customer" was enough, because the earlier get_order result carried CUST-42, and the memory advisor put that result back in front of the model. The model pulled the ID out of it and called get_customer_orders with it.

And something that needs two tools in one question:

Move ORD-10002 to PROCESSING and tell me what status it was in before

Done! Here's a summary:

- **Order:** ORD-10002
- **Previous status:** PENDING
- **New status:** PROCESSING

The order for Marcus Adeyemi (Mechanical Switch Kit, €34.99) has been
successfully moved to PROCESSING.

That one made two calls, in the only order that could work: get_order to learn the status before changing it, then update_order_status to change it. Nothing in our code sequenced those. The model worked out that the answer needed the old value, and that reading it after the update would be too late.

That is a write to a real record, chosen by the model on its own. The MCP specification's security considerations for tools say a client should ask the user to confirm a sensitive operation, and OWASP's LLM06:2025 Excessive Agency asks for a person to approve high-impact actions before they are taken. In Class 12 we add that confirmation: the cancellation tool stops and asks a person before it acts.

Watching the Model Decide

So far the tool calls have been read backwards out of the answers: the agent mentioned DHL, so it must have called get_order. That inference is usually right, and the logging turned on at the start of this section shows the decisions themselves.

Scroll back to the last question, the one that moved ORD-10002. With the transport's connection chatter stripped out, these are the lines between the question and the answer:

DefaultToolCallingManager   : Executing tool call: get_order
HttpClientStreamableHttpTransport : Sending message JSONRPCRequest[jsonrpc=2.0,
    method=tools/call, id=cc889cd4-6,
    params=CallToolRequest[name=get_order, arguments={orderId=ORD-10002}]]

DefaultToolCallingManager   : Executing tool call: update_order_status
HttpClientStreamableHttpTransport : Sending message JSONRPCRequest[jsonrpc=2.0,
    method=tools/call, id=cc889cd4-7,
    params=CallToolRequest[name=update_order_status,
                           arguments={orderId=ORD-10002, newStatus=PROCESSING}]]

Each pair of lines shows one decision and the request it produced.

Executing tool call is Spring AI acting on something the model has already asked for: its reply was a request for get_order. The arguments are the model's too. Nothing in our code decided that orderId should be ORD-10002, only that a tool taking an orderId exists, and the model filled it in from the question by reading the schema.

The same pair appears again for update_order_status. The ordering matters: the question asked for the status the order was in before the change, so the status had to be read first and updated second. If the model had updated first, the call would have succeeded and the answer would have been wrong, because by then the old value would be gone. The whole exchange, including the round trip the log cannot show, runs like this:

The two arrows in the middle are the ones the log cannot show: the get_order result went back to the model, and only then did it ask for update_order_status. The last two arrows are the loop ending: the model is asked once more, and this time it answers without asking for another tool.

It is worth leaving these levels on while we experiment. When the agent does something surprising, this log says whether the model chose a different tool, sent different arguments, or never called a tool at all, and those need different fixes.

The model decides, every time

None of those questions names a tool. Whether a tool is called at all is the model's judgement, made from the descriptions order-service supplied in Class 3. A question that "should" call a tool sometimes does not, and when that happens the tool's description is the first place to look, before our own code.

When a Tool Fails

Ask for an order that does not exist:

Where is order ORD-99999?

Agent: There's no order with that ID. Could you check the number? Order IDs
look like ORD-10001.

Our code did not see an error. This is the default and it is usually what we want: Spring AI wraps a tool failure in a ToolExecutionException, and the default ToolExecutionExceptionProcessor turns it into a message for the model rather than letting it reach us. The model then recovers, which is why the error messages in Class 3 were written to say what would work instead.

That instruction in the system prompt, to tell the user what the tool said, is safe because those Class 3 messages were written for a reader. A tool that returns a raw exception message would put that text in front of the user, so change the instruction or sanitise the message first. Where a failure ends up depends on one property:

The property in the middle decides where a tool failure ends up. The pair standing on its own is the other case: connecting to a server, or listing its tools at startup, raises an exception in the ordinary way.

The wrapping covers more than exceptions thrown inside a tool: connection-level failures are treated the same way. A network error, or a server that died mid-conversation, also becomes a message the model reads.

To have those surface as exceptions instead, so a try/catch around the ChatClient call can handle them, set spring.ai.tools.throw-exception-on-error to true. For finer control, register a processor bean built with DefaultToolExecutionExceptionProcessor.builder().alwaysThrow(true).build(); the builder also offers rethrowExceptions(...) to rethrow only chosen types.

Class 10 looks at what a startup failure means when one of several servers is unavailable.

Streaming

Waiting for a complete answer before showing anything reads badly when the answer is long, and a tool-using agent is often slow enough for that to matter. ChatClient can hand back the answer in pieces as the model produces them. Add this method to SupportAgentService, alongside chat rather than in place of it:

public Flux<String> chatStream(String conversationId, String userMessage) {
    return chatClient.prompt()
            .advisors(advisor -> advisor.param(ChatMemory.CONVERSATION_ID, conversationId))
            .user(userMessage)
            .stream()
            .content();
}

Flux is reactor.core.publisher.Flux, and Reactor is already on the classpath because the MCP client depends on it, so support-agent/pom.xml stays as it is.

A Flux is a description of work, not the work itself. Nothing happens when chatStream returns. No request goes to the model, no tool is called, and no text exists yet. All of that starts when something subscribes to the Flux:

// Wrong: prints the recipe, not the answer
System.out.println("Agent: " + agent.chatStream("cli", input));

Agent: FluxFilter

Concatenating a Flux with a string calls toString() on it, and Reactor answers with the name of its internal operator class. The application succeeds, no request goes to the model, and nothing is billed. The line printed does not explain why.

To use it from the command line, subscribe and print each piece as it arrives:

System.out.print("Agent: ");
agent.chatStream("cli", input)
        .doOnNext(System.out::print)
        .blockLast();
System.out.println();

doOnNext runs for every chunk the model sends, and blockLast waits for the last one so the loop does not ask for the next question while the answer is still arriving. Blocking is right here because the console is idle in the meantime anyway. In the browser endpoint of the next section it would not be, which is the main reason that endpoint stays with the plain chat method.

The tool-use loop still runs underneath a stream. Tool calls happen between chunks, so the output pauses while a tool executes and picks up again when the model carries on.

The Endpoint the Browser Uses

Everything so far has gone through the command line, and that was deliberate. A terminal puts the question, the answer and the log in one place, while a browser hides all of that behind a chat bubble:

What you see	Command line	Browser chat panel
the question and the answer	yes	yes
which tool the model chose	yes, on the `DefaultToolCallingManager` line	only in the agent's own terminal
the arguments it sent	yes, on the `JSONRPCRequest` line	only in the agent's own terminal
the order things happened in	yes, all in one place	no
what someone using the support desk would see	no	yes

The browser is worth having as well. Class 1 mentioned the small React frontend that ships with the course project. Its chat panel sends each message to /api/chat, so the agent needs a handler on that path for the panel to work, and from then on we can use either.

The controller is a Spring MVC @RestController, and up to now the agent has been a plain application without a web server of its own, which is why spring-boot-starter was enough. Add the web starter to support-agent/pom.xml first:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

The annotations below come from org.springframework.web.bind.annotation, in the spring-web jar, and that jar is already on the classpath, because the MCP client starter brings Spring MVC with it. What the web starter adds is the embedded Tomcat server. Without it the application starts as a non-web application, and nothing listens on any port to serve /api/chat.

Adding the starter also means the agent now starts a web server, and order-service already holds port 8080. Give the agent one of its own in support-agent/src/main/resources/application.yaml:

server:
  port: 8081

Now create support-agent/src/main/java/com/themcpguy/supportdesk/agent/web/SupportController.java:

package com.themcpguy.supportdesk.agent.web;

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import com.themcpguy.supportdesk.agent.service.SupportAgentService;

@RestController
public class SupportController {

    private final SupportAgentService agent;

    SupportController(SupportAgentService agent) {
        this.agent = agent;
    }

    public record ChatRequest(String conversationId, String message) {}
    public record ChatReply(String reply) {}

    @PostMapping("/api/chat")
    public ChatReply chat(@RequestBody ChatRequest request) {
        return new ChatReply(agent.chat(request.conversationId(), request.message()));
    }
}

ChatRequest and ChatReply cover what the agent needs from the chat panel. Each message is a POST request carrying a conversationId, a message and an orderId, which is the order open in the panel or null, and the panel reads a reply back. The record declares only the first two for now, and the orderId is dropped on deserialisation, because Spring Boot configures Jackson to ignore unknown properties. In Class 8 we declare it and attach the order it names. The conversationId is the same one the memory advisor keys on, so each browser conversation keeps its own history. It arrives from the browser and nothing checks who sent it, so in an application with real users that id has to come from the signed-in user instead of from the request body.

Now start the frontend, in a third terminal alongside the two services:

You run this, in a third terminal
cd frontend
npm install
npm run dev

npm install is only needed the first time. The frontend starts on http://localhost:5173.

Restart the agent without -Dspring-boot.run.profiles=cli, so that it serves HTTP instead of waiting on the terminal, then open that page and put the same questions to it that we asked at the command line. Start with Where is order ORD-10001? and carry on through the follow-up about what they ordered. It is the same agent, the same tools and the same conversation memory, so the answers should say much the same as they did at the command line, even though the wording will differ. The terminal we started the agent in still prints each tool call, so we can follow those there while the conversation itself happens in the browser.

The frontend stays unchanged for the rest of the course. Three later classes light up pieces it already carries:

Frontend feature	Starts working in	What the console shows instead
the refund-email button on each order	Class 8	the same email, as printed lines
the progress bar during a long job	Class 11	the same progress, as printed lines
the confirmation dialog before a cancellation	Class 12	a question at the prompt

The Same Agent, on a Local Model

If you do not want to try the agent on a local model, skip ahead to What We Built.

Ollama has to be running for this, and it is a separate process from anything we have started so far. On macOS and Windows the desktop app runs it in the background; on Linux it is ollama serve. Two commands confirm it is up and that the model is there:

You run this
ollama list
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:11434/api/tags

ollama list should include qwen3:8b, and the second command should print 200. If it prints 000 instead, nothing is listening on port 11434 and Ollama is not running.

On Windows: the Ollama check and the run command (click to expand)

The check needs the same changes as the key check at the top of this class: curl.exe instead of curl, because Windows PowerShell treats curl as an alias for Invoke-WebRequest, and NUL for the null device:

You run this, in PowerShell
ollama list
curl.exe -s -o NUL -w "%{http_code}`n" http://localhost:11434/api/tags

The mvn command further down this section needs backticks for the line continuations and quotes around each -D property, because PowerShell splits an unquoted property at the dots, as in Run It:

You run this, in PowerShell
mvn -pl support-agent spring-boot:run `
  "-Dspring-boot.run.profiles=cli" `
  "-Dspring-boot.run.arguments=--spring.ai.model.chat=ollama"

Notice that starting Ollama does not involve choosing a model. The server holds whatever has been pulled and loads one when it is asked for, so the model is named on each request, not at startup. Sending a request without one is rejected:

{"error":"model is required"}

The name in our request comes from spring.ai.ollama.chat.model, which is why that property is qwen3:8b. Change it to any other pulled model and the agent asks for that one instead, without restarting Ollama itself. ollama ps shows what is loaded at any moment, and models are unloaded again after a few idle minutes, which is why the first question after a pause is slower than the ones that follow.

With that up and both starters in place, switching model is one property. Edit spring.ai.model.chat to ollama, or leave the file alone and override it on the command line, which is easier when comparing the two:

You run this
mvn -pl support-agent spring-boot:run \
  -Dspring-boot.run.profiles=cli \
  -Dspring-boot.run.arguments=--spring.ai.model.chat=ollama

Nothing else changes. SupportAgentService and SupportCli are untouched, because neither names a provider, and ChatClient.Builder is handed whichever ChatModel was built. The API key is not needed either: with ollama selected the Anthropic auto-configuration does not run, so nothing reads the ${ANTHROPIC_API_KEY} placeholder in the file. The agent runs without any key set.

Where is order ORD-10001?

Agent: Order **ORD-10001** is currently **SHIPPED** via **DHL** with tracking
number **DHL-88213**. It was shipped on **May 12, 2026**, and the estimated
delivery date is **May 14, 2026**. Let me know if you need further details!

It handles a single lookup well. The multi-step question is the one to try next:

Move ORD-10002 to PROCESSING and tell me what status it was in before

This is where a model this size struggles. Answering it needs two tools in the right order, and qwen3:8b sometimes calls update_order_status first:

DefaultToolCallingManager : Executing tool call: update_order_status
HttpClientStreamableHttpTransport : Sending message JSONRPCRequest[jsonrpc=2.0,
    method=tools/call, id=7c814d2d-6,
    params=CallToolRequest[name=update_order_status,
                           arguments={orderId=ORD-10002, newStatus=PROCESSING}]]

That is the first tool call in the log. The update goes through, and with it the old value, so the question can no longer be answered in full.

What comes back after that varies. The model may state a status it did not look up. It may also confirm the change without mentioning the previous status, which is the outcome we saw here: the order really did move to PROCESSING, and the second half of the question went unanswered. That one is easier to miss than a wrong value, because the reply reads as though the job is done.

The same question, on the two models, as the log recorded it:

Model	First tool call	Second tool call	What the answer could say
`claude-sonnet-4-6`	`get_order`, reading the status `PENDING`	`update_order_status`, setting `PROCESSING`	both halves: the old status and the new one
`qwen3:8b`, on the run captured here	`update_order_status`, setting `PROCESSING`	none	only that the change was made, because the old value was already gone

Run it a few times. The order is not fixed from run to run, and a smaller model plans less reliably than a larger one.

If both an API key and Ollama are set up, that question is worth putting to each of them and comparing the tool calls in the log. The tools, their descriptions and the system prompt are identical across the two runs, so anything that differs comes from the model's own planning. That comparison is the most direct way to judge whether a local model is good enough for a particular job. This course cannot answer that in general: it depends on the model, on how well the tool descriptions are written, and on how much the task forgives a wrong order of calls.

When the Model Never Answers

One outcome we have not covered yet: a question goes out and nothing comes back at all. We hit this while writing the class. The tool call had already run, so the order really was updated, and then the conversation stopped. The browser kept spinning, and no terminal printed anything. The warning in The Provider described one cause, a generation looping until it filled the context window. Ollama also serves one request at a time per loaded model, so the looping request does not even have to be ours: anything else using the same Ollama holds the queue, and our question waits behind it.

This is the path the question takes:

The last hop is the one that does not have a time limit. The one timeout our file does have, request-timeout under spring.ai.mcp.client, bounds the MCP calls to order-service and does not cover the call to the model. That call goes through Spring's RestClient. The HTTP clients Spring Boot auto-configures do not set a read timeout by default, so the thread carrying the question blocks on the socket read for as long as the server stays silent. A thread dump shows it directly: jps prints the JVM process IDs, jstack <pid> prints every thread's stack, and the stuck thread reads like this, trimmed to the frames that matter:

"http-nio-8081-exec-2" daemon prio=5 runnable
   java.lang.Thread.State: RUNNABLE
	at java.net.Socket$SocketInputStream.read(Socket.java:974)
	...
	at org.springframework.web.client.DefaultRestClient$DefaultRequestBodyUriSpec.exchangeInternal(DefaultRestClient.java:614)
	at org.springframework.ai.ollama.api.OllamaApi.chat(OllamaApi.java:119)
	...
	at org.springframework.ai.chat.client.advisor.ToolCallingAdvisor.adviseCall(ToolCallingAdvisor.java:150)
	...
	at com.themcpguy.supportdesk.agent.service.SupportAgentService.chat(SupportAgentService.java:39)
	at com.themcpguy.supportdesk.agent.web.SupportController.chat(SupportController.java:25)

Reading from the bottom up: the controller called the service, the service called the ChatClient, the tool-calling advisor sent the conversation to the model, and OllamaApi.chat is reading a socket that nothing is writing to. The thread stays exactly here until something limits the read.

Two properties put that limit in place. Add them to support-agent/src/main/resources/application.yaml:

support-agent/src/main/resources/application.yaml
spring:
  http:
    clients:
      read-timeout: 10s
  ai:
    retry:
      max-attempts: 1

spring.http.clients.read-timeout is Spring Boot's setting for the HTTP clients it auto-configures, which is where Spring AI's Ollama client comes from. With it set, ten seconds of silence on the socket ends the call with a ResourceAccessException whose message says what happened: I/O error on POST request for "http://localhost:11434/api/chat": Read timed out.

The second property matters because Spring AI retries a failed model call before letting the exception out. The default allows ten retries, with pauses that start at two seconds, grow five times longer each round, and are capped at three minutes by spring.ai.retry.backoff.max-interval. With the timeout alone, the agent would log a warning per attempt and keep retrying for roughly twenty-five minutes:

16:22:05.794  WARN o.s.a.r.a.SpringAiRetryAutoConfiguration : Retry error. Retry count:1
16:22:18.804  WARN o.s.a.r.a.SpringAiRetryAutoConfiguration : Retry error. Retry count:2

max-attempts counts the retries after the first attempt, so 1 allows one retry and the second failure is final. With read-timeout: 10s, that gives:

Setting	Attempts	Pauses between them	Roughly how long before the error arrives
the default, `max-attempts: 10`	the first attempt plus ten retries	2s, 10s, 50s, then 3 minutes each	about 25 minutes
`max-attempts: 1`, what this class sets	the first attempt plus one retry	2s	about 20 seconds
`max-attempts: 0`	the first attempt only	none	about 10 seconds, the read timeout

The HTTP client underneath RestClient does not add to those figures, because it does not resend a POST whose response timed out. Ten seconds and a single retry make a stall become an error quickly while we experiment, and are a development setup.

The CLI already handles it: the catch in SupportCli prints the message and the You: prompt returns. The browser needs one more change, because an exception that escapes the controller becomes a 500, and the chat panel shows any non-200 response as support-agent returned 500. support-agent is built in Class 7., which points at the wrong cause entirely. Catching the exception in the controller and answering with an ordinary reply puts the explanation in the chat itself:

support-agent/src/main/java/com/themcpguy/supportdesk/agent/web/SupportController.java
package com.themcpguy.supportdesk.agent.web;

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.ResourceAccessException;

import com.themcpguy.supportdesk.agent.service.SupportAgentService;

@RestController
public class SupportController {

    private final SupportAgentService agent;

    SupportController(SupportAgentService agent) {
        this.agent = agent;
    }

    public record ChatRequest(String conversationId, String message) {}
    public record ChatReply(String reply) {}

    @PostMapping("/api/chat")
    public ChatReply chat(@RequestBody ChatRequest request) {
        try {
            return new ChatReply(agent.chat(request.conversationId(), request.message()));
        } catch (ResourceAccessException e) {
            return new ChatReply("The model did not answer in time, so the request was stopped. Ask again in a moment.");
        }
    }
}

The panel prints whatever arrives in reply, so the sentence shows up as a normal message in the conversation, and whoever asked knows what happened.

To watch the handling work without waiting for a real stall, take the model away: quit Ollama, ask anything in the browser, and the sentence appears immediately. With Ollama stopped, the connection is refused outright, which fails faster than a silent socket but ends in the same ResourceAccessException and the same catch. Start Ollama again and the next question behaves normally.

Neither property reaches the Anthropic provider. That starter wraps the official Anthropic Java SDK, which brings its own HTTP client and its own retry loop. So spring.http.clients.read-timeout bounds only the clients Boot builds, which in this project means the Ollama call, and spring.ai.retry does not reach the Anthropic path either:

Property	The Ollama call	The Anthropic call	The MCP calls to `order-service`
`spring.http.clients.read-timeout`	yes, 10s as we set it	no	no
`spring.ai.retry.max-attempts`	yes, 1 as we set it	no	no
`spring.ai.anthropic.timeout`	no	yes, 60s by default	no
`spring.ai.anthropic.max-retries`	no	yes, 2 by default	no
`spring.ai.mcp.client.request-timeout`	no	no	yes, 30s in our file, 20s by default

The SDK retries transient failures on its own, twice by default, and spring.ai.anthropic.timeout bounds each request at 60 seconds unless we change it. A stalled Anthropic call therefore also ends in an error, after a longer wait. On the Ollama path, an answer that honestly takes longer than ten seconds trips the limit just like a stall; if that happens, raise the value. Any finite value keeps the agent from waiting forever.

What We Built

support-agent answers questions about orders in English, using tools it discovered at startup from a server it does not import. The Java added in this class is small: a service, a controller and a console loop. Most of the behaviour comes from the system prompt and the tool descriptions.

In Class 8 we use the other two client-side primitives, resources and prompts.

Sources

Pricing, Claude Docs: the $3 and $15 per million tokens for claude-sonnet-4-6.
Versions, Claude Docs: that anthropic-version is required on every request, that 2023-06-01 is the current value, and what a version does and does not promise.
Upgrade Notes, Spring AI Reference: that Spring AI 2.0.0 flattened the chat option properties and kept the .options.* keys as deprecated.
Chat Memory, Spring AI Reference: the sliding window of the most recent messages, twenty by default.
Tool Calling, Spring AI Reference: that tool failures are wrapped in a ToolExecutionException, and that spring.ai.tools.throw-exception-on-error defaults to false.
Anthropic Chat, Spring AI Reference: the flattened spring.ai.anthropic.chat.model property, the 60-second spring.ai.anthropic.timeout and the two spring.ai.anthropic.max-retries.
Migrating the Anthropic Module to the Official Java SDK, Spring AI Reference: that the Anthropic module stopped taking a Spring Retry RetryTemplate, which is why spring.ai.retry does not reach it.
OpenAI Chat, Spring AI Reference: the spring.ai.retry defaults behind the retry table, a max-attempts of 10, a first pause of two seconds, a multiplier of five and a three-minute cap.
Ollama Chat, Spring AI Reference: that spring.ai.model.chat selects the provider and each chat auto-configuration is enabled when the property is absent.
MCP Client Boot Starter, Spring AI Reference: that spring.ai.mcp.client.request-timeout defaults to 20s and bounds MCP requests, and that clients are initialized on creation.
RestClient, WebClient, and HTTP Service Clients, Spring Boot: that spring.http.clients.read-timeout is Spring Boot's global setting for the HTTP clients it auto-configures.
spring-ai-mcp 2.0.0 POM: that the MCP client starter's own dependency, spring-ai-mcp, brings in mcp-spring-webmvc.
mcp-spring-webmvc 2.0.0 POM: that it depends on spring-webmvc, which is what puts spring-web on the agent's classpath before the web starter is added.
spring-boot-starter-web 4.1.0 POM: that the web starter brings spring-boot-starter-tomcat, the embedded server.
MCP specification: Tools: what a tool definition carries, and the client security considerations, including prompting for user confirmation on sensitive operations.
LLM06:2025 Excessive Agency, OWASP Top 10 for LLM Applications: that an agent able to change records needs human approval for high-impact actions.
Qwen3-8B model card: the thinking-mode values, the greedy-decoding warning quoted in the class, and the 40,960 max_position_embeddings behind the context window figure.
Ollama FAQ: that OLLAMA_NUM_PARALLEL defaults to 1, that models unload after five idle minutes, and that Ollama binds port 11434.
Ollama API: that a request without a model is rejected with {"error":"model is required"}.
Ollama Troubleshooting: the server log paths on macOS and Windows.

Next: Class 8: Consuming Resources and Prompts. Reading a resource into a conversation, and running the refund-email prompt.

What We'll Cover​

Set the Key as an Environment Variable​

The Provider​

For OpenAI instead​

Running the model locally instead​

The Agent​

What Happens When Someone Asks a Question​

The Three Lines​

A Command Line to Try It​

Run It​

Watching the Model Decide​

When a Tool Fails​

Streaming​

The Endpoint the Browser Uses​

The Same Agent, on a Local Model​

When the Model Never Answers​

What We Built​

Further Reading​

Sources​