Class 7: Handing the Tools to a Model
Duration: ~35 minutes | Level: Intermediate | Prerequisites: Class 6: Connecting a Client, and either an API key from Anthropic or OpenAI, or Ollama running locally. This is the first class that needs a model.
What We'll Cover
- Configuring a provider, and keeping the key out of the repository
- Running a model on your own machine instead, with no account
ChatClient, the discovered tools, a system prompt and conversation memory- What happens when a tool fails, and why our code does not see it by default
- Streaming, and the endpoint the browser talks to
The Key Goes in the Environment
export ANTHROPIC_API_KEY="sk-ant-api03-..."
# or, for OpenAI
export OPENAI_API_KEY="sk-proj-..."
Adding that to ~/.zshrc or ~/.bashrc makes it survive a new terminal.
We use the environment rather than application.yaml because the YAML file gets committed. A key written there enters the repository's history with the first commit that contains it, and it stays in that history after the line is deleted. At that point the only real fix is to revoke the key and issue a new one.
The Provider
Add one starter to support-agent/pom.xml:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-anthropic</artifactId>
</dependency>
Then the provider block in support-agent/src/main/resources/application.yaml:
spring:
ai:
anthropic:
api-key: ${ANTHROPIC_API_KEY}
chat:
model: claude-sonnet-4-6
max-tokens: 4096
${ANTHROPIC_API_KEY} reads the environment variable at startup, so the key itself is never in the file.
For OpenAI, swap the starter for spring-ai-starter-model-openai and the block for:
spring:
ai:
openai:
api-key: ${OPENAI_API_KEY}
chat:
model: gpt-4o
Older examples use spring.ai.anthropic.chat.options.model. Those keys still work in 2.0.0 but are deprecated for removal, so this course uses the flattened form, spring.ai.anthropic.chat.model.
Running the model locally instead
Everything from here works against a model on your own machine, with no account and no key. Ollama runs it and exposes an HTTP API on port 11434:
ollama pull qwen3:8b
Then use spring-ai-starter-model-ollama and:
spring:
ai:
ollama:
chat:
model: qwen3:8b
options:
temperature: 0.0
The choice of model matters more here than for plain chat, because this class needs tool calling: the model has to pick one of four tools and produce well-formed arguments for it. Qwen 3, Llama 3.1 and later, and Mistral Nemo all support it. Many smaller models do not, and some that list it still get it wrong often enough to be frustrating.
temperature: 0.0 is deliberate. A smaller model choosing between similar tools does better with the least random setting, and it makes behaviour repeatable while working through the course.
There is a section at the end of this class showing what qwen3:8b actually answered.
The Agent
Create support-agent/src/main/java/com/themcpguy/supportdesk/agent/SupportAgentService.java:
package com.themcpguy.supportdesk.agent;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
import org.springframework.stereotype.Service;
@Service
public class SupportAgentService {
private final ChatClient chatClient;
SupportAgentService(ChatClient.Builder builder, SyncMcpToolCallbackProvider mcpTools) {
this.chatClient = builder
.defaultSystem("""
You are a support agent for an online shop. You answer questions
about orders using the tools you have been given.
Guidelines:
- Use a tool to find out anything about an order. Never guess an
order ID, a status, a total or a delivery date.
- Order IDs look like ORD-10001 and customer IDs like CUST-42. If
the user gives you something that is not in that form, ask.
- Quote amounts with two decimal places and the currency.
- If a tool returns an error, tell the user what it said and what
they could try instead.
- Keep answers to a few sentences unless asked for detail.
""")
.defaultTools(mcpTools)
.defaultAdvisors(MessageChatMemoryAdvisor.builder(
MessageWindowChatMemory.builder().build()).build())
.build();
}
public String chat(String conversationId, String userMessage) {
return chatClient.prompt()
.advisors(advisor -> advisor.param(ChatMemory.CONVERSATION_ID, conversationId))
.user(userMessage)
.call()
.content();
}
}
Three lines in the builder do the work.
defaultTools(mcpTools) hands over everything discovered in Class 6. SyncMcpToolCallbackProvider is the bridge: it holds the tools from every configured connection and resolves them when a request is made, so a server that reconnects does not need the agent restarted. defaultTools accepts ToolCallback instances, providers like this one, and @Tool-annotated beans. The older defaultToolCallbacks(...) overloads are deprecated in 2.0 for removal in 3.0.
defaultSystem(...) is where the agent's behaviour is set. The instruction not to guess is the important one: without it a model will happily state a delivery date it has not looked up. The instruction about ID formats saves a round trip, because the model asks rather than calling a tool with something that cannot work.
MessageChatMemoryAdvisor replays earlier turns, so "and when does it arrive?" has something to refer to. MessageWindowChatMemory keeps a fixed number of recent messages rather than the whole history, which bounds what each request costs.
chat takes a conversationId because the memory advisor needs to know which history to replay. Every request needs one.
A Command Line to Try It
package com.themcpguy.supportdesk.agent;
import java.util.Scanner;
import org.springframework.boot.CommandLineRunner;
import org.springframework.stereotype.Component;
@Component
@Profile("cli") // so it does not read standard input when serving the browser
public class SupportCli implements CommandLineRunner {
private final SupportAgentService agent;
SupportCli(SupportAgentService agent) {
this.agent = agent;
}
@Override
public void run(String... args) {
System.out.println("Support desk ready. Ask about an order. Type 'quit' to exit.\n");
try (Scanner scanner = new Scanner(System.in)) {
while (scanner.hasNextLine()) {
String input = scanner.nextLine().trim();
if (input.isBlank()) {
continue;
}
if (input.equalsIgnoreCase("quit") || input.equalsIgnoreCase("exit")) {
break;
}
try {
System.out.println("Agent: " + agent.chat("cli", input));
}
catch (Exception e) {
System.err.println("Error: " + e.getMessage());
}
System.out.println();
}
}
}
}
The try/catch is deliberate. Without it the first failure ends the session, which is tiresome while experimenting.
Run It
Start order-service, then the agent with the cli profile:
mvn -pl support-agent spring-boot:run -Dspring-boot.run.profiles=cli
Without -Dspring-boot.run.profiles=cli the agent starts and serves HTTP without reading standard input, which is what we want from Class 12 onwards when the browser is driving it.
Where is order ORD-10001?
The model calls get_order, reads the shipment, and answers in prose. Then a follow-up that names nothing:
And what did they order?
That resolves because the previous exchange is replayed alongside it. The model already has the order and answers without calling a tool again.
Something that needs a different tool:
Has CUST-42 ordered anything else?
And something that needs two:
Move ORD-10002 to PROCESSING and tell me what status it was in before
None of those questions names a tool. Whether a tool is called at all is the model's judgement, made from the descriptions order-service supplied in Class 3. Ask something answerable without a tool and no MCP request is made. That means a question that "should" call a tool sometimes does not, and when that happens the tool's description is the first place to look, before our own code.
When a Tool Fails
Ask for an order that does not exist:
Where is order ORD-99999?
Agent: There's no order with that ID. Could you check the number? Order IDs
look like ORD-10001.
Our code never saw an error. This is the default and it is usually what we want: Spring AI wraps a tool failure in a ToolExecutionException, and the default ToolExecutionExceptionProcessor turns it into a message for the model rather than letting it reach us. The model then recovers, which is why the error messages in Class 3 were written to say what would work instead.
It goes further than expected. Connection-level failures are treated the same way. A network error, or a server that died mid-conversation, also becomes a message the model reads.
To have those surface as exceptions instead, so a try/catch around the ChatClient call can handle them, set spring.ai.tools.throw-exception-on-error to true. For finer control, register a processor bean built with DefaultToolExecutionExceptionProcessor.builder().alwaysThrow(true).build(); the builder also offers rethrowExceptions(...) to rethrow only chosen types.
Failures outside the tool loop behave differently. Connecting to a server, or listing its tools at startup, raise exceptions in the ordinary way. Class 10 looks at what that means when one of several servers is unavailable.
Streaming
Waiting for a complete answer before showing anything reads badly when the answer is long. ChatClient can stream instead:
public Flux<String> chatStream(String conversationId, String userMessage) {
return chatClient.prompt()
.advisors(advisor -> advisor.param(ChatMemory.CONVERSATION_ID, conversationId))
.user(userMessage)
.stream()
.content();
}
The tool-use loop still runs underneath. Tool calls happen between chunks, so the stream pauses while a tool executes and resumes when the model continues.
The Endpoint the Browser Uses
The frontend from Class 1 has a chat panel with nothing behind it. Give it something:
package com.themcpguy.supportdesk.agent;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class SupportController {
private final SupportAgentService agent;
SupportController(SupportAgentService agent) {
this.agent = agent;
}
public record ChatRequest(String conversationId, String message) {}
public record ChatReply(String reply) {}
@PostMapping("/api/chat")
public ChatReply chat(@RequestBody ChatRequest request) {
return new ChatReply(agent.chat(request.conversationId(), request.message()));
}
}
That needs spring-boot-starter-web in support-agent/pom.xml, and a port of its own so it does not collide with order-service:
server:
port: 8081
Start the frontend as in Class 1 and the chat panel answers. We do not change the frontend in this course; it is here so the later classes are easier to follow, and every one of them also works from the command line.
The Same Agent, on a Local Model
Swap the starter for spring-ai-starter-model-ollama and the provider block for the Ollama one above. Nothing else changes: SupportAgentService and SupportCli are untouched, because neither names a provider.
Where is order ORD-10001?
Agent: Order ORD-10001 has been shipped. It went out with DHL, tracking
DHL-88213, and the estimated delivery is 14 May 2026. The total was 179.99.
It handles a single lookup well. Where a model this size struggles is the multi-step question: asked to change a status and report the previous one, it sometimes calls update_order_status first and then has no way to know what the status used to be. A larger model calls get_order first. This is worth seeing rather than reading about, because it is the kind of difference that decides whether a local model is usable for a given job.
What We Built
support-agent answers questions about orders in English, using tools it discovered at startup from a server it does not import. The system prompt and the tool descriptions decide its behaviour; the Java in this class is a service, a controller and a loop.
Class 8 uses the other two primitives from the client side.
Next: Class 8: Consuming Resources and Prompts. Reading a resource into a conversation, and running the refund-email prompt.