Module 4: ChatClient with MCP Tools

Duration: ~45 minutes | Level: Intermediate | Prerequisites: Module 3 complete, MCP connection working, tools listing correctly

What You'll Build

A command-line AI agent that you can converse with in plain English. The agent has access to a filesystem MCP server and uses it transparently when answering questions about files and directories.

You'll also understand:

How Spring AI's ChatClient API works in depth
Why the tool-use loop is automatic and what that means for your code
How conversation memory works and why it matters
How to control the model's behaviour with system prompts
How to stream responses for long outputs

The ChatClient API

ChatClient is Spring AI's fluent API for interacting with language models. Think of it as a WebClient or RestTemplate, but for AI conversations rather than HTTP requests.

The basic pattern:

String response = chatClient.prompt()
    .user("Your question here")
    .call()
    .content();

Every method in the chain configures one aspect of the request:

.prompt(): starts a new prompt builder
.system("..."): sets a system message (the model's persona and instructions)
.user("..."): sets the user message (the question or task)
.tools(...): specifies which tools the model can use
.advisors(...): adds advisors (middleware) for cross-cutting concerns like memory
.call(): sends the request and waits for the response
.content(): extracts the text content from the response

What is an Advisor?

An advisor is Spring AI's middleware pattern: a small component that wraps each prompt/response cycle and can inspect, modify, or react to what flows through. The model never sees advisors directly; they sit between your code and the ChatClient call.

Common things advisors do:

Conversation memory (this module): pull previous messages from storage before the call, save the new exchange after the response.
RAG (retrieval-augmented generation): look up relevant documents from a vector store and inject them as context.
Token budgeting: trim or summarise the conversation if it would exceed the model's context window.
Logging / auditing: record every prompt and response for compliance.
Safety filtering: scan responses for sensitive content before returning to the caller.

You add advisors with .defaultAdvisors(...) (apply to every request) or .advisors(...) (per-request). The order matters: advisors run as a chain. Spring AI ships several built-in advisors; you'll meet MessageChatMemoryAdvisor first.

The Tool-Use Loop, Explained

When you add tools to a ChatClient request, the conversation may go through multiple rounds before you get a final answer:

Your application code makes one call: chatClient.prompt().user("...").call().content(). Spring AI handles the entire sequence shown above, including calling back to the MCP server with the tool arguments the model requested, and then calling the model again with the result.

This loop can run multiple times if the model needs to call several tools to answer a question. Spring AI handles all iterations automatically. The .call() method doesn't return until the model produces a final text response (not a tool request).

Why this matters: You write business logic, not protocol plumbing. The model decides which tools to call and in what order. Your code just provides the tools and asks the question.

Build the Agent

Create the application class and a conversational agent service:

package com.themcpguy.springai.agent;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class AgentApplication {
    public static void main(String[] args) {
        SpringApplication.run(AgentApplication.class, args);
    }
}

package com.themcpguy.springai.agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
import org.springframework.stereotype.Service;

@Service
public class FilesystemAgentService {

    private final ChatClient chatClient;

    FilesystemAgentService(ChatClient.Builder builder,
                           SyncMcpToolCallbackProvider mcpTools) {
        this.chatClient = builder
            .defaultSystem("""
                You are a helpful filesystem assistant. You have access to tools that let you
                read, write, and explore the filesystem at /tmp.

                When the user asks about files or directories, use the available tools to
                get accurate information rather than guessing. Be concise and precise.
                If a file is large, summarise rather than printing everything.
                """)
            .defaultTools(mcpTools)
            .defaultAdvisors(MessageChatMemoryAdvisor.builder(
                MessageWindowChatMemory.builder().build()).build())
            .build();
    }

    public String chat(String userMessage) {
        return chatClient.prompt()
            .user(userMessage)
            .call()
            .content();
    }
}

Three important things happen in the ChatClient.Builder configuration:

.defaultSystem(...): sets the system prompt. This tells the model who it is and how to behave. "Default" means it applies to every request from this ChatClient instance unless explicitly overridden.

.defaultTools(mcpTools): registers all MCP tools from all connected servers. defaultTools(Object...) accepts a heterogeneous mix: a ToolCallback, a ToolCallbackProvider (this SyncMcpToolCallbackProvider, whose callbacks are resolved lazily at request time), arrays or collections of either, or plain @Tool-annotated POJOs. The model sees every resulting tool and can invoke any of them. It replaces the older defaultToolCallbacks(...) overloads, which are deprecated in Spring AI 2.0 (@Deprecated(since = "2.0.0", forRemoval = true)) and scheduled for removal in 3.0.

.defaultAdvisors(MessageChatMemoryAdvisor.builder(MessageWindowChatMemory.builder().build()).build()): adds conversation memory. Without this, every .call() is a fresh conversation; the model has no memory of previous messages. With it, the conversation history is automatically included in each request. MessageWindowChatMemory is the in-memory ChatMemory implementation shipped with Spring AI 2.0.x; it keeps a sliding window of recent messages.

Add a CLI Interface

package com.themcpguy.springai.agent;

import org.springframework.boot.CommandLineRunner;
import org.springframework.stereotype.Component;

import java.util.Scanner;

@Component
public class AgentCli implements CommandLineRunner {

    private final FilesystemAgentService agent;

    AgentCli(FilesystemAgentService agent) {
        this.agent = agent;
    }

    @Override
    public void run(String... args) {
        System.out.println("Filesystem Agent ready. Type your question (or 'quit' to exit):");
        System.out.println();

        try (Scanner scanner = new Scanner(System.in)) {
            while (scanner.hasNextLine()) {
                String input = scanner.nextLine().trim();

                if (input.isBlank()) continue;
                if (input.equalsIgnoreCase("quit")) break;

                System.out.println("Agent: " + agent.chat(input));
                System.out.println();
            }
        }

        System.out.println("Goodbye.");
    }
}

Run it and try these prompts in sequence to see conversation memory in action:

You: What's in /tmp?
Agent: [lists the contents of /tmp]

You: How big is the largest file there?
Agent: [uses get_file_info on the file it found previously, it remembers the context]

You: Create a file called hello.txt with the text "Hello from Spring AI"
Agent: [uses write_file]

You: Read it back to me
Agent: [uses read_text_file to read hello.txt]

You: quit

How Conversation Memory Works

MessageWindowChatMemory is a simple in-memory store that keeps a sliding window of the recent message history for a conversation. MessageChatMemoryAdvisor is an advisor, Spring AI middleware that wraps each request/response cycle.

On each request, MessageChatMemoryAdvisor:

Retrieves stored messages for the current conversation
Prepends them to the current request's message list
Sends the full history to the model
Stores the new user message and model response in memory

This is why the agent remembers "the largest file", that information was in a previous response, which is included in subsequent requests.

Conversation IDs

MessageWindowChatMemory supports multiple concurrent conversations via a conversation ID. The default ID is used when you don't specify one, making this suitable for a single-user CLI. For a web application serving multiple users:

// Per-user conversation management
public String chat(String userId, String message) {
    return chatClient.prompt()
        .user(message)
        .advisors(advisor -> advisor.param(
            ChatMemory.CONVERSATION_ID, userId))
        .call()
        .content();
}

Each user gets their own independent conversation history.

Memory and Token Limits

Every AI model has a context window, a limit on how many tokens (roughly, words) can appear in a single request. Include too much history and you'll hit the limit.

For production applications, use a token-budget chat memory advisor (the exact class name has shifted across Spring AI versions; consult the current docs) in combination with memory to automatically trim older messages when the history grows too large. Or implement a custom ChatMemory that stores conversations in a database and retrieves only the most recent N messages.

System Prompts: Shaping Model Behaviour

The system prompt is your contract with the model. It defines:

Scope: what the agent is and isn't allowed to help with
Tone: formal, casual, technical, concise
Tool usage guidance: when to use tools vs when to rely on knowledge
Output format: structured data, prose, code blocks

A weak system prompt produces inconsistent behaviour. A strong one gives you predictability.

For the filesystem agent, consider this more detailed version:

.defaultSystem("""
    You are a filesystem assistant with read and write access to /tmp.
    
    CAPABILITIES:
    - Read file contents with read_text_file
    - List directory contents with list_directory
    - Create files with write_file
    - Search by name with search_files
    - Get file metadata with get_file_info
    
    BEHAVIOUR:
    - Always use tools for current information, do not guess file contents or sizes
    - When listing directories, use list_directory rather than directory_tree unless the user explicitly wants a tree view
    - Confirm before writing or modifying files unless the user's intent is unambiguous
    - For large files (> 10KB), summarise content rather than printing everything
    - If a path is outside /tmp, explain that you only have access to /tmp
    """)

The more specific your instructions, the more predictable the model's behaviour. This is especially important when tools have side effects (writing, deleting), explicit confirmation instructions prevent the model from acting without user intent.

Streaming Responses

For long responses, reading large files, generating code, summarising documents, waiting for the complete response before displaying anything creates a poor user experience. Spring AI supports streaming:

import reactor.core.publisher.Flux;

public Flux<String> chatStream(String userMessage) {
    return chatClient.prompt()
        .user(userMessage)
        .stream()
        .content();
}

For the CLI:

System.out.print("Agent: ");
agent.chatStream(input)
    .doOnNext(System.out::print)
    .doOnComplete(() -> System.out.println())
    .blockLast(); // block only at the CLI boundary

.stream() returns a reactive Flux<String> where each element is a token (or small batch of tokens) as it arrives from the model. For web applications using Spring WebFlux, you can return this Flux directly to the HTTP response for server-sent events, making token streaming to the browser completely straightforward.

Note: tool calls are not streamed, Spring AI accumulates the complete tool request, executes the tool, and then resumes streaming. The streaming visible to users is the text portions of the response.

Handling Tool Errors

What happens when a tool fails? For example, list_directory on a path that doesn't exist?

The MCP server returns an error in the tool result. Spring AI passes this error back to the model as a tool result with error information. The model then typically explains to the user what went wrong, without throwing an exception in your application code.

Test it:

You: What's in /nonexistent?
Agent: The directory /nonexistent doesn't appear to exist. I encountered an error when trying to list its contents. Would you like me to check a different path?

This is MCP error handling working as designed, the model receives the error context and can reason about it. Be careful with a common misconception here: by default, connection-level failures during a tool call (network errors, authentication failures, a server that died mid-conversation) also end up with the model, not with your code. Spring AI wraps the underlying SDK exception (such as McpError) in a ToolExecutionException, but the default ToolExecutionExceptionProcessor converts that exception into an error message and sends it back to the model as the tool result. If you want such failures to propagate to your application code instead, so a try/catch around the ChatClient call can handle them, set spring.ai.tools.throw-exception-on-error=true, or register a ToolExecutionExceptionProcessor bean built with DefaultToolExecutionExceptionProcessor.builder().alwaysThrow(true).build() (the builder also offers rethrowExceptions(...) to rethrow only specific exception types). Failures that happen outside the tool loop, such as connecting to the server or listing tools at startup, do surface as exceptions in your application.

The Complete application.properties

# AI Provider
spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
spring.ai.anthropic.chat.model=claude-sonnet-4-6
spring.ai.anthropic.chat.max-tokens=4096

# MCP, filesystem server
spring.ai.mcp.client.toolcallback.enabled=true
spring.ai.mcp.client.stdio.connections.filesystem.command=npx
spring.ai.mcp.client.stdio.connections.filesystem.args=-y,@modelcontextprotocol/server-filesystem,/tmp

# Suppress Spring Boot banner for cleaner CLI output
spring.main.banner-mode=off

# Show only your own logs in the CLI
logging.level.root=WARN
logging.level.com.themcpguy=INFO

Next: Module 5: Multi-Server Agents. Connect to two MCP servers simultaneously and build an agent with capabilities from both.

What You'll Build​

The ChatClient API​

What is an Advisor?​

The Tool-Use Loop, Explained​

Build the Agent​

Add a CLI Interface​

How Conversation Memory Works​

Conversation IDs​

Memory and Token Limits​

System Prompts: Shaping Model Behaviour​

Streaming Responses​

Handling Tool Errors​

The Complete application.properties​