Skip to main content

Module 8: Production Configuration

Duration: ~45 minutes | Level: Intermediate → Advanced | Prerequisites: All previous modules. Some familiarity with Spring Boot Actuator is helpful but not required.

What You'll Learn​

Building an AI application that works on your laptop is Module 4. Building one that runs reliably in production, with correct secrets management, meaningful health checks, observable metrics, and graceful degradation, is this module.

The techniques here are not AI-specific. They're standard production Spring Boot practices applied to the specific concerns of LLM and MCP integrations.

Externalising Configuration​

Never put API keys, server URLs, or environment-specific settings in application.properties. That file is committed to version control. Your API key in source control is a security incident.

Spring Boot's externalized configuration stack handles this correctly:

Lowest priority → application.properties (committed to source)
                → application-{profile}.properties
                → Environment variables
                → System properties (-Dkey=value)
                → Command-line arguments (--key=value)  ← Highest priority

Your application.properties contains only safe defaults:

# application.properties, committed to source control, no secrets
spring.ai.anthropic.chat.model=claude-sonnet-4-6
spring.ai.anthropic.chat.max-tokens=4096
spring.ai.mcp.client.toolcallback.enabled=true
spring.ai.mcp.server.name=order-management
spring.ai.mcp.server.version=1.0.0

Secrets come from environment variables:

# application.properties references env vars, never contains values
spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
spring.ai.mcp.client.streamable-http.connections.remote-tools.url=${MCP_SERVER_URL:http://localhost:8080}

The syntax ${MCP_SERVER_URL:http://localhost:8080} sets a default (http://localhost:8080) when the env var isn't set, useful for local development.

In Production Environments​

Docker/Kubernetes: Inject via env in pod specs or docker run -e. Never bake secrets into images.

AWS: Use Systems Manager Parameter Store or Secrets Manager with the Spring Cloud AWS integration.

HashiCorp Vault: Use spring-cloud-vault-config. Vault injects secrets at startup and can rotate them.

Kubernetes Secrets: Mount as environment variables or files. Use sealed secrets for encrypted storage in Git.

Profile-Specific Configuration​

Use Spring profiles to separate development and production settings:

# application-dev.properties, for local development
spring.ai.mcp.client.stdio.connections.filesystem.command=npx
spring.ai.mcp.client.stdio.connections.filesystem.args=-y,@modelcontextprotocol/server-filesystem,/tmp
logging.level.org.springframework.ai=DEBUG
# application-prod.properties, for production
spring.ai.mcp.client.streamable-http.connections.remote-filesystem.url=${FILESYSTEM_MCP_URL}
logging.level.org.springframework.ai=WARN

Activate a profile with SPRING_PROFILES_ACTIVE=prod environment variable or --spring.profiles.active=prod command-line argument.

Spring Boot Actuator Integration​

Add Actuator for operational endpoints:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Configure what's exposed:

# Expose health and info endpoints over HTTP
management.endpoints.web.exposure.include=health,info,metrics,prometheus
management.endpoint.health.show-details=when-authorized

# Actuator on a separate port (keeps it off the public internet)
management.server.port=8081

Health Indicators​

Spring AI 2.0.x ships no built-in health indicator for MCP connections, so you write a small custom one. Each configured MCP client exposes a ping() operation, which is exactly what a health check needs. Note that in Spring Boot 4 the health API moved to the org.springframework.boot.health.contributor package, out of the old actuate package:

import java.util.List;
import io.modelcontextprotocol.client.McpSyncClient;
import org.springframework.boot.health.contributor.Health;
import org.springframework.boot.health.contributor.HealthIndicator;
import org.springframework.stereotype.Component;

@Component("mcpClientHealthIndicator")
public class McpClientHealthIndicator implements HealthIndicator {

    private final List<McpSyncClient> clients;

    public McpClientHealthIndicator(List<McpSyncClient> clients) {
        this.clients = clients;
    }

    @Override
    public Health health() {
        Health.Builder builder = Health.up();
        for (McpSyncClient client : clients) {
            String name = client.getClientInfo().name();
            try {
                client.ping();
                builder.withDetail(name, "UP");
            } catch (Exception ex) {
                builder.down().withDetail(name, "DOWN: " + ex.getMessage());
            }
        }
        return builder.build();
    }
}

Spring Boot derives the contributor name by stripping the HealthIndicator suffix from the bean name, so this bean surfaces as mcpClient in /actuator/health.

Check health:

curl http://localhost:8081/actuator/health | python -m json.tool

Expected when healthy:

{
  "status": "UP",
  "components": {
    "mcpClient": {
      "status": "UP",
      "details": {
        "filesystem": "UP",
        "order-server": "UP"
      }
    },
    "diskSpace": { "status": "UP" },
    "ping": { "status": "UP" }
  }
}

If an MCP server goes down, the custom indicator reports DOWN when any ping fails, and the overall status becomes DOWN (or DEGRADED if you configure separate readiness and liveness probes).

Kubernetes Probes​

Configure separate liveness and readiness probes, a common production pattern:

# Liveness: is the process healthy? Restart if not.
management.endpoint.health.group.liveness.include=ping,diskSpace

# Readiness: is the app ready to serve requests? Remove from load balancer if not.
management.endpoint.health.group.readiness.include=mcpClient

In your Kubernetes deployment:

livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8081
  initialDelaySeconds: 30
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /actuator/health/readiness
    port: 8081
  initialDelaySeconds: 10
  periodSeconds: 5

If an MCP connection drops, the pod is removed from the load balancer (readiness fails) but not restarted (liveness still passes). This is correct behaviour, the application is healthy but temporarily degraded.

Micrometer Metrics​

Spring AI integrates with Micrometer for AI-specific metrics. Add the Micrometer Registry for your metrics backend:

<!-- Prometheus (most common in Kubernetes) -->
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Spring AI automatically records:

Observation / MetricDescription
spring.ai.chat.clientChatClient call()/stream() duration and status
gen_ai.client.operationChatModel (and EmbeddingModel) call duration and status
gen_ai.client.token.usageInput and output token counts (recorded by the ChatModel)
spring.ai.toolTool call duration and status during chat model interactions

Spring AI does not emit an MCP-specific metric. MCP client health is surfaced through the custom health indicator shown earlier, not through a dedicated Micrometer metric.

Custom Metrics for Your Tools​

Instrument your tool methods directly when you need business-level metrics:

package com.themcpguy.orders;

import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Timer;
import org.springframework.ai.mcp.annotation.McpTool;
import org.springframework.stereotype.Component;

@Component
public class OrderMcpTools {

    private final OrderService orderService;
    private final Counter orderLookups;
    private final Counter orderUpdates;
    private final Timer orderLookupTimer;

    OrderMcpTools(OrderService orderService, MeterRegistry registry) {
        this.orderService = orderService;
        this.orderLookups = Counter.builder("mcp.tool.order.lookups")
            .description("Number of order lookup tool calls")
            .register(registry);
        this.orderUpdates = Counter.builder("mcp.tool.order.updates")
            .description("Number of order status update tool calls")
            .register(registry);
        this.orderLookupTimer = Timer.builder("mcp.tool.order.lookup.duration")
            .description("Duration of order lookup tool calls")
            .register(registry);
    }

    @McpTool(name = "get_order", description = "Look up an order by its unique order ID.")
    public Order getOrder(String orderId) {
        return orderLookupTimer.record(() -> {
            orderLookups.increment();
            return orderService.findById(orderId)
                .orElseThrow(() -> new IllegalArgumentException(
                    "Order not found: " + orderId));
        });
    }

    @McpTool(name = "update_order_status", description = "Update the status of an order.")
    public Order updateOrderStatus(String orderId, String newStatus) {
        orderUpdates.increment();
        return orderService.updateStatus(orderId, newStatus);
    }
}

Your custom metrics appear alongside Spring AI's built-in metrics in the Prometheus scrape and your Grafana dashboards.

Retry and Resilience​

LLM APIs are external services with real failure modes: rate limits, transient network errors, temporary provider outages. Build resilience into your AI calls.

Retry with Resilience4j​

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot4</artifactId>
    <version>2.4.0</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aspectj</artifactId>
</dependency>

Resilience4j is not managed by the Spring Boot BOM, so pin the version explicitly. The @Retry annotation is an AspectJ aspect that Resilience4j silently skips when AspectJ is not on the classpath, so the AspectJ starter is required. In Spring Boot 4 this starter is spring-boot-starter-aspectj (the old spring-boot-starter-aop name no longer exists).

package com.themcpguy.springai.agent;

import io.github.resilience4j.retry.annotation.Retry;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class ResilientAgentService {

    private final ChatClient chatClient;

    ResilientAgentService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @Retry(name = "ai-chat", fallbackMethod = "chatFallback")
    public String chat(String message) {
        return chatClient.prompt()
            .user(message)
            .call()
            .content();
    }

    public String chatFallback(String message, Throwable ex) {
        // Return a graceful fallback when the AI service is unavailable
        return "I'm temporarily unable to process your request due to a service issue. " +
               "Please try again in a moment. (Error: " + ex.getMessage() + ")";
    }
}

Configure retry behaviour in application.properties:

# Retry configuration for AI calls
resilience4j.retry.instances.ai-chat.max-attempts=3
resilience4j.retry.instances.ai-chat.wait-duration=2s
resilience4j.retry.instances.ai-chat.retry-exceptions=\
  org.springframework.web.client.ResourceAccessException,\
  java.net.ConnectException

# Exponential backoff to avoid hammering a struggling service
resilience4j.retry.instances.ai-chat.enable-exponential-backoff=true
resilience4j.retry.instances.ai-chat.exponential-backoff-multiplier=2

Timeout Configuration​

Configure timeouts for both the LLM API call and MCP operations:

# Spring AI tool call timeout (how long to wait for any MCP operation to complete)
# Applies to all stdio and remote connections; no per-connection timeout exists in 2.0.x.
spring.ai.mcp.client.request-timeout=30s

AnthropicChatOptions in Spring AI 2.0.x has a built-in timeout for the HTTP call to Anthropic (default 60 seconds). Set it via a property:

# Timeout for the HTTP call to the Anthropic API (default 60s)
spring.ai.anthropic.timeout=60s

Or programmatically:

AnthropicChatOptions options = AnthropicChatOptions.builder()
    .timeout(Duration.ofSeconds(60))
    .build();

For lower-level HTTP tuning (interceptors, connection pool, dispatcher), the Anthropic model in 2.0.x runs on the official Anthropic SDK over OkHttp, not Spring's RestClient. Provide an AnthropicHttpClientBuilderCustomizer bean to customise the underlying OkHttp client:

@Bean
public AnthropicHttpClientBuilderCustomizer anthropicHttpCustomizer() {
    return builder -> {
        // customise the OkHttp-backed client builder here
    };
}

Set timeouts conservatively. LLM calls for complex tasks with multiple tool rounds can take 30-60 seconds. Too-short timeouts cause false failures. Too-long timeouts mask real problems.

Logging and Observability​

Structured logging makes AI applications observable. Use Logback (Spring Boot default) with JSON output for log aggregation systems:

<!-- logback-spring.xml -->
<configuration>
    <springProfile name="prod">
        <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
            <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
        </appender>
        <root level="WARN">
            <appender-ref ref="STDOUT"/>
        </root>
        <logger name="com.themcpguy" level="INFO"/>
    </springProfile>

    <springProfile name="!prod">
        <include resource="org/springframework/boot/logging/logback/base.xml"/>
        <logger name="org.springframework.ai.mcp" level="DEBUG"/>
    </springProfile>
</configuration>

What to Log at INFO​

Your tool methods should log meaningful business events, not protocol details:

@McpTool(name = "update_order_status", description = "Update the status of an order.")
public Order updateOrderStatus(String orderId, String newStatus) {
    log.info("Order status update: orderId={} newStatus={}", orderId, newStatus);
    Order updated = orderService.updateStatus(orderId, newStatus);
    log.info("Order updated successfully: orderId={} previousStatus={} newStatus={}",
        orderId, updated.status(), newStatus);
    return updated;
}

Use structured logging parameters (not string concatenation) so log aggregation tools can index the fields.

Distributed Tracing​

For microservice deployments, add distributed tracing to track a request from the user through the agent, through tool calls, to the MCP server:

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-zipkin</artifactId>
</dependency>

Spring AI propagates trace context through ChatClient calls and tool invocations automatically. The trace includes the LLM call, each tool call, and the MCP operations, giving you end-to-end visibility into agent execution.

Resource Limits​

MCP servers are processes. Be explicit about process limits in production.

Stdio servers (child processes):

  • Monitor child process memory and CPU via JVM process management or your container runtime
  • Set Xmx limits on any JVM-based MCP servers you start
  • Consider whether stdio servers should be external services rather than child processes at scale

Streamable HTTP connections (remote HTTP servers):

  • Configure connection pool sizes
  • Monitor open connection counts (streaming responses and legacy SSE clients hold persistent connections)
  • Use circuit breakers (Resilience4j @CircuitBreaker) to stop calling a failing remote server

The legacy spring.ai.mcp.client.sse.* properties still work in Spring AI 2.0 but are deprecated for removal, so new production deployments should use streamable-http.

LLM Token Budget​

Every AI request costs money (or compute resources, for local models). Monitor token usage and set budget limits:

@Bean
public ChatClientCustomizer tokenBudgetCustomizer(TokenBudgetAdvisor tokenBudgetAdvisor) {
    return builder -> builder.defaultAdvisors(
        tokenBudgetAdvisor  // Custom CallAdvisor you implement to enforce a token budget
    );
}

TokenBudgetAdvisor is a class you write yourself. Implement CallAdvisor (org.springframework.ai.chat.client.advisor.api.CallAdvisor), call the chain, then inspect ChatResponse.getMetadata().getUsage() from the returned ChatClientResponse and alert or throttle when usage exceeds your thresholds. Do not confuse it with Spring AI's built-in SafeGuardAdvisor, which is a sensitive-word content filter and requires a List<String> of sensitive words in its constructor.

Deployment Checklist​

Before going to production, verify:

  • API keys loaded from environment variables (not application.properties)
  • Spring profile configured for production (SPRING_PROFILES_ACTIVE=prod)
  • Actuator endpoints on a separate port, not publicly accessible
  • Health checks configured and responding correctly
  • Micrometer metrics registered and visible in your monitoring system
  • Retry logic configured for LLM calls
  • Timeouts set for all external calls (LLM API, MCP servers)
  • Logging configured with structured JSON output
  • MCP server processes (for stdio) have appropriate resource limits
  • At least one E2E smoke test passing against the production configuration
  • Fallback responses implemented for when the AI service is unavailable

Congratulations​

You've completed the Spring AI + MCP Integration course. You can now:

  • Build Spring Boot applications that consume MCP servers as AI tool providers
  • Expose Spring services as MCP tools that any MCP-compatible client can use
  • Test AI-integrated applications reliably without burning API credits
  • Ship production-grade AI features with proper observability and resilience

The companion code repository at projects/mcp-spring-ai-course/ contains complete, compilable implementations for each module. The next course in the series, Building Remote MCP Servers, covers HTTP transport in depth, OAuth authentication, and running MCP servers at scale. It is coming soon.

Further Reading: