Fault Tolerance

The langchain4j-cdi-fault-tolerance module integrates MicroProfile Fault Tolerance with your AI services. This enables resilient AI service calls with automatic retries, timeouts, circuit breakers, and fallback mechanisms.

Setup

<dependency>
    <groupId>dev.langchain4j.cdi.mp</groupId>
    <artifactId>langchain4j-cdi-fault-tolerance</artifactId>
    <version>${langchain4j-cdi.version}</version>
</dependency>

Retry

Automatically retry failed AI calls:

@RegisterAIService(chatModelName = "#default")
public interface ChatBot {

    @Retry(maxRetries = 3, delay = 1000)
    String chat(String userMessage);
}

Timeout

Set a maximum duration for AI calls:

@RegisterAIService(chatModelName = "#default")
public interface ChatBot {

    @Timeout(5000)
    String chat(String userMessage);
}

Circuit Breaker

Prevent cascading failures when the AI provider is down:

@RegisterAIService(chatModelName = "#default")
public interface ChatBot {

    @CircuitBreaker(requestVolumeThreshold = 10, failureRatio = 0.5, delay = 30000)
    String chat(String userMessage);
}

Fallback

Provide alternative responses when the AI service is unavailable:

@RegisterAIService(chatModelName = "#default")
public interface ChatBot {

    @Fallback(fallbackMethod = "chatFallback")
    String chat(String userMessage);

    default String chatFallback(String userMessage) {
        return "I'm sorry, the AI service is currently unavailable. Please try again later.";
    }
}

Combining Strategies

Multiple fault tolerance strategies can be combined:

@RegisterAIService(chatModelName = "#default")
public interface ResilientChatBot {

    @Retry(maxRetries = 3, delay = 500)
    @Timeout(10000)
    @CircuitBreaker(requestVolumeThreshold = 20, failureRatio = 0.5)
    @Fallback(fallbackMethod = "chatFallback")
    String chat(String userMessage);

    default String chatFallback(String userMessage) {
        return "Service temporarily unavailable.";
    }
}