Client Side Load Balancer in Spring Boot Microservices | Spring Cloud LoadBalancer

Client Side Load Balancer

Client Side Load Balancer

Load Balancer: Helps in distributing traffic to multiple instances of a servers. Helps in preventing single server from being overloaded with huge traffic.

Client Side Load BalancerSpring Cloud LoadBalancerRestTemplateFeignClient

Client Side Load Balancer

Spring Cloud
Load Balancer

Client Side Load Balancer

LB

Table of Contents

• 1. Client Side Load Balancer
• 2. Client Side Load Balancing with RestTemplate
• 3. Using Spring Cloud Load Balancer
• 4. Load Balancing Algorithms
• 5. Using RandomLoadBalancer Instead of RoundRobinLoadBalancer
• 6. Changing the Load Balancing Algorithm for Product Service
• 7. Different Algorithms for Product Service and Other Services
• 8. Custom LoadBalancer Logic
• 9. FeignClient and Load Balancer

1. Client Side Load Balancer

Load Balancer:

• Helps in distributing traffic to multiple instances of a servers.
• Helps in preventing single server from being overloaded with huge traffic.

Load Balancer Types:

Server Side: Centralized Load Balancer like Nginx, ELB (Elastic Load balancer). Like a separate microservice.

Server-side load balancer diagram

Client Side: Load balancing capability present inside the client (the caller) or it uses a library to take the decision. Like Spring cloud LoadBalancer, Netflix Ribbon, Istio with Sidecar etc.

Client-side load balancer and service discovery diagram

Pre-requisite:

2. Client Side Load Balancing with RestTemplate

In Service Discovery topic, we have seen that:

Microservices Service Discovery diagram

• After setting up the Service Discovery.
• In RestTemplate, we fetch the List of Instances using DiscoveryClient object.
• Then client is manually choosing the instances from the list and invoking the endpoint.

Java

@RestController
@RequestMapping("/orders")
public class OrderController {
    @Autowired
    DiscoveryClient discoveryClient;
    @Autowired
    RestTemplate restTemplate;
    @GetMapping("/{id}")
    public void callProductAPI(@PathVariable String id) {
        List<ServiceInstance> instances =
                discoveryClient.getInstances("product-service");
        URI uri = instances.get(0).getUri();
        String response = restTemplate.getForObject(
                uri + "/products/" + id, String.class);
        System.out.println("Response from Product api call is: " + response);
    }
}

Java

@Configuration
public class Config {
    @Bean
    public RestTemplate restTemplateObj() {
        return new RestTemplate();
    }
}

3. Using Spring Cloud Load Balancer

Now, lets use "Spring Cloud Load Balancer" and see the above example again:

1. Add dependency

XML

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Java

@RestController
@RequestMapping("/orders")
public class OrderController {
    @Autowired
    RestTemplate restTemplate;
    @Value("${product.service.baseurl}")
    String productBaseURL;
    @GetMapping("/{id}")
    public void callProductAPI(@PathVariable String id) {
        String response = restTemplate.getForObject(
                productBaseURL + "/products/" + id, String.class);
        System.out.println("Response from Product api call is: " + response);
    }
}

Java

@Configuration
public class Config {
    @Bean
    /*
      With this annotation, framework start
      intercepting the request, call Service Discovery,
      get instances & apply load balancing algorithm.
    */
    @LoadBalanced
    public RestTemplate restTemplateObj() {
        return new RestTemplate();
    }
}

application.properties

Properties

server.port=8081
spring.application.name=order-service

eureka.client.service-url.defaultZone=http://localhost:8761/eureka

product.service.baseurl=http://product-service

4. Load Balancing Algorithms

In RestTemplate topic, we have already seen below diagram and understood it in-depth:

Distributed microservices synchronous communication RestTemplate diagram

Load balancer algorithm structure diagram

ReactiveLoadBalancer serviceId selection diagram

• Each Load Balancing Algorithm is attached to a ServiceId. ServiceId is nothing but a client name like : "product-service".
• By default Spring Cloud Load Balancer picks RoundRobinLoadBalancer algorithm.
• Means if we are not going with Default Algorithm, then we need to define for which client (or serviceId) which algorithm we need.
• Spring Cloud Load Balancer supports only 2 algorithms i.e. RoundRobin and Random.
• If we want different algorithm like Weighted, Least Connection etc. then either we need to write custom implementation or switch to service mesh like Istio with Side car.

5. Using RandomLoadBalancer Instead of RoundRobinLoadBalancer

If we want to use say RandomLoadBalancer instead of Default RoundRobinLoadBalancer, we can override it in configuration.

(Same as previous)

Spring Cloud Load Balancer configuration and application properties code screenshot

6. Changing the Load Balancing Algorithm for Product Service

Now Changing the Load Balancing algorithm for Product Service from default RoundRobinLoadBalancer to RandomLoadBalancer.

Java

@SpringBootApplication
@EnableFeignClients
@LoadBalancerClient(
        name = "product-service",
        configuration = LoadBalancerProductClientConfig.class
)
public class OrderserviceApplication {
    public static void main(String[] args) {
        SpringApplication.run(OrderserviceApplication.class, args);
    }
}

** Normal Config class(like above) generally invokes only at application startup.
But this LoadBalancerClient config invokes at runtime too, when "product-service" is used: "http://product-service/xyz"

Java

@Configuration
public class LoadBalancerProductClientConfig {
    @Bean
    public ReactorLoadBalancer<ServiceInstance> productClientLoadBalancer(
            LoadBalancerClientFactory factory) {
        return new RandomLoadBalancer(
                factory.getLazyProvider(
                        "product-service",
                        ServiceInstanceListSupplier.class
                ),
                "product-service"
        );
    }
}

ServiceInstanceListSupplier, is actually the one, which helps to invoke Service Discovery and ask for instances of "product-service".

New RandomLoadBalancer(ServiceInstanceListSupplier, serviceId)
we pass serviceId because it mapped this algorithm to this service ID only. So for "product-service" only RandomLoadBalancer is used.

7. Different Algorithms for Product Service and Other Services

What if, for "product-service" I want Algorithm A and for all other clients (or ServiceId), I want Algorithm B

Java

@SpringBootApplication
@EnableFeignClients
/*
     First child clients config will be
     loaded, then default one.
*/
@LoadBalancerClients(
        defaultConfiguration = LoadBalancerGlobalConfig.class,
        value = {
                @LoadBalancerClient(
                        name = "product-service",
                        configuration = LoadBalancerProductClientConfig.class
                )
        }
)
public class OrderserviceApplication {
    public static void main(String[] args) {
        SpringApplication.run(OrderserviceApplication.class, args);
    }
}

• As we already know, this LoadBalancerClients config is loaded at runtime for each service ID, So this environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME) will resolved at runtime.
• @ConditionalOnMissingBean is required at Global config class, because say for "product-service" two ReactorLoadBalancer bean should not get created one from LoadBalancerProductClientConfig and another from LoadBalancerGlobalConfig.
• So below, for "product-service" Random Load balancer algorithm will be used and for all other services "RoundRobin Algorithm will be used.

Java

@Configuration
@ConditionalOnMissingBean(ReactorLoadBalancer.class)
public class LoadBalancerGlobalConfig {
    @Bean
    public ReactorLoadBalancer<ServiceInstance> randomLoadBalancer(
            LoadBalancerClientFactory factory,
            Environment environment) {
        String serviceId =
                environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RoundRobinLoadBalancer(
                factory.getLazyProvider(
                        serviceId,
                        ServiceInstanceListSupplier.class
                ),
                serviceId
        );
    }
}

8. Custom LoadBalancer Logic

If we want to use say our CustomLoadBalancer logic:

Custom LoadBalancer implementation code screenshot

Java

public class MyCustomLoadBalancer implements
        ReactorServiceInstanceLoadBalancer {
    private final ObjectProvider<ServiceInstanceListSupplier>
            serviceInstanceSuppliers;
    private final String serviceId;
    public MyCustomLoadBalancer(
            ObjectProvider<ServiceInstanceListSupplier>
                    serviceInstanceSuppliers,
            String serviceId) {
        this.serviceInstanceSuppliers = serviceInstanceSuppliers;
        this.serviceId = serviceId;
    }
    @Override
    public Mono<Response<ServiceInstance>> choose(Request request) {
        return serviceInstanceSuppliers
                .getIfAvailable()
                .get()
                .next()
                .map(instances -> {
                    if (instances == null || instances.isEmpty()) {
                        return new EmptyResponse();
                    }
                    // Custom load balancing algorithm (demo: pick first instance)
                    return new DefaultResponse(instances.get(0));
                });
    }
}

Java

@Configuration
public class LoadBalancerProductClientConfig {
    @Bean
    public ReactorLoadBalancer<ServiceInstance> productClientLoadBalancer(
            LoadBalancerClientFactory factory) {
        return new MyCustomLoadBalancer(
                factory.getLazyProvider(
                        "product-service",
                        ServiceInstanceListSupplier.class
                ),
                "product-service"
        );
    }
}

9. FeignClient and Load Balancer

During Service Discovery, we have already seen how FeignClient handles load balancer automatically.

So for load balancer everything is same like RestTemplate, no change at all.

FeignClient and load balancer code screenshot

Java

@RestController
@RequestMapping("/orders")
public class OrderController {
    @Autowired
    ProductClient productClient;
    @GetMapping("/{id}")
    public void callProductAPI(@PathVariable String id) {
        String response = productClient.getProductById(id);
        System.out.println("Response from Product api call is: " + response);
    }
}

Java

@SpringBootApplication
@EnableFeignClients
@LoadBalancerClient(
        name = "product-service",
        configuration = LoadBalancerProductClientConfig.class
)
public class OrderserviceApplication {
    public static void main(String[] args) {
        SpringApplication.run(OrderserviceApplication.class, args);
    }
}

Java

@Configuration
public class LoadBalancerProductClientConfig {
    @Bean
    public ReactorLoadBalancer<ServiceInstance> productClientLoadBalancer(
            LoadBalancerClientFactory factory) {
        return new RandomLoadBalancer(
                factory.getLazyProvider(
                        "product-service",
                        ServiceInstanceListSupplier.class
                ),
                "product-service"
        );
    }
}