REST client with desired NFRs using Spring RestTemplate

May 19, 2020 cloud native performance spring boot
Share on:

Background

In this contemporary world of enterprise application development, Microservice Architecture has become defacto paradigm. With this new paradigm, an application is going to have myriad set of independent and autonomous (micro)services which will be calling each other. One of the fundamental characteristics of Microservice Architecture is

Services must be easily consumable

Hence most of the services implemented will be exposing REST APIs. In order to consume these REST APIs, each Microservice application will have to implement a REST client. As most of the applications are built using Spring Boot, it is quite obvious that REST clients must be realized by using Spring's RestTemplate. Since Performance of application is of paramount importance, input / output operation that happens whilst invoking REST api via RestTemplate assumes lot of significance. Hence I felt a need to not only pen down various aspects to be kept in mind while implementing REST client using RestTemplate & Apache HttpClient but also demonstrate it via a full blown implementation

Source Code @ Github

If you have come this far :) and are convinced to go through remaining of the article, this is what I will be covering :

  1. Optimal connection pool management
  2. Resilient REST client invocation
  3. Observable connection pool
  4. Implementing secured REST client

1. Optimal Connection Pool Management

I am pretty much sure that most of us would have encountered issues pertaining to connection pool of REST clients - e.g. Connection pool getting exhausted because of incorrect configuration, or connections once acquired not being released and cleaned up etc. One can optimize the overall implementation if one really knows key configurable parameters and ways to manage them.

1.1 Customizing RequestConfig

RequestConfig needs to be customized for finer control on key HTTP connection parameters -

  • CONNECT_TIMEOUT - Indicates timeout in milliseconds until a connection is established
  • CONNECTION_REQUEST_TIMEOUT - Indicates timeout when requesting a connection from the connection manager
  • SOCKET_TIMEOUT - Indicates timeout for waiting for data OR maximum period of inactivity between 2 consecutive data packets

Configuring HttpClient using RequestConfig

1private RequestConfig prepareHttpClientRequestConfig() {
2  RequestConfig requestConfig =  RequestConfig
3                                  .custom()
4                                  .setConnectTimeout(CONNECT_TIMEOUT)
5                                  .setConnectionRequestTimeout(CONNECTION_REQUEST_TIMEOUT)
6                                  .setSocketTimeout(SOCKET_TIMEOUT)
7                                  .build();
8  return requestConfig;
9}

1.2 Connection pool management via PoolingHttpClientConnectionManager

ConnectionManager needs to be instantiated so that application can manage certain key configurations -

  • MAX_CONNECTIONS - Sets maximum number of total connections
  • MAX_PER_ROUTE_CONNECTION - Sets maximum number of connections per route
  • VALIDATE_AFTER_INACTIVITY - Sets duration after which persistent connections needs to be re-validated before leasing

HttpClient configuration for ConnectionPool using PoolingHttpClientConnectionManager

1public PoolingHttpClientConnectionManager poolingConnectionManager() {
2	PoolingHttpClientConnectionManager poolingConnectionManager =
3		new PoolingHttpClientConnectionManager(getConnectionSocketFactoryRegistry());
4	poolingConnectionManager.setMaxTotal(MAX_CONNECTIONS);
5	poolingConnectionManager.setDefaultMaxPerRoute(MAX_PER_ROUTE_CONNECTION);
6	poolingConnectionManager.setValidateAfterInactivity(VALIDATE_AFTER_INACTIVITY_IN_MILLIS);
7	return poolingConnectionManager;
8}

1.3 Configuring ConnectionKeepAlive strategy

This configuration is also equally important, as its absence from Http Headers will lead httpclient to assume that connection can be kept alive till eternity. For finer control on http connection parameters it would be prudent to implement a custom keep alive strategy

Http client config with custom defined keep alive strategy

 1private ConnectionKeepAliveStrategy connectionKeepAliveStrategy() {
 2  return new DefaultConnectionKeepAliveStrategy() {
 3    @Override
 4    public long getKeepAliveDuration(final HttpResponse response, final HttpContext context) {
 5      long keepAliveDuration = super.getKeepAliveDuration(response, context);
 6      if (keepAliveDuration < 0) {
 7        keepAliveDuration = DEFAULT_KEEP_ALIVE_TIME_MILLIS;
 8      } else if (keepAliveDuration > MAX_KEEP_ALIVE_TIME_MILLIS) {
 9        keepAliveDuration = MAX_KEEP_ALIVE_TIME_MILLIS;
10      }
11      return keepAliveDuration;
12    }
13  };
14}

1.4 Housekeeping connection pool

So far we saw that connection pool as an application resource is being managed with various configuration parameters. As these resources are key to optimal functioning of application, it needs to perform periodic house keeping which does the following -

  • Evicts connection that are expired due to prolonged period of inactivity
  • Evicts connection that have been idle based on application configured IDLE_CONNECTION_WAIT_TIME_SECS
  • From performance standpoint, it is recommended to execute such house keeping jobs at configured interval in a separate thread with its own thread pool configuration using ScheduledThreadPoolExecutor

Http client config with house keeping of idle and expired connections

 1public Runnable idleAndExpiredConnectionProcessor(final PoolingHttpClientConnectionManager connectionManager) {
 2  return new Runnable() {
 3    @Override
 4    @Scheduled(fixedDelay = 20000)
 5    public void run() {
 6      try {
 7        if (connectionManager != null) {
 8            connectionManager.closeExpiredConnections();
 9            connectionManager.closeIdleConnections(IDLE_CONNECTION_WAIT_TIME_SECS, TimeUnit.SECONDS);
10        }
11      } catch (Exception e) {
12        // Log errors
13      }
14    }
15  };
16}

Note - Considering the fact that Cloud Native applications are implemented considering 12 factors, all the above configurations should be managed via externalized configurations.

2. Resilient invocation

While we all know the reasons for adopting Microservice Architecture, we are also cognizant of the fact that it comes with its own set of complexities and challenges. With distributed architecture, few of the major pain points for any application which consumes REST API are - 

  • Socket Exception - Caused by temporary server overload due to which it rejects incoming requests
  • Timeout Exception - Caused by temporary input / output latency. E.g. Extremely slow DB query resulting in timeout

Phew ! Things are going damn crazy with Microservice Architecture :) Thankfully we have Spring to our rescue with Spring Retry library. We can use its capabilities to make our REST API calls more resilient. I have used @Retryable and @Recover to configure retry strategy with exponential back-off and fallback strategy.

For our application we have configured retry (using @Retryable) for RemoteServiceUnavailableException with maxAttempts_ as 8 and with an exponential back-off policy i.e. 1000 * (2 to the power of n) seconds, where n = no. of retry attempt. We have also configured a fallback strategy using @Recover_

Spring Retry in itself is a candidate for a separate blog, hence we are not delving deeper beyond this.

Retryable RestClient using Spring Retry

1@Retryable(value = {RemoteServiceUnavailableException.class}, maxAttempts = 8,
2    backoff = @Backoff(delay = 1000, multiplier = 2), label = "generate-alias-retry-label")
3String generateAlias(String cardNo);
4
5@Recover
6String fallbackForGenerateAlias(Throwable th, String cardNo);

3. Observable connection pool

As we all know that in distributed architecture, observability of application / infrastructure resources is extremely important. The same holds good for Http connection pool. ConnectionManager provides methods to fetch its total statistics and statistics for each route at a given point of time. In order to avoid any performance issues, it is recommended to execute such metric logger at configured interval in a separate thread with its own thread pool configuration using ScheduledThreadPoolExecutor

HttpClient metrics extractor

 1public Runnable connectionPoolMetricsLogger(final PoolingHttpClientConnectionManager connectionManager) {
 2  return new Runnable() {
 3    @Override
 4    @Scheduled(fixedDelay = 30000)
 5    public void run() {
 6      final StringBuilder buffer = new StringBuilder();
 7      try {
 8        if (connectionManager != null) {
 9          final PoolStats totalPoolStats = connectionManager.getTotalStats();
10          log.info(" ** HTTP Client Connection Pool Stats : Available = {}, Leased = {}, Pending = {}, Max = {} **",
11              totalPoolStats.getAvailable(), totalPoolStats.getLeased(), totalPoolStats.getPending(), totalPoolStats.getMax());
12          connectionManager
13            .getRoutes()
14            .stream()
15            .forEach(route -> {
16              final PoolStats routeStats = connectionManager.getStats(route);
17              buffer
18                .append(" ++ HTTP Client Connection Pool Route Pool Stats ++ ")
19                .append(" Route : " + route.toString())
20                .append(" Available : " + routeStats.getAvailable())
21                .append(" Leased : " + routeStats.getLeased())
22                .append(" Pending : " + routeStats.getPending())
23                .append(" Max : " + routeStats.getMax());
24            });
25          log.info(buffer.toString());
26        }
27      } catch (Exception e) {
28        log.error("Exception occurred whilst logging http connection pool stats. msg = {}, e = {}", e.getMessage(), e);
29      }
30    }
31  };
32}

Captured metrics can be used for better monitoring and alerting which can not only assist in proactive addressing of Http Client's connection pool issues, but it may also help in troubleshooting production incidents pertaining to connection pool. This implicitly will help in reducing MTTR

4. Implementing secured REST client

For lot of APIs, depending on the nature of data it deals with, one may need to implement secured REST client. This is also demonstrated in code by using HttpClientSslHelper which will be responsible for setting up the SSLContext. This SSLContext can than be used to instantiate Registry of ConnectionSocketFactory.

Note - HttpClientSslHelper should be conditionally instantiated based on what kind of security strategy (i.e. Untrusted / Trusted) needs to be in place for the corresponding environment and profile (viz test, CI, dev etc.)

Conclusion

By knowing and understanding various aspects of HttpClient along with its key configuration parameters we can now build a highly performant, resilient, observable and secured REST client using RestTemplate.

comments powered by Disqus