Transaction Management in Microservices

As we know that without transactions there won’t be a use of any real-time application. The transaction could be anything like fund transfer, email sending, placing the order, bill payment etc. In a simpler words transaction is a unit of work which will be performed in any business application which persists data in the certain state, but each unit of work should be entirely committed or rolled back to ensure data integrity.

As a part of this Blog, I will talk about different ways of managing transactions in a distributed Microservice architecture. I will also cover how transaction management is different in Microservices compared to Monolithic applications.

Monolithic vs Microservices

Monolithic applications primarily stick to ACID properties while dealing with transactions. The Biggest advantage for Monolithic application in transaction management is a single and common database server. Transactions can be initiated at the database level and can be committed or rolled back based on the final outcome of the transaction. This way of initiating a transaction, performing operations and persisting or non-persistence of data is much easier to develop. However, this advantage becomes a serious challenge while running high volume production application. A simple lock on a critical table can lead to disastrous results like non-availability of the entire application.

Microservices guidelines strongly recommend you to use the Single Repository Principle(SRP), which means each Microservice maintains its own database and no other service should access the other service’s database directly. There is no direct and simple way of maintaining ACID principles across multiple databases. This is where the real challenge lies for transaction management in Microservices.

Pragmatic approaches for SRP

Even though Microservices guidelines strongly recommend you to use a separate database server for each Microservice, as a designer we take certain compromises for practical reasons in the early stages of Microservices development.

Following are the ways where you implement Microservices but with a certain compromise:

• Maintain a set of tables for specific to service in a database                                                                                                               When you maintain all the tables in a single database, keep prefix or suffix to each table to indicate the Microservice it belongs to. Do not maintain any foreign key constraints across tables belonging to different Microservices. When the right time occurs move tables specific database server belongs the specific Microservice.
• Create and use separate database schema for each service                                                                                                                Here keep the schema name close to the corresponding Microservice.

But in both the above cases, do not allow one Microservice to access the database of other Microservice directly. If one Microservice need another Microservice data, it should call service endpoint specific to that required data.

In case of high demand services, there is a possibility of the push for maintaining the common database server across multiple Microservices citing possible network latencies while invoking other sets of Microservices. The recommended solution for this problem is data duplication rather than going for the single database server. If we choose data duplication we may not be able to achieve immediate consistency, but you need to ensure eventual consistency of the data.

Ways to handle transactions in Microservices

Over a period of time Microservices community deviced different ways of handling transactions across Microservices. Some of the approaches address transaction management at the design level and others at design and coding level.

Following are the list of different approaches which we can use in managing transactions. The good part is that we can use one or all of these below approaches in a given Microservices environment. In a given environment, two Microservices can use one approach and other can follow the different approach for transaction management.

• Avoiding transactions across Microservices
• Two-Phase Commit Protocol
  • XA Standard
  • REST-AT Standard Draft
• Eventual Consistency and Compensation

Avoiding transactions across Microservices

Let’s take the example of twitter application

Assume that If Twitter maintains two different Microservices for user profile information and tweets information then want to show the last tweet date time of User on User profile screen, so storing/updating tweet’s date, time everytime whenever Users tweets any new tweet in Users table, so we need to maintain distributed transaction across tweets and users information Microservices.

Another way is to get the last tweet’s date, time of users while showing user’s profile information by making a separate request to tweets microservice to get the latest tweet’s date time and show it dynamically, this way we can stop avoiding the distributed transactions across multiple services without storing tweet’s date time in the User table.

This approach addresses the transaction management challenge primarily at the design level.

Two-Phase Commit Protocol

This mechanism is designed initially for distributed systems. As Microservices architecture inherently distributed systems in nature, we can use the Two-phase commit protocol (or 2PC) as one of the approaches. Primary drivers in a distributed transaction management are the message broker/transaction coordinator.

The distributed transaction contains two steps:

1. Prepare phase
2. Commit or Rollback phase

Prepare Phase:

All the participants of a transaction in this phase will be prepared for the commit and inform the transaction coordinator/message broker that they are ready for completing the transaction

Commit or Rollback phase:

In this phase, transaction coordinator will issue one of the commands they are a commit or a rollback to all the participants.
The main issue with the 2PC approach is that it is a bit slow compared to the time for the operation on a single Microservice because it has to coordinate the transaction between services even if all the microservices are on the same network, still operation will be slow. So we need to be careful while implementing this for high demand services.

While Implementing 2PC you can follow either of the two standards that are described below.

• XA Standard

The 2PC has the specification i.e XA standard for implementing the distributed transactions across in all the supporting software components. All JTA-compliant application servers support this standard out of the box. The participants in a distributed transactions could be the different Microservices which have the dedicated databases.

But anyways to make use of this specification, Microservices must be deployed in a single JTA platform, it is not always possible in a Microservices architecture.

• REST-AT Standard Draft

The REST-AT is also one of the standards, it is still under development by RedHat and not yet out of the draft, but this standard is supported by WildFly application server currently out of the box.

By using this standard, application server acts as a transaction coordinator with a specific REST API for handling distributed transactions.

To access the local resources of a Microservice in a distributed transaction, still, we need to deploy these resources in a single JTA platform.

Eventual Consistency and Compensation

When we are dealing with distributed transactions in Microservices, we need to make sure that system should be eventually consistent at some point of time in the future. This model doesn’t force to use ACID transactions across microservice but forces some mechanism for ensuring for consistency.

Each service which involved in the transaction should be responsible to update the user with the proper status of the transaction even if next consecutive services are failed to respond and should handle them whenever services are up and make sure all the scheduled transactions are completed and data in the system is consistent.

SUMMARY

As a part of this blog, we talked about differences in transaction management between Monolithic application and Microservices. We also covered different ways of implementing Transaction management in Microservices environment. After reading this article, I hope you will be in better position to understand the nuances of Transaction Management in Microservices area and equipped to take an informed decision while addressing challenges in this specific area.

References :

https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/

https://spring.io/guides/gs/managing-transactions/