- Introduction and top 3 questions
- Monolith vs Microservice
- Microservices architecture
- API Gateway
- Service discovery and Intercommunication
Introduction of Microservices
Microservice architecture provides a system of development for large team size to develop and maintain a large and scalable product. For example, a product like Uber where Millions of ride happening daily, will need microservice architecture for sure, like Netflix kind of system cannot provide such smooth and never go down experience with monolith architecture.
Top 3 questions
Q1. On what basis we should make the microservices
The answer is not based on throughput or latency, it should be based on the semantic separation of product module, which a team can own. Assume Bookmyshow, ticket booking is having very high throughput and system is getting slow, then suppose someone suggest an idea of splitting the booking service to booking request and booking confirmation is wrong design of microservice, the entire booking has to be one microservice for sure, because that is a semantic module of a product, yes we can split payment as a service, we can split inventory, availability as a service. And to address the throughput and latency issue with booking service, we can think of a different architecture and ways to optimize the performance.
Q2. When to create a new microservice?
To start with you don’t need a microservice architecture, but if you know that within 6-12 months, it’s going to a good scale, start the basic architecture from day 1. To start with you can split Authentication, Communication as a separate service and deploy them on a single small/medium instance, and keep the other service into a single machine. And keep the services a separate package and use function calls in the same project, so that in future you can split those packages and call it another microservice, and replace a function call with API calls. So the foundation can be kept on day 1, however, implementation can be done later depends on traffic and scale.
Q3. What are the standard micro-service I should think of?
This entirely depends on product and the business, however considering products like Uber, e-commerce, BookMyShow, Cleartrip and Amazon kind of products, few things are pretty straight forward, like Supply, Inventory, Availability, Booking, Payment are the core modules which will be there almost every, search and recommendation can be a module, Authentication and Communication will be surely a microservice.
Monolith vs Microservice
- Good for creating MVP or initial phase of product building
- Monolith is faster to code because everything is in the same system, same DB
- Easy to understand and development is significantly faster compared to microservices
- Deployment is very easy
- Low cost – if you have limited money, don’t bother about microservices
- Anything goes wrong, almost all system goes down
- Microservices can give separate ownership of module to a team, scaling the product, team and technology becomes all aligned
- Monolith is not largely scalable
- Monolith deployment may take the entire system down or need to deploy the whole
- Almost dedicated DevOps is required
- Any DB change, migration or query issue will impact the whole DB
- High cost – if you predictably know that MAU/DAU gonna rise, or there can be a surge in traffic, it’s worth investing in microservices
- Even if one service is down, the system can still be up
Microservices architecture is based on splitting up the semantic-based module along with their own database or other supporting software needed, like a queue or caching software or indexing software etc.
- Microservices architecture typically wants to use REST architecture also and want to create microservices which will serve REST APIs which different method types.
- Now a days separation of Front development with back-end development is also very common, and Front end typically can be developed in ReactJS or AngularJS or another other, where as all the APIs will be developed by back-end splitting into different microservice.
- Front-end components can also be deployed into a separate microservice, or it can be cached on either Gateway level or even in CDN or Edge server.
- Back-end service will first reach to Network load balancer, which will forward the request based on certain rules like India traffic to separate Gateway and USA traffic to separate Gateway. For smaller scale, this is not necessary to be done.
- Most important is, different microservice should have their separate database and that isolation is critical to make the architecture truly a microservices.
- The different team responsible for their service can create different architecture for DB as well, like implementing Master-slave or Distributed or anything else. They might use SQL or NoSQL DB, or few might not need any persistence, they might use just Redis/Hazelcast, or few might use Solr/ES for their IR (Information Retrieval) system, or few might use Spark/Storm/Flink as well. For example, an Availability service in Uber, will not need a structured database, you can work with Redis, which is persistable as well as a cache to perform better on latency and throughput.
- Microservices are always deployed in a cluster and all the information about all the microservices and their instance (with IP and port), will be stored in Registry service.
- API gateway is the heart of microservice architecture, which I’ll discuss in detail separately.
- Different microservices can be connected to Gateway, and requested to be forwarded to respective microservice based on many parameters, standard is based on URL, if it contains /booking then go to Booking service, if it contains /payment then to go payment service, this is done through reverse proxy or few people call it dynamic routing as well.
- Gateway will also act as Load balancer to route the traffic to different instances of a cluster.
- Different microservices can communicate with each other synchronously(in the request time) using REST API as well as asynchronously using a Messaging queue/pub-sub kind of system.
- What if I’ve to call multiple microservice to get data, how do I do that?
- What if microservice doesn’t take my request further? Or gateway timeout happens?
- What is the approach of authentication, and security on which layer?
Answer to all the above questions is API Gateway.
API Gateway can also be considered as a service whose sole purpose is to create an abstraction of certain things which would be common among all the microservice, like authentication, monitoring etc. API Gateway is also designed for request routing and load balancing among the cluster of microservices.
The objectives of the API Gateway
- Security, SSL offloading
- Authentication and Authorization
- Request monitoring, Logging to understand tech matrix – ELK, Logstash
- Dynamic routing – Request forwarding based on end point
- Load balancing – distribute the load among a cluster of microservices
- Orchestration – To make multiple request to different services and aggregate
- Protocol translation (for example HTTP to XMPP, or HTTPS to HTTPS)
- Rate Limiting – controlling the rate of traffic over a network
- Circuit breaker – to protect against transient failure, to prevent cascading failures
Service discovery and Intercommunication
Q1. Why do I need service discovery?
Let’s imagine that one service like say Search service need to get the dynamic price, so it has to make a service that has a REST API call to Pricing service, so Search service needs to know the network location (IP address and port) of a service instance. In a traditional application running on physical hardware, the network locations of service instances are relatively static. However, in cloud architecture Service instances have dynamically assigned network locations, the set of service instances changes dynamically because of autoscaling, failures, and upgrades. Consequently, we need to use a more elaborate service discovery mechanism.
We can do client side service discover or server side client discovery, both have few benefits and drawbacks. Below is a diagram to see server side service discovery mechanism
Q2. What is a service registry?
The service registry is a key part of service discovery. It is a database containing the network locations of service instances. A service registry needs to be highly available and up to date, so they need to maintain multiple replications of registry service as well, to avoid SPOF.
Typically all the services will need to make a REST API call in a certain frequency (30 seconds) to registry service, here we are believing that registry service is not having too many variations of request, so we can assume with high confidence that registry service will not need much to do scale in or out, so it would be almost static address.
Few examples of registry services are Netflix Eureka, Zookeeper, etcd, consul.
Q3. Why do we need Inter-communication?
In case of monolith architecture, one module say Search needs a dynamic price, then it only has to do it to make a function call to Pricing module and get the latest price, now the same thing is quite complex in micro-service architecture, because now they exist in different servers altogether, so they need to use certain mechanism to communicate to other services.
Q4. What are the ways to communicate?
Communication can be one to one, or can to one to many also, communication can be synchronous and asynchronous also. In case of Search service needs a dynamic price, so it would surely make a synch call to Pricing because it can not wait for any delay at all, whereas in case of same Search service if it has to record the search keyword(or phrase) for customer search pattern analysis, it would make asynch call to Search Analytics service. Asynch call can be made by asynch call feature in code, it using a standard message-bus. A message bus, can be a pub-sub kind of software or can be a queue kind of pattern also. ActiveMQ or RabbitMQ are pretty standard for queue pattern, Kafka is pub-sub implementation.
Q5. Difference in Messaging queue and Pub-sub?
Messaging Queue is typically designed for one to one communication, and it’s called producer-consumer pattern. A message queue receives incoming messages and ensures that each message for a given topic or channel is delivered to and processed by exactly one consumer.
Message queues can support high rates of consumption by adding multiple consumers for each topic, but only one consumer will receive each message on the topic. Which consumer receives which message is determined by the implementation of the message queue. To ensure that a message is only processed by one consumer, each message is deleted from the queue once it has been received and processed by a consumer (i.e. once a consumer has acknowledged consumption of the message to the messaging system). In the case of message queue, it is not necessary to process messages in order.
On the contrary to message queuing, publish-subscribe messaging allows multiple consumers to receive each message in a topic. Further, pub-sub messaging ensures that each consumer receives messages in a topic in the exact order in which they were received by the messaging system.
Publish-subscribe use cases are frequently associated with stateful applications. Stateful applications care about the order of messages received because the ordering of messages determines the state of the application and will, as a result, impact the correctness of whatever processing logic the application needs to apply.
Technologies such as Apache ActiveMQ, Amazon SQS, IBM Websphere MQ, RabbitMQ, and RocketMQwere initially designed primarily for message queuing use cases. Other technologies such as Apache Kafka and Google Cloud Pub/Sub were designed primarily to support publish-subscribe use cases. Other, often newer solutions such as Apache Pulsar provide support for both message queuing and pub-sub messaging.
Every system would typically use a combination of communication mechanism to interact. Synch and asynch, HTTP and pub-sub, it all depends on what request needs exactly runtime response of eventual response.