Microservices with Layers

It's common to use the term Microservices to refer to the pattern of software development which emphasises deploying smaller services which communicate with each other using web service APIs, instead of deploying larger services which use internal APIs (i.e. direct procedure calls which require bundling in the code).

There are lots of advantages to smaller services: smaller teams, separate source control repositories, more agile, independently scalable but some pitfalls when you apply this approach: more error checking, runtime overhead, coordination and availability requirements across services, and coordination across teams when interfaces change.

It takes balancing a lot of factors to make the right decision, and it's costly to refactor when you make a poor choice. Some important considerations include whether or not the databases of the services are easy to decouple, or would they benefit from transactional integrity between the data? If you can't do updates in transactions, you may need to handle a lot more error cases which is expensive from a coding perspective. What's the likelihood APIs will need to change in incompatible ways between services? If you split tightly coupled logic across services, you will spend more time managing the boundary between them if they are decoupled rather than coupled for example. If two databases are decoupled when they should be coupled, you'll have more overhead inserting and updating ever record twice, and writing and testing the code to deal with the potential for errors when only one insert of the two was saved. How do you separate projects so they can be versioned independently, but share source code so they stay coordinated without lots of extra "glue code"?

I could write a lot more about these tradeoffs, but suffice it to say they are difficult decisions to always get right, and trying to get them right can take a lot of time in a stage of development when you should be quickly prototyping things. It would be much better way if we could build our code in such a way that we can easily repackage it later based on dependencies without having to stop and rewrite the whole thing. It would be nice to be able to move code to decouple things a step at a time and when a piece can be decoupled, it's split off by itself by changing the packaging. First you move it to a new source repository so it's versioned independently but still built with the old source code. That may be enough to decouple the teams, but you can start separating it into a new process as a next step. We need a way to run the same code in both coupled and decoupled configurations and easily move features back and forth across separable boundaries, without breaking contracts and only dealing with necessary error checking.

Layers let you do just that. In the early stages of development, it's fastest to build things in one piece - e.g. one file per class, or even everything in one file. As it evolves you split pieces out to avoid any one piece from getting too large. You keep a piece of logic together if it implements a logical unit - things that belong together because they are created or updated at the same time, and/or because the code is likely to be changed at the same time. Properties stay near the methods which manipulate them for example. Parts of the code which have the same dependencies will also be near each other in the file in this approach when different parts of the same type have different structural dependencies (e.g. UI and database).

When one file is getting too large, you can take a unit of functionality and put it into a new layer and have that layer extend any layers it depends upon. For example, many applications have a "User" which stores data about a user in the system. It would be logical to split the user's profile data (i.e. address, name, etc.) from the behavioral history data (e.g. the last time they logged in, the last pages they visited, etc.). Now there's probably some code in our User file which is used by both pieces. Let's say the user's id and login name. We can split this into three layers: user.core, user.profile, and user.history. When we combine all three, we have the same code we started with, but when we run core and profile in one process and core and history in another, we have a simple micro-services deployment.

If the two need to communicate using APIs, we can detect that it's a remote method call (i.e. it's calling a method that's part of the same bundle of layers, but not in the same process) and hide that notion from the program if the client can use synchronous remote calls. If not, they can put the remote method call in a data binding expression, or code it up to process the results in a callback. Either way, we can still run that code using a local method call so we can still provide an incremental way to run both configurations.

To deploy microservices, it's common to have a central dispatcher service which delegates the service call to the appropriate service. Framework layers can be easily added to generate the configuration for these dispatchers so it stays in sync with the current configuration.

Let's say that today you have a huge code base that is only deployable as a monolithic service and you'd like to break it up into pieces. While you could start making major surgery to the code, you'll be touching so many files and the timeframe will be large that you'll need to do it on a branch. In this scenario, if you have "many changes" going into the old code base, you might never complete your project or suffer a major hiccup during the transition.

With StrataCode's robust, type-safe splitting and merging, you can use the IntelliJ plugin to break apart your code into layers. You can compare the merged output with the original each step of the way. You'll know immediately when you have made two layers independent and can collect the set of APIs that they use from each other. All along the way, you are building the original system so there's no need to branch. Before you launch, run some A/B tests between the old and new systems to roll it out slowly and cut over when the new system works better.