Avoiding copies for customizations

by Jeffrey Vroom

Question: Have you ever had to copy lots of code to customize it, then struggled long-term to maintain the copies?

I've seen this strategy overload more than one company in technical debt. They copied an entire code base, or created a branch in their source control and then changed large amounts of code to quickly make changes for the next new customer. When you don't have time to make a plugin or refactor using object oriented principles to share the code, and instead copy code wholesale, you generate technical debt quickly. Pretty soon, a small engineering team can no longer maintain feature demands for existing customers while making customized copies for new customers. Small patches to shared code need to be merged dozens of times, not just for the code but for all of the automated tests that were copied too. When some platform evolution is required, an even larger effort is required to migrate all of the customers.

For those sales driven companies that follow this "slash and burn" style development, at some point they lose the ability to maintain the core product and all of the customizations in way that's competitive.

To avoid the technical debt, most engineers avoid the copies by adding "feature toggles", ways to add the new code but enable it only for the new customer. This approach eliminates the copied code, but it takes careful design to keep the code manageable. One feature toggle can be used in dozens of places in the code, and once here are dozens of feature toggles, the code becomes a tricky maze that's hard to read and maintain.

For some projects, it makes more sense to build a plugin architecture, or others might use object-oriented inheritance and polymorphism, combined with the inversion-of-control pattern to make customizations more explicit and manageable. Let's compare each of these design approaches to managing customizations with StrataCode and layers.

Plugin frameworks

Wordpress is a great example of a plugin framework which offers built-in customization points (aka hinge points): specific API contracts used by developers to extend or modify the behavior of the UI from separate code. The API contracts for plugins are carefully designed to solve use cases customers need and typically give the plugin the ability to add new fields, menu items, editor screens, document types, or more. Plugins can customize page layout and style sheets in many cases and frequently use external services, all with their own database and hosting infrastructure.

All-in-all, it takes a lot of work to design, build and maintain a great plugin framework. It's like a mini-application framework by itself with install, upgrade, compatibility constraints, ordering, uninstall, etc. Plugin-frameworks are so successful it's worth the effort for some application areas where there are diverse requirements (e.g. content management and e-commerce). Still it is very common to run into the limits of what you can customize in a plugin, or have version problems upgrading. It takes planning and coordinating on both sides to make it work when the interfaces involved are complex.

Object oriented

When a plugin design does not make sense, an easier option is to use object oriented programming APIs and build an ad-hoc plugin system using inheritance and polymorphism. Instead of copying code, refactor so the customizations are separated in a subclass. But object oriented programming by itself is inadequate to make easily customizable systems. It also needs a factory pattern to separate the use of an instance from it's concrete type to allow the type to be customized. But this pattern breaks as soon as the second plugin you install into one application needs to customize the same class as it does not offer a way to chain classes without the second plugin being explicitly based on awareness of the first.

Inversion of control

In the Java world, the most common and powerful factory pattern is IOC - inversion of control - as implemented by Guice and Spring. This pattern separates the configuration of the instance of the class from the code that uses it. Through the configuration (either annotations, properties, XML files or markup) the IOC container builds a graph of component instances, and wires everything together.

To customize with an IOC container, only the configuration is copied and updated. As long as the published APIs are not changed (which includes any APIs used in the configuration) the application won't break. But during an upgrade, the original configuration that was copied may have itself changed and include some important new features. There's no great way to ensure this happens and that nothing was missed.

That's a lot to think about on each code change so it's no wonder that rigorous O/O design with IOC component configuration does not work well when the major corporate goal is "time to market". It takes careful planning for customization hooks and as business requirements change, it's easy to break the API contracts and thus all of those configuration copies that were made. During the design, it sometimes happens that properties are exposed to the configuration layer that do not require customization, or worse hard-coded values that do need configuration. The developers of those components must take care with each change to maintain compatibility for old configuration files or upgrades are not seamless. It's in particular awkward to add new features that require new configuration. The developer must modify their copy to add configuration required by the new feature all in lock-step.

There's another problem with inversion of control when used in plugin-frameworks. When one plugin replaces a class with its own customized implementation, it will not work properly with another that does the same thing.

StrataCode layers

StrataCode provides a new option that lets developers build systems rapidly, with plain classes where code and configuration are in the same place. Instances are defined just like Java classes but using the 'object' operator instead of the class operator. Objects can be nested and even form component graphs when using the @Component annotation. StrataCode's code-generation will convert this all to a nice readable, debuggable Java file via incremental code-transformations.

The first time a property needs to be customized, it can be split out into a separate layer without changing any API contracts. This works for object instances or field initializers in a subclass.

With layers, it's possible to define a plug-in interface just by choosing the types, properties and methods to expose through modification. Layers complement the standard object-oriented design patterns that can be used with code, to move fields, methods up the stack without affecting code down the stack.

When clients are given the ability to modify a type, they can redefine the behavior of that component - including inserting new children or even replacing a child component. Plugin developers see a static-typed set of APIs that can use, override and extend with much less work on your part.

There's less planning for future customizations during design and fewer design decisions about what's in code versus what's in configuration. All code and these configuration layers are traceable in the IDE for easy find and replace. After each code change, tools quickly validate all layers that use that code to detect structural integrity problems without running the code.

With layers, knowing how easy it is to customize code later, write code that's simpler up front, and easier to read and modify.

Assembling component graphs

One of the reasons developers use an IOC framework is to configure component graphs. Sometimes references between objects create cycles or upstream reference - where a component references one that points back to it. Java's field initialization cannot express that requirement and provides no easy workaround. With StrataCode, just add the @Component annotation to the class (or a sub-class) to indicate that it may have cyclic references. It modifies the source code for this class to use a multi-stage initialization sequence. Lots of classes will work after adding the annotation but sometimes code must be moved from the constructor into the init or start methods.

Runtime or compile time configuration?

Most IOC systems have some runtime overhead - parsing and applying the configuration every time you start the application. With StrataCode, use dynamic layers for dynamic configuration, and compiled layers to avoid the overhead, both driven by the same source code. This provides several benefits at once: tunable runtime-efficiency, coding-time agility, and deployment flexibility. Overall, because the designs are simpler, and customizations implemented as part of the language, not with an external configuration or plugin system, there's less code that's more easily navigated in the IDE, or developer tools.

Arbitrary customization

Let's quickly return to the scenario where customers need rapidly customized applications and fast time to market. With StrataCode layers, build simple domain models and wire them together with statically typed references as normal fields. As customers need custom changes, augment features without copying code. Each customer with customizations can have one or more layers unique to them. As new customers have overlapping requirements, these layers can be reused or further customized. Refactor the code as requirements come in to minimize the total code footprint, all the time tracking customizations from specific customer references and requirements. Never say "no" to new custom features and never copy code unnecessarily to support two variants of the same feature. Best of all, it's a system that supports incremental designs with traceable references for manageability.