StrataCode Data Binding

It's relatively common for user-interface frameworks built these days to implement some form of reactive, declarative programming commonly called data binding. Data binding is an event-driven software pattern that relates properties via expressions, automatically updating properties when other properties change. Simple rules between properties can be enforced via these simple expressions. To implement more complex relationships, it's recommended to implement one or more methods that are perhaps fired by property changes.

StrataCode's implementation of data binding provides a clean, continuous way to make code more declarative, without adding many new concepts, and forcing a declarative style on the application with no easy exit. It's data binding system is based on Java expressions making the code readable and easier to move code in or out of a data binding expression. Additionally, invoke methods from binding events or use a method call as part of a data binding expression. The resulting style of programming integrates data binding as a natural way that scales as code-complexity increases.

Data binding expressions are converted into static method calls on the Bind class during the transformation from StrataCode to Java. One of the challenges of data binding frameworks is making complex expressions efficient and easy to debug so this design focuses on those goals. You can set breakpoints in any get/set methods. Quickly see the listeners on any property. Set tracing overall or on specific properties. StrataCode's IntelliJ plugin makes it easy to switch from StrataCode to the generated Java and back again.

There are three types of bindings:

:= forward
=: reverse
:=: bidirectional

Here's a simple example which uses:

A forward binding to set the "submitEnabled" property when there's a non-empty entry
A reverse binding to call FormManger.submitForm(entry) method when the submitEvent changes

    class UserForm {
          String entry;
          boolean submitEnabled := entry != null && entry.length() > 0;
          SubmitEvent submitEvent;
          submitEvent =: FormManager.submitEntry(entry);
    }

Here's an example using schtml which includes a bi-directional binding:

     <form submitEvent="=: addEntry()">
           <input type="text" value=":=: entryText" disabled=":= entryText.length() == 0">
     </form>

Excel-like formula for Java properties

Excel provides a grid of cells that specify either a plain value, or a formula involving live expressions. StrataCode data binding adds the live expression capability to components with properties. The result is a quick way to convert normal Java expressions to binding expressions that can react and make an application more declarative. Layers allow configuration or binding expressions to be moved downstream, customized for a different developer or replaced to create a new version.

Data binding expressions can use most any Java expression where changes are detected for properties, or arguments to methods, conditionals, etc. When necessary, the code generation process injects 'send event' calls into the setX methods for properties used. A warning is issued if some part of the expression does not send necessary events or not marked as bindable via annotations. You can use data binding with existing Java libraries although sometimes you need to write wrapper classes or add these sendEvent calls to get reactive behavior. Annotations help guide the generated code when necessary.

StrataCode's data binding system is based on experience with building large systems with data binding techniques. To benefit the most from data binding, we've found that you can effectively expose a rich, easily configured layer of features and functionality in binding rules to cover a large share of the business rules. For more complex rules, use intermediate properties to simplify an expression, or evaluate properties in a method figured when other properties change when logic requires "step-by-step" control. These techniques help encapsulate logic, rules, making code easy to read, and applications more configurable.

With StrataCode layers, technical users can collaborate in multiple ways with developers. No longer do developers have to design in customizability and managability up front or to make structural changes to add it later. Instead, they can develop the cleanest, simplest code, knowing customization later will be an easy cut-and-paste, or just let downstream layers override their defaults. And building customization apis is much easier because these layering and editing features are built-in to the runtime, IDE, and management UIs. To create a "customization layer", a developer can cut and paste the relevant code into a new file using static typing. It can be at that point handed off to another team - operations, marketing, etc.

In some situations, more than one developer needs to simultaneously work on code in the same file, because types can combine different aspects of the same feature. StrataCode helps developer workflows by making it straightforward to separate these aspects into separate files with a well-defined contract, assembling these pieces in the generated code. Comments, order of items in the file, are maintained making the generated code easy to read and debug. Developers maintain the same level of control over the assets they need and offload aspects to others.

Examples of Data Binding

Here are some basic examples showing how data binding in StrataCode works. A forward data binding expressions can be any Java expression attached to the ":=" assignment operator:

class MyClass {
  int a := 2 * b;
  int b = 3;
}

Changes are propagated along the equals sign in the direction of the colon(:) in an assignment. With The := operator, the left side updates when the right side changes. So in the example above, when b is changed, a is set to 2 times b.

There are two other data binding operators. With =: the right side of the Java expressions is updated or evaluated when the left hand side changes. With :=: the change goes both ways. In this case, the expression must have an inverse expression which StrataCode can evaluate when the left hand side changes or an error is generated.

Any Java expression can be used on the right side of a := or =: expression. For :=, events are automatically propagated through expressions which invalidate the binding when passed as method arguments, used in arithmetic, conditional, and ternary (aka question-mark expressions).

The =: bindings are StrataCode's answer to replacing closures with more traceable property change events. Code stays in the type system to improve debugging. When the property on the left hand side changes, the expression on the right hand side is executed. That event can be delivered right away, or asynchronously so it supports a method call via RPC. For 'swing', binding events are run right away unless the current thread is not the UI thread in which case they are run later by the UI thread. Frameworks have flexibility to code-generate meta-data as well as controlling the state using a few thread-local variables to manage the context.

When a binding is performed on a method which has side-effects, i.e. it's return value changes even though it's parameters did not change, or perhaps a setX method does not fire a changed event, bindings can become stale. Data binding has a refresh operator to validate and update a binding on-demand. There's an API to refresh the binding for a particular property, or an object tree, or all bindings. In the web framework, set 'refreshBindings' on the html page tag object for all bindings in that page to be refreshed after UI events are delivered, and after client/server changes have been applied.

Data binding

StrataCode supports data binding using Java expressions on a type's properties: typically fields or JavaBean style getX/setX methods.

Data binding simple examples

Foo {
  int b = 3;
  int a := b;
}

A's value is set whenever b's value changes. A is initialized to b's initial value.

Foo {
   int b;
   int a = 3;
   a =: b;    // sets b to 3 on init and b tracks changes from a
}

In this example, b's value is set whenever a's value changes, b is initialized to a's initial value.

Foo {
   b = 3;
   a :=: b;
}

If a changes, b is set. If b changes, a is set. A is initialized to b's initial value. Methods and array operations are supported as well:

class Foo {
  int[] c = {1,2,3};
  int i = 0;
  int d :=: c[i];   // d is initialized to 1 and tracks changes
                    // to either i or c.  if d is changed, it
                    // updates the appropriate element of c.
}

For expressions which invoke a method, bi-directional bindings are supported only if the method declares a BindSettings annotation with the reverseMethod attribute set. see the getting started for more details on that.

But reverse-only binding expression to method calls are allowed. In this case, the binding has nothing to do with defining the value of the property. Instead the method is called when the property is changed. So in this case, you can even pass the value of the property to the method asa parameter to implement a simple property change handler:

// Calls the method changeHandler whenever d changes with the
// current value of d.
int d =: changeHandler(d);  

public void changeHandler(int currentD) {
   System.out.println("*** d changed: " + currentD);
}

Implement raised/lowered border swap based on the selected state of the JToggleButton:

class BevelToggle extends JToggleButton {
   border := selected ? new SoftBevelBorder(BevelBorder.LOWERED) : new SoftBevelBorder(BevelBorder.RAISED);
}

Removing or replacing bindings

One of the challenges with programming with rules is turning off the old rule when you dispose the object. StrataCode stores bindings with the objects they belong to. When you dispose of the objects using a utility method provided, the bindings that object defines goes away as well.

Another challenge is removing an old rule when you want to customize it with a new rule. StrataCode makes this easy as well. When the binding expression is either := or :=: (i.e. it has some "forward" component), there is a single binding expression that is "owned" by the field. If you define a new forward binding on the same field in a subclass or sub layer, it replaces the previous forward binding expression on that field.

That rule ensures only one rule defines the value for any given property. But it is very convenient to have multiple reverse-only bindings for the same property so this rule does not apply to them. You can use multiple =: definitions on the same property and all of them will take effect. The only way to remove a reverse only definition is using the API: Bind.removePropertyBindings(Object obj, String propertyName).

When an object is disposed by StrataCode (DynUtil.dispose(Object obj)), it's bindings are automatically removed on all fields. If you discard an instance you created explicitly in code, and it uses data binding expressions, you'll need to call Bind.removeBindings(object) or better DynUtil.dispose(object) to also remove it from the dynamic typing system if that is active and to remove all bindings on child objects as well. Otherwise, the listeners added by those binding expressions won't be removed and the instances methods will still fire. Eventually all of the information will be garbage collected and the bindings go away at that point but it's best to dispose of instances you create to stop those bindings as soon as possible. When you use the -vb option, you see all binding events that are delivered so it's a quick way to validate if there are any binding leaks.

StrataCode object instances created through the frameworks are disposed automatically. For example, session scoped components will be have their bindings removed when the session expires, or elements of a 'repeatTag' are disposed when they are removed.

Sending events

Data binding rules run when properties change. At the runtime level, there needs to be a "sendEvent" call made, usually in the setX method to signal a property change. You have a few choices on how to add these events.

When StrataCode compiles this code, it injects the necessary "sendEvent" call automatically only when properties are used in a binding expression. If you bind to a field, that field is turned into getX/setX methods and all references to that field are converted automatically as needed.
If you mark a property with the @Bindable annotation, StrataCode will add the sendEvent call even if that property is not used in binding expressions. You will need to do this if you use StrataCode to generate compiled libraries where users do not have the source.
You can send events manually in code to trigger events. If you do, you should mark the property the annotation @Binding(manual=true) to tell StrataCode the property is bindable. Otherwise, it will either add the binding logic again itself (for code it compiles) or give you a warning (for compiled code).

Performance and Scalability

To implement this pattern most effectively StrataCode uses code injection and compile-time validation of bindings to make the most powerful, bindable Java expressions available. StrataCode detects which expressions cannot be made bindable and warns you at compile time. You can control the warnings through easy annotations. It's easy to create bindable libraries of objects and methods to provide even more power in your dynamic relations including objects which trigger events to invalidate all methods on the instance, method pairs that implement complex bi-directional relationships. When an expression is not bindable via event firing, you can mark bindings or objects to be refreshed - where the bindings are validated and only fired if their values have changed. This is not as low-latency and scalable as event firing but a nice fallback to code injection. In fact, many modern reactive frameworks only support this model and do hand waving at complex applications that use bindings (i.e. more than 10K bindings).

In StrataCode a data binding is simply an instance of a 'binding' created when that property is initialized. The binding registers listeners for any properties that might affect it's state. For forward bindings, each property only has one binding at a time. If you add another forward binding for the same property, it replaces the old one. The process of sending an event and firing a binding allocates no memory.

Two step evaluation, eventual consistency

For efficiency in larger binding graphs, each change first invalidates any bindings that are still valid. After the change has completed, it validates all invalid bindings and then updates the result. This reduces the chance a method is called with properties in an inconsistent state. Ultimately though, data binding uses an "eventually consistent" model so it's important to account for some patterns of inconsistency in getX/setX and methods triggered from bindings.

Event queuing

APIs allow framework code to queue change events to avoid firing bindings during certain contexts. With the sync system for example, all incoming changes are applied and change events are queued up. At that point, all changes are first invalidated, the all re-validated to minimize method calls and other computation used in validating the binding.

Same value check

Use @Bindable(sameValueCheck=true) on a property to cause the setX method to do an 'equal' test on the original value before sending a change. The default is to send the change event even if the setX is called with the same value. Because the binding objects caches the last value, it can often stop an unnecessary change from propagating even without the sameValueCheck=true.

Do later bindings

Use @Bindable(doLater=true) to apply the binding after the current operation completes. This is a useful way to manage UI refresh in response to some user interaction for example, or to avoid re-validating some data structure multiple times or when the system is in the midst of propagating changes and is in a inconsistent state. Use @Bindable(doLater=true,priority=10) or @Bindable(doLater=true,priority=-10) to control the order of doLaters. Higher priority doLater methods are called before lower priority ones. The web framework uses Element.REFRESHTAGPRIORITY (-5) as the 'refreshTags' priority. Use -6 or smaller to run code after the tags have been all been refreshed or the default of 0 to run before tags are refreshed.

Run bindings after a delay

Use @Bindable(delay=10) to insert an integer number of milliseconds delay from when the binding is validated and ultimately the property is set for a forward binding or for a reverse when the expression is evaluated or reverse property set.

Multi-threading

Frameworks have a flexible set of APIs and features to give application code the appearance of a safe, declarative, single-threaded environment for domain model objects. Behind the scenes, the framework code makes efficient use of multiple threads, including support for collaboration features where one client can receive change events from another.

Before delivering user-interface events, framework code sets up a CurrentScopeContext - a list of all of the available ScopeContexts for the current thread. Each context can require a read or write lock based on how the framework code allows that ScopeContext to be used. These locks are acquired on a "all or nothing basis" to avoid deadlocks.

For the brower or a desktop application, there might only be one ScopeContext for the whole process (global). For the webFramework, there are a number of ScopeContexts to manage the multi-user, multi-purpose nature of client/server request handling. Although ScopeContexts can be added and customized, by default the webFramework supports a separate ScopeContext for each important category of state that reflects the way the server is used: global (shared by all requests), per-application (shared by all requests to some named application that represents a subset of the requests), per user-session, per application-user-session, per browser window, and per request.

When these scope contexts are shared between end-users, it's possible to keep data structures synchronized enabling real-time collaboration using data binding and/or the sync framework. In essence, there will be overlapping types and layers where some subset of properties are set in one context and listened to by another. Mark these properties with @Bindable(crossStop=true).

Here's a basic description of how this is implemented. When a cross-scope binding is initialized, it's CurrentScopeContext is retrieved from thread-local state and saved as the "initialization context" for the binding. Before applying the binding, the code will check if the CurrentScopeContext matches the "initialization context". If not, it's put into a queue for that context. If there's a waiter for that context, it will wake up and process it but only after acquiring the right locks. This ensures it won't start until the current operation completessince the modifier will have a lock required to change a property in the shared component. Once the locks are available, those events are delivered either via the sync system or sent to the client on its next request.

TODO: It's possible to detect properties that need crossScope with just an extra thread-local lookup for each sendEvent, or provide an option to do this to validate if it's set properly. For now, this feature has not been used extensively so using it carefully until we have more code using it. Or just set it by default for certain frameworks that might have multi-threaded behavior but long-term it should be possible to make these collaborate scenarios manageable, thread-safe, and secure especially once we integrate with a multi-server sync service.

Debugging Problems

The first step is to set in the generated setX and getX methods. While it's possible to set breakpoints on the field, it's usually better to use the IntelliJ plugin and navigate from the source to the generated source and set the breakpoints there. If a sendEvent is supposed to fire another method, when you step over the sendEvent call it should stop in the other setX (unless queuing is enabled). If not, step into the sendEvent and you will quickly find the list of listeners.

You can enable tracing of all binding events using -vb to figure out what's happening at the fine-grained level without the debugger. It may be best to reduce the volume of output and only set @Bindable(trace=true) or @Bindable(verbose=true) on the properties you need to follow. Enable -vba, to get all setX traffic for all bindable properties.

Recursive bindings

It's possible for conflicting binding rules to be defined which generate infinite loops. StrataCode catches some error at compile time, but not all. At runtime, it has a recursion limit and generates helpful diagnostics as to the bindings involved and deactivates them to avoid further problems that can become an obstacle in debugging.

By turning on tracing, it's also easy to track down which bindings are at fault, as you trace the flow of values.

See debugging for details.