GWT

Sync Overview

The StrataCode sync framework is based on a powerful and flexible design but makes easy things easy as well. Some strengths of the sync architecture: See these examples: UnitConverter, TodoList, and Program Editor.

Sync Details

This section discusses how to use the sync framework and some details on how it works under the hood.

Sync Runtimes

You typically run StrataCode with a single list of layers, some of which may be limited to run only on the client, some only on the server and the rest run on both. The layer definition file specifies it's runtime requirements and the system automatically partitions the stack of layers into a set of runtimes. You can add additional servers by adding new runtimes and control how objects are synchronized to those runtimes. StrataCode automatically builds and starts all processes needed to run the resulting application.

SyncMode

The synchronization framework implements client/server communication in an automatic or directed way, based on how you configure your frameworks. The @Sync(syncMode) annotation enables or disables synchronization for a given layer, type or property.

You can set the sync mode explicitly using the @Sync annotation, inherit it from the current layer, or have it computed automatically based on the which types and properties exist in more than one runtime. This gives you complete control over which objects and properties are synchronized without having to specify @Sync on every class just by how you structure your code in layers.

When you do need to configure the Sync mode manually use @Sync:

  // Turn off synchronization 
  @Sync(syncMode=SyncMode.Disabled)
  public class MyClass ...

The syncMode may be one of:

To override the type-level syncMode for any given property, just set the @Sync annotation on that property.

A property inherits the first @Sync annotation of the first type which sets @Sync mode after it's defined in a type hierarchy. This lets you change the syncMode in the type hierarchy, e.g. turn it off for a base class and all of the properties defined in that base class, then turn it on in a sub-class so any new properties added will be synchronized. If your base classes do not set @Sync, even the base class properties will inherit the @Sync on the subclass.

If there is no @Sync tag on a type, it uses the @Sync attribute defined for the layer. If there's no @Sync tag for the layer or the type, the type inherits the @Sync annotation defined on a base-class.

By default, setting @Sync on a type will not affect inherited properties from a base class. You can change that by setting @Sync(includeSuper=true), or by adding a property assignment or just 'override propertyName' in the subclass. Whenever a type the layer of a type refers to a property, it's @Sync attribute will apply, even if the property is not defined in that type.

If no @Sync annotation is found for a property, type, or layer that property is not synchronized.

Destinations

By default you are synchronizing to the default destination - the server if you are on the client and the client if you are on the server. To sync between servers, there is a way to configure destinations. In this case, the server will generate different sync profiles for each destination and manage different sync states.

Scopes

Scopes are a feature of the StrataCode code generation system that let you manage an object's lifecycle - i.e. how it is created, how references are resolved, and when it is stopped. They let you use declarative models, using a single name space, even when the instances in that name-space are created and deleted at different times (e.g. per-request, per-user, per-session, etc), and perhaps driven by additional runtime parameters (e.g. per-merchant, per-store, per entry in a list).

Scopes make StrataCode's synchronization system very flexible and powerful.

The default scope is called "global" and is equivalent to using static variables in Java. In the web server context, the session scope manages objects which are per client browser. Request scope disposes of instances after each request for stateless applications (TODO: request scope is not implemented yet).

You can use scopes for even more dynamic lifecycles such as per-list item. In that case, you can inject additional variables for the list index and current list item.

Scopes are a flexible hinge point so frameworks can add scopes like per-product catalog, per-inventory source etc. Even templates which are written for a single catalog can then be used in a multi-catalog scenario as long as at any given time. The key constraint is the framework must be able to choose a single catalog and make it available for each template.

Scopes are a safe name-space sandbox for the current context to access all of the types and properties it needs, declaratively in a strongly-typed compile time model without worrying about lifecycle or dynamic behavior. At the same time they can implement rich dynamic behavior: organizing cells into a grid, or line-items in an order. When you change the scope of an object in a subsequent layer, they let you reuse logic in new contexts - e.g. adding multi-tenant or multiple shipping addresses as a feature by flipping a switch.

To implement a scope, framework code has the necessary hooks available so they do not require much code at all. A new scope can use thread-local state to pass parameters to code placed into the generated getX methods to lookup the right instance. A scope can add variables to the object's creation. In this case, you frequently generate a method to create a new instance passing those variables (e.g. the current list index, the current item). The framework manages creating new items, updating those properties, etc. behind the scenes so your code is simple and declarative and always reflecting the current context.

When using scopes there are two types of nesting of inner objects, just like in Java: static and instance. In this case, static instances might not be "one-per-class" like in Java but instead you can obtain a reference to one globally. In contract, inner instances are defined in the context of their parent object. In this case, scopes may just separate out one group of children from another, or might provide a way to resolve children which are not directly stored with the parent, but instead are just associated with it.

At runtime, the framework extends the scope apis to control the scope. These APIs include the registry for scopes. A method to resolve an object instance given it's type name (for a global object reference). You also can find all of the child objects of a particular parent that have a given scope.

Scopes can force the use of static inner objects so the system generates static getX methods. That lets you nest objects with scopes inside each other without fear of dependencies or data leakage - e.g. if you accidentally store a per-user value in a global property, letting other users see that property.

When you use inner instance objects, the inner object must have no scope, live in the same scope or be a nested scope of the parent. That's enforced at compile time.

Objects and classes can be assigned a scope via the @Scope annotation or the scope<scopeName> operator. Some scopes will add additional fields to the current context which you can use in your object definitions. Just as with other objects, the framework manages the create and destroy calls for you.

SyncContext

On the client, session, window, and global scopes are collapsed into one SyncContext because each browser is only managing one window and one user's session. On the server however, each scope may have its own SyncContext or use it's parent's syncContext. The global sync context is used to share data between all users on the server. Using the session sync context keeps that data private to the user.

The SyncContxt organizes changes in SyncLayers. These record changes for a given batch for each sync group. When an instance is synchronized, it is given to the sync context. The syncContext adds property change listeners on all synchronized properties. Any property values which implement IChangeable have a different listener added to them so we listen for change events on the current property value itself.

When property changes come in for a given instance, they are applied to the SyncContext according to the current syncState. This is a global, thread-local status flag that's set by the framework. It can have one of five values:

You can save and restore the SyncState via the method SyncManager.setSyncState though typically this should only be done by framework code.

The SyncLayer records property changes as well as the creation of new objects. It keeps all of the changes in order and detects when a change has been overridden and omits that change automatically.

IChangeable

The StrataCode data binding system supports the IChangeable interface for objects to issue events generated programmatically. You use IChangeable both when you define an object like sc.util.ArrayList which updates by value or an object which updates all of its properties at once. When StrataCode detects an IChangeable on object "a" in an "a.b" reference chain, a listener is added to update that expression when object a calls Bind.sendChangedEvent(a,null). If the value of "b" is IChangeable the binding will also update when it's value is fired.

The synchronization system similarly listens on IChangeable in both ways. When the default event fires, all properties of that object are "refreshed". If their value has changed, they are synchronized. If the default event fires on a property value, the owner property is refreshed.

Cloneable

Property types that are synchronized should be cloneable using Java's normal clone method. An initial copy is made and stored so the sync system can revert changes and also reconize the deltas in what has changed to send diffs in a more concise format.

SyncManager

Each SyncManager manages the sync state for a given destination. The client only has one SyncManager, the server will have one for all clients, and one for each server to server connection.

The SyncManager stores the list of sync types. Each sync type has a list of sync properties.

The SyncManager also manages adding of synchronized instances. As instances are added to the sync manager, it is either given or finds the right scope for that instance, and places the instance under control of the proper SyncContext.

Synchronization Format

The SyncLayer serializes itself into a stream using the StrataCode language. The client sends down a layer of changes, the server responds with its own layer of changes.

This format uses a package declaration, modify definitions, property assignments, field definitions for new objects when there are constructor args, otherwise object statements.

Frameworks can customize the handling of a particular instance type by registering a SyncHandler for an instance. The SyncHandler can substitute a different instance or override how the value is turned into a string in the language format. The SyncHandler can generate code to add or remove an element from a list.

This format makes it easy to read the changes going over the wire. When you enable trace and verbose on the SyncManager you can easily detect properties getting synchronized that should not or vice versa. You also have diagnostics up front as which types and properties are synchronized and with what options.

When the server has changes to send to the client, it first puts them into the same serialization format, then parses that format into a JavaModel, then converts that into JS code to merge the changes into the JS runtime. This produces a format which is very efficient for the client to apply. It can simply eval the resulting code, preventing parsing of the code in Javascript itself. It's also nice for debugging because you can set breakpoints in the generated code while it's being applied. The downside is that the parsing and formatting of the JS code is not fast for large data sets. The format generated by synchronization can be parsed in JS almost as quickly as JSON or soon a binary version that will be easier to maintain and faster at runtime.

When your updates include code changes, as detected by the StrataCode refresh process, those are processed with the same system.

Sync Groups

By default all properties on all types are updated via the same sync operation. In rare cases, you might want to update some properties without updating others. In these cases, you use:

  @Sync(groupName="highPriority")
  class HighPriorityType {

  }

At the API level, you call SyncManager.sendSync("highPriority") explicitly to sync just the properties in that group.

Traceable Security

Though it may seem like the system is essentially "open", with free exchange between client and server ultimately StrataCode will produce much more secure applications than average. That's because all client/server communication will be based on traceable, typed interfaces. Being able to quickly audit and incrementally change the exposed interfaces from client to server will ultimately produce stronger and more secure applications than those which build in security policy by forcing developers to write more code.

For security, the system ensures that only types which are defined to be synchronized in the source code can be accessed or manipulated via the sync protocol (TODO: this is not yet implemented!). By default, this includes only model code which exists on the client anyway, typically the data model manipulated by the user interface. If you are careful to only put sensitive code in server layers, the system will be secure. Sensitive code tends to be code which forms database queries or manipulates other secure server resources. This code requires dependencies on server code and will always run in server-specific layers. That will ensure it is not run on the client and not accessible to the client.

Initial Layer

The SyncContext maintains an initial layer on the server which represents the initial state of the objects created on the client. Any changes made are stored. When the client refreshes the page, the client's objects are restored to the initial state but any changes made to the initial layer since then are applied.

This lets the synchronization mechanism retain all application state - even transient user inteface state that is not stored in the database. That state is refreshed as part of the initial state of the application automatically.

Failover

When the server fails, the sync command won't find a session which matches this client. In that situation, the client syncs its data to the server to restore any state it has. By keeping the state for the application on both the client and the server at the same time in sync, your application gets fault tolerance for free. If the browser refreshes, even fine-grained application state is restored. If the server session goes away, you pay a small one time cost to restore the session from the client. Zero code overhead. Because the client and server stay in sync, they can exchange minimal information, decreasing latency and server processing overhead.