Web framework topics:

How clientServer mode works

In clientServer mode, the UI and model layers run on both the client and the server. The initial HTML for the page is rendered on the server and displayed in the browser. The browser then downloads and runs the JS files. The browser may re-render the page the first time or the tag objects will map directly onto the DOM objects generated in the server.

When the user changes the input tag's value, the binding runs on the client to update the 'message' property of it's version of the HelloWorld instance. That change is sync'd to the server in a JSON request. That JSON request is applied on the server and changes the message property that is bound to an expression that changes the reply property. That change is sync'd back to the client in the response JSON. This exchange happens all in one XHR request. The logs in both client and server consoles show the exchange using the -vs option.

How server-only mode works

In server only mode, no application code runs in the browser. Instead, one small .js file manages the back and forth communication with the server supporting real time and most of the schtml features using the same protocol. The initial page response contains the server-rendered HTML and some embedded data that defines the list of 'server tags' - those which trigger events the server needs. On each sync the browser makes to the server, this list of server tags is updated if necessary along with the HTML of any changed tags.

For the simpleSync example, the input tag is a server tag. When it's value changes, the change is sent the server using an XHR request and applied to the server-side instance for that input tag for that window. That change triggers bindings which causes the innerHTML of the div tag to be returned in the response.

Top level pages versus components

When a .schtml file starts with <html> it's treated as a top-level page and assigned a URL unless it's marked with abstract='true'. Other .schtml files generate a type in the system that can be used with extends inside of another template or used from code.

URLs, query parameters and mapping to properties of templates

At build time, the code generator collects a list of URLs defined by pages in the system. The @URL annotation identifies a page type and controls it's URL mapping. Set @URL(pattern="..") where the pattern can be a static string (see the UnitConverter example), or can be mapped to properties in the page object. See the Pattern and the Pattern language for the list of property types. Each type defines both how properties are parsed and formatted using Parselets, but also using a hard-coded implementation that runs in the browser. For example, use this @URL pattern to set an integer blogId and a String property called postShortName that must be a valid identifier:

   @URL(pattern="/blog/[{blogId=integerLiteral}/][{postShortName=identifier}/]")

Requested like:

  /blog/3/myPost

Use the @QueryParam annotation on a top-level page property to map it to a query parameter in the URL.

See the simpleBlog example for use of @QueryParam and @URL based parameters using patterns. It's possible to set properties of the template page and use them in the URL and vice versa, in a declarative way that works with the back button and bookmarks to navigate you to the right screen even on stateful pages.

When properties are changed, the javascript history object is used to change the location bar to reflect the new state so bookmarks will restore the new state of those properties. The History api can be used on either the client or the server from Java to add a new history entry, or go back to a previous one.

If the .schtml page does not include an @URL annotation, but the top level tag is 'html', it will automatically be mapped to a URL using the default framework layers. This works as follows. The html tag is mapped to the class HtmlPage which has the @URL annotation with the "subTypesOnly" value, meaning that it will not have a page on its own, but that sub types will be exposed as URLs. Because the @URL annotation on HtmlPage does not specify a pattern attribute, the relative path name (or "templatePathName") of the file with the .html suffix is used (or as configured by Template.resultSuffix).

So PageName.schtml is access via PageName.html. The layer's configuration sets or inherits the path prefix to prepend for files in it's directory. By default, files are accessed from their path relative to the layer's directory. So .schtml files in the layer directory are accessed as /fileName.html.

Testing web pages

The @URL annotation also specifies a list of testURLs that are used in the TestPageLoader as part of the -tv/-tw options to scc. It's easy to validate that all pages load successfully without errors and capture the HTML, javascript console, and server log all rendered in a "testMode" that removes dates and more. When messages contain non-repeatable data, the sctest.logFilter layer is used to remove or replace those lines so that it's easy to generate verifiable tests based on readable log output.

The -tv/-tw options also will look for a test script next to each schtml file that is named "testPageName.scr". If that file exists, the TestPageLoader will open the page using a scopeContextName that refers to the page URL, then run the test script in the context of that scope context name. Commands in that script can be run against the client, server, or both for clientServer mode pages setting the cmd.targetRuntime property. The same test scripts typically work (or can be made to work) in both serverOnly and clientServer mode without change making it much easier to test both modes in an automated way. Because .scr files are editable in the IDE, it's easy to build them and maintain them as the code changes.

Reactive, procedural, and content tags

Tags in an schtml page are converted into code in one of three ways. Reactive tags are turned into tag objects using binding expressions to update the start tag text and/or the tag's body. For a tag to be reactive, it should have some dynamic content and not use procedural features of the language like an if statement, for loop or local variable state. Procedural tags are converted into an output method that takes a StringBuilder called 'out' and an optional context parameter. A tag that is purely static content will be included as a simple string in it's parent tag.

When a tag is reactive, the generated code can incrementally refresh attributes and/or the body as properties change. It does this using a multi-pass strategy of invalidating the start tag txt or body, then later refreshing the invalid tags.

When a tag is procedural, the application uses an API to refresh the tag when it changes.

To allow a tag to be reactive, use fields in the tag object, not local variables (i.e. use <%! not <% to define a variable value used in the tag body). If you need conditional logic, use the ? expression, not an if statement. Use the repeat attribute instead of a loop.

Tag objects that rely on features that require stateful behavior must be defined in a reactive context. When in a procedural context, tag objects behave a lot like a traditional JSP in the features they offer. When an application requires a highly procedural rendering process, it makes sense to break out the declarative/reactive components into separate objects and use a render method to convert those objects into HTML.

TODO: Currently if a tag object has an 'id' it's required that it be defined in a reactive context. Because id is also used for css, it means you will get an error for valid HTML that should work. It would be easy to allow id to be used in a procedural context but for now I'm using it to help call attention to the difference. If a tag has an 'id' and it gets moved into a procedurla context, it flags an error. If a tag has an id, you can refer to it using Java. It would be possible to increase the scope of uses of tag objects procedurally that we could even manage... basically keep track of the tag objects created in a method call and destroy them after the method call. Or maybe keep them around until the next call and do some type of refresh between the old and new lists generated.

Generating css using the .sccss format

StrataCode does not yet have the ideal css wrapper language. We'd like to offer full 'find-usages and refactoring' support for styles, like you can do for Java and build in sensible variables, expressions, and other features using a compatible syntax to css itself.

As a step, the current .sccss extension uses the same base template language that sits underneath .schtml files (and other formats like .scxml) but generates css instead of html. It supports the same <% %> operators as .schtml but only one class is generated with one large outputBody method.

These templates can either be evaluated in the server at build/compile time, or when the .css file is requested. In the browser, dynamic sccss files also can be updated on the fly as properties change. The model supports fully "isomorphic dynamic css" and will only improve with better integration of the concepts of layering and data binding are added to the syntax.

Here's an example:

      <%!
         String borderColor = "#777";
         // With the SC template language, you can break back into a template expression for any string using >%
         // The template language is really Java where strings are turned inside-out
         String dropShadow = %>
            border-style: solid;
            border-width: 1px;
            border-color: <%= borderColor %>;
            box-shadow: 5px 5px 2px #aaaaaa;
         <%;
      %>

      #navMenu li ul {
         <%= dropShadow %>
      }

      #navMenu li ul ul {
         <%= dropShadow %>
      }

      .toggleButtonSelected {
         border-color: <%= borderColorSelected %>; 
      }

Relative URLs

Many web frameworks require use of absolute URLs for all references within an application, but this makes it hard to build reusable modules and requires a lot of manual refactoring when things change. A URL that starts with "/" makes sense when referring to a global aspect of the application, but sometimes it makes sense to just refer to a path relative to the current file's context.

For this case, the web framework supports relative URLs to improve the reusability, and management of components. For tag attributes that refer to URLs, like href and src, an absolute URL is rendered just as it was provided. But relative URLs will be rewritten using a function called getRelURL() that works even if the source template is included or extended from a template with a different top-level URL. It also reacts to changes made to the top-level page URL if your page needs to change it based on local navigation.

TODO: We'd like to provide warnings in the IDE for URLs which cannot be resolved, offer navigation from the IDE for tag attributes and have a flag to display warnings for unresolveable URLs at build time. Detecting and avoiding broken links to internal pages is an important problem to get right.

Images, static html etc.

For management of static files - resources for a desktop app, or web files in the web root, layers merge directory trees at build time, and later if you 'refresh' the page. More flexible than the typical doc root while still offering traceability, tooling, and reliability. Apps that do not conflict in the URLs they offer can be easily merged or separated.

Flexible path management

Layer definition files provide path management features to support flexible layer file organization that maps to flexible build directory organization. Different types of files, with different path prefixes can map to different locations in the build dir.

Rather than a monolithic doc root, source assets are managed in containers appropriate for the project and developer at hand, setup by the framework developer.

For multi-variant testing, workflow, review and staging the layered structure is a perfect fit because the directory structure only contains the deltas being made.

Details of tag objects

At code-generation time, the .schtml template page is converted into a Java class which is the root of a tag object tree. Important tags in the .schtml file are converted into inner classes of their enclosing tag object's class. A tag which is pure static content is usually converted to a String inside of it's enclosing tag's outputBody method for more efficient operation.

If an id attribute is specified, it is used as the class name. If not, a default name is chosen based on the tag name unless the tag is the top-level page in which case the file name is used.

Each tag object has a default base class chosen using the tag name. For example, the input tag class extends sc.lang.html.Input class by default. If the tag uses extends, the specified class is used instead.

Configuring a tag component

When a sub tag extends a component tag, the sub tag may need to configure or override properties of the component. For the .schtml file, component tags define properties using the <%! operator and those properties are settable in the sub tag as attributes.

See this in the PersonPage example:

file: test/js/simplePerson/PersonPage.schtml
<html>
<body>
<%! 
// Template declarations - add an inner class and two fields
class Person {
   String name;
   int credits;
   Person(String n, int c) {
      name = n;
      credits = c;
   }
}

Person buyer = new Person("Franklin", 250), seller = new Person("Mario", 5800); 
%>

<h2>People</h2>
Here are the people:

<!-- Define PersonTemplate an 'abstract' template meaning it is not drawn itself.  
     It has a "data" field of type Person we can populate through attributes. -->
<div abstract="true" id="PersonTemplate">
  <%! Person data; %>
  <h3><%= data.name %></h3>
  <p id="creditsParagraph">Credits: <span><%= data.credits %></span></p>
</div>

<!-- Render the PersonTemplate tags once binding data to buyer and then again for seller -->
<div extends="PersonTemplate" data="= buyer"/>
<div extends="PersonTemplate" data="= seller"/>
 
</body>
</html>

The tag with id="PersonTemplate" defines an abstract tag class that has a 'data' property of type Person. When that tag is extended below, data is set to a Person instance.

Tags with abstract='true' are not rendered.

It's possible to define tag macros in their own file. Just like with Java, the class name is based on the file name.

Inheriting attributes and content

When one tag extends another it inherits or overrides the attributes of the super tag. By default, the body of the sub tag is merged with the inherited body using the default value of bodyMerge="merge". Set the bodyMerge attribute to append, prepend or replace the body of the inherited tag instead. The bodyMerge attribute only affects the body. Attributes will continue to be inherited and overridden with any value of bodyMerge like they do by default.

When using the default bodyMerge="merge" mode for the body, any child tags of the sub tag are merged with child tags of the super tag if they have the same id. Tags with the same id inside of the body are themselves merged and so on. Singleton tags like head, body, and html will always merge or replace the previous one so the page does not end up with more than one.

When merging tags, use the orderValue attribute to change the order of tags. The default orderValue is 0, so set orderValue to 1 for a tag to be placed after all other tags in that body, and -1 for the tag to be placed before all other tags.

To replace a tag, there are two options. Use the attribute tagMerge="replace" to replace the tag object at compile time. In this case, both the attributes and body of the downstream tag replace those of the upstream one. The value of the tagMerge cannot point to an expression since it's evaluated at compile time.

The other way to replace a tag is at runtime using the replaceWith attribute which can be set at runtime with an expression. In this case, when the vaue of replaceWith is null, the original tag is rendered. Otherwise, the tagObject pointed to by the replaceWith expression is rendered in its place.

Be careful not to extend a div tag from an input tag or another case where the tag implementation class does not match.

Setting attributes to expressions

When setting an attribute to a constant string, use standard HTML syntax: attributeName="constant value".

When setting an attribute to a variable just one time, use: attributeName="= variableName".

If the variable's value will change, to have the binding also change use a forward binding expression: attributeName=":= variableName".

Client/server tag object API

The tag object API is supported on both the server and in the browser.

It exposes the most important features of the JS "document object model", aka the DOM, to Java code. It provides bindable properties so templates can react to events such as click, mouse move, change, or resize. Other bindable properties let code bind to values like clientWidth, and clientWidth. Tags can be controlled with properties like style, visible, and more.

Here is a list of the main classes and links to the api doc:

TODO: Should these have a 'Tag' suffix in the name? Maybe DivTag, ATag would look a little better than just Div and A as class names.

It's possible to substitute a custom class for a tag name using the tagPackageList. This property of the LayeredSystem specifies a list of packages to search in order for the HtmlPage, Div, etc. classes. It's usually set by framework layers to point to a local package of templates or classes to use for each class. For example, the html.core layer calls:

  system.addTagPackageDirectory("sc.html.tag", this, 1);

to register the sc.html.tag package. The layer html.schtml extends html.core and defines a default HtmlPage.schtml template with a package name of sc.html.tag. So when that layer is included, StrataCode uses these classes as the default value for the 'extends' attribute for the <html> tag.

This class serves as a good example for how framework features are implemented and how you can adjust behavior (see HtmlPage).

Tag visibility

Set the 'visible' attribute to false to remove this tag from the parent's body. When set back to true, the parent's body is re-rendered to include the tag again. When the tag is invisible, it's useful to specify a substitute in some cases. To support this, add a child tag with the special id="alt". When the parent is invisible, the alt tag is put in it's place and vice versa. At runtime, to refer to the alt tag in code use "<parentId>alt" (e.g. fooalt if the parent tag has id="foo").

file: test/html/simpleVis/Vis.schtml
<html>
<body>
   <%! boolean vis; %>
   <div id="visElem" visible=":= vis">
      This tag is visible 
      <div id="alt">
         The tag is not visible 
      </div>
   </div>
   <input type="button" value=':= vis ? "Make invis" : "Make vis"' 
          clickEvent="=: vis = !vis"/>
</body>
</html>

Alternatively, control the tag visibility using the style attribute as in normal HTML, but using a binding to make the tag appear and disappear. With a boolean property called 'vis' use style=':= vis ? "display:none" : ""'

Another way to control the contents rendered for a tag is to use the replaceWith attribute below.

TODO: Currently when your tag is not visible, the bindings still run which means there is more overhead than necessary and care must be taken to avoid errors in the bindings in that state (like adding null tests). We should fix by activating/de-activating the bindings, or possibly creating/destroying the object.

Detecting mouse over

The hovered property is set to true when your mouse is over an element. Bind to this property to adjust styles, trigger code etc for dynamic mouse over behavior.

Dynamic styles

Set the style attribute using expressions or data binding for dynamic styles in the tag object. For example:

style=':= "background-color: " + (colorIsRed ? "red" : "blue")'

TODO: Should support a CSSStyleDeclaration property e.g. tag.style.width := panelWidth - 10;

Changing the tag's css class

Like other bindable attributes, set class to the value of a binding expression.

class=':= hovered ? "highlightClass" : ""'

DOM Events

There are properties in the tag object for each of the DOM events that are available on any element. Use reverse-only bindings to call a method when an event is fired. A reverse-only binding can also perform an assignment. These expressions can refer to the value of the Event being fired to the timestamp, currentTarget, button pressed, etc. using properties in the event.

Here is the list of event properties that are supported:

clickEvent, dblClickEvent, mouseDownEvent mouseMoveEvent, mouseOverEvent, mouseOutEvent, mouseUpEvent, keyDownEvent, keyPressEvent, keyUPEvent, submitEvent, changeEvent, focusEvent, blurEvent, and a special combo event: mouseDownMoveUp for tracking click, drag, release.

For example:

<div id="clickMe" clickEvent='=: clicked = clicked + 1'>
   Clicked on <%= clicked %> times.
</div>

Client implementation of DOM events

Tag objects running in javascript, receive DOM events in an event handler that immediately refreshes any bindings. Bindings that require tags to be refreshed, invalidate those tags and schedule a 'do later handler' that is run after all pending events have been delivered.

So in the clickMe example above, only the body of the clickMe tag is re-evaluated when the button is clicked and that happens right away without calling the server.

Server tag implementation of DOM events

A tagObject runs in 'server tag' mode when it's marked with exec='server', or the template page was compiled without the JS runtime. In either case, DOM events are sent from the client to the server. Bindings are processed there and the server tag object's content is invalidated. Once all events have been processed, any changed tags send their content back to the client where it's updated.

To implement this, when the initial page is rendered, metadata about the server tags is sent to the client along with the initial HTML. A small JS file called stags.js reads this data and adds the necessary event listeners so it can send over 'sync' messages to set the property.

In the server tag implementation of the clickMe example, the clickEvent is sent to the server as part of a 'sync' operation in response to the button being clicked. The server increments the clicked value, fires the binding which invalidates the body of the clickMe tag. After all events have been delivered, any changes tags are refreshed. The innerHTML property change event fires and the change is returned to the client where it's applied.

Repeating Tags

Set the repeat attribute of a tag to a java.util.List or array (Object[]), and the tag is repeated, once for each element. By default, the element is available as 'repeatVar', but it's more convenient usually to use repeatVarName to provide the name of a new field created with a type matching the element type. So if you provide a List<Book> and set repeatVarName="book", the new field is created with the right type.

file: example/simpleRepeat/ui/ShowBooks.schtml
<html>
   <%!
      class Author {
         String name;
         Book[] books;

         Author(String name, Book[] books) {
            this.name = name;
            this.books = books;
            for (Book book:books)
               book.author = this;
         }
      }

      class Book {
         String title;
         Author author;

         Book(String title) {
            this.title = title;
         }
      }

      Author[] authors = new Author[]{new Author("Isaac Asimov", new Book[] {new Book("I Robot"), new Book("Foundation")}),
                                      new Author("Octavia Butler", new Book[] {new Book("Adulthood Rites")}),
                                      new Author("William Gibson", new Book[] {new Book("Neuromancer")})};
   %>
   <body>
      <!-- Set 'repeat' on a div tag to repeat the div and it's contents using wrap=false - the default for repeat on a 'div' -->
      <div id="showAuthors" repeat="= authors" repeatVarName="author">
         <span><%= author.name %></span>
         <ul>
            <li id="showBooks" repeat="= author.books" repeatVarName="book"><%= book.title %></li>
         </ul>
      </div>

   </body>
</html>

The 'wrap' property controls whether the repeat tag is itself repeated. With wrap='false', the tag with the repeat attribute is itself rendered once for each element. With wrap='true', only the body of the tag is repeated.

The default value for 'wrap' depends on the tag name. For dl, and ul (which both are typically list containers), wrap defaults to true. For all other tags wrap defaults to false.

Use the class sc.util.ArrayList or send change events on the list value for the repeat tag to update as the list changes. For incremental changes made to the list, incremental changes are made to the corresponding DOM in common cases. See the EditBooks example:

file: example/simpleRepeat/ui/EditBooks.schtml
<%@
   import java.util.Arrays;
   import sc.obj.ManualGetSet;
   import sc.obj.Sync;
   import sc.obj.IObjectId;
%>
<html>
   <%!
      @Sync
      class Author implements IObjectId {
         String name;
         List<Book> books = new ArrayList<Book>();

         Author(String name, Book[] books) {
            this.name = name;
            if (books != null) {
               this.books.addAll(Arrays.asList(books));
               for (Book book:books)
                  book.author = this;
            }
         }

         public String getObjectId() {
            return name.replace(" ", "_");
         }
      }

      @Sync
      class Book implements IObjectId {
         String title;
         Author author;

         Book(String title) {
            this.title = title;
         }

         String getObjectId() {
            return title.replace(" ", "_");
         }
      }

      @Sync
      ArrayList<Author> authors = new ArrayList<Author>(Arrays.asList(
           new Author[]{new Author("Isaac Asimov", new Book[] {new Book("I Robot"), new Book("Foundation")}),
                        new Author("Octavia Butler", new Book[] {new Book("Adulthood Rites")}),
                        new Author("William Gibson", new Book[] {new Book("Neuromancer")})}));

      @Sync
      Author currentAuthor;

      @Sync
      Book currentBook;

      void changeCurrentAuthor(Author author) {
         if (author != currentAuthor) {
            this.currentBook = null;
         }
         currentAuthor = author;
      }
      void changeCurrentBook(Book book) {
         if (book != null)
            currentAuthor = book.author;
         currentBook = book;
      }
      void addBook(Book book) {
         currentAuthor.books.add(book);
         book.author = currentAuthor;
      }
      void removeBook() {
         currentAuthor.books.remove(currentBook);
         currentBook = null;
      }
      void removeAuthor() {
         authors.remove(currentAuthor);
         currentAuthor = null;
      }
   %>
   <body>
      <div id="appFrame">
         <!-- using repeatSync=true for control of the repeatTags from test scripts -->
         <dl id="showAuthors" repeat=":= authors" repeatVarName="author" repeatSync="true">
            <dt id="authorName" clickEvent="=: changeCurrentAuthor(author)" class=':= currentAuthor == author ? "selected" : ""'><%= author.name %>:</dt>
            <dd id="books">
               <span id="booksList" repeat=":= author.books" repeatVarName="book" clickEvent="=: changeCurrentBook(book)" 
                     class=':= currentBook == book ? "selected" : ""' repeatSync="true">
                  <%= book.title + (repeatIndex == author.books.size()-1 ? "" : ",") %>
               </span>
            </dd>
         </dl>
         Title: <input type="text" id="newBookTitle"/>
                <input type="button" id="addBookButton" value="Add book"
                       clickEvent="=: addBook(new Book(newBookTitle.value))"
                       clickEvent='=: newBookTitle.value = ""'
                       disabled=":= currentAuthor == null || TextUtil.length(newBookTitle.value) == 0"/>
         <p/>
         Name: <input type="text" id="newAuthorName"/>
               <input type="button" id="addAuthorButton" value="Add author"
                      clickEvent="=: authors.add(new Author(newAuthorName.value, null))"
                      clickEvent='=: newAuthorName.value = ""'
                      disabled=":= TextUtil.length(newAuthorName.value) == 0"/>
         <p/>
         <input type="button" id="removeAuthorButton" value="Remove author" clickEvent="=: removeAuthor()" disabled=":= currentAuthor == null"/>
         <input type="button" id="removeBookButton" value="Remove book" clickEvent="=: removeBook()" disabled=":= currentBook == null"/>
      </div>
   </body>
   <style>
      .selected {
         background: #aaa;
      }
   </style>
</html>

More control over the repeated tag

When different array elements need different tag objects for display, use the repeatWrapper attribute to specify an implementation of the IRepeatWrapper interface. In the createElement method, create and return a tag object for each element.

TODO: should we just look at the 'extends' or 'implements' of the repeat tag and if it implements IRepeatWrapper, we can use the definition of the tag as the repeatWrapper and avoid one extra class. We currently don't seem to use the inner tag class which gets generated in almost all of the places we use repeatWrapper.

Replacing content with other tags

Use the replaceWith attribute to specify an expression that evaluates to a template object to use in place of the current tag. If the expression evaluates to null, the current tag is rendered. If the expression evaluates to another tag object, that tag object is added to the page as content that replaces the owner tag entirely.

Document, Location, Window

These classes emulate parts of the corresponding JS apis, but allow the apis to be used on either the client or the server and even to make changes on the server that affect the client.

Set window.location.pathname to change the browser's URL from either the server or the client.

Listen to changes to window.innerWidth/innerHeight to receive notifications of the window resize.

Use window.screen.width and height to help detect mobile devices.

Use document to listen for global mouse events. See example.html.mouseMove.

The document.activeElement like the similar DOM property helps with focus navigation, and is used to save/restore the focus after an focused element is refreshed.

javadoc: Document, Location, Window. Screen.

Page focus

Call the focus() method on a tag object from either the client or the server to apply focus to that element.

Use the document.activeElement property to see or change the focused element.

Imports

Just like JSP, you can add imports for the page and annotations on the top-level class object using the <%@ annotation as the first thing in the file.

<%@ import myClass; %>

Annotations

To annotate the class corresponding to the page itself, you use:

<%@ @AnnotationName(annotationValues=...) %>

Live in-browser programming

StrataCode as a framework supports changing the code while the application is running. With the right framework support, many changes can be applied to both the server and client code at runtime without a restart of the server process or a refresh of the browser window.

Implementation details

Read about the implementation of java to javascript and tag objects.