Package sc.parser

Class Language

  • All Implemented Interfaces:
    IFileProcessor, IComponent, IStoppable
    Direct Known Subclasses:
    BaseLanguage, Calc

    public abstract class Language
    extends LayerFileComponent
    This is the abstract base class for all language grammars defined in the system (e.g. JavaLanguage, SCLanguage, HTMLLanguage, etc. It implements some features shared by all languages. For example, each language an parse a text string and produce a language element - either an entire file (when you use the default "start parselet") or for a given chunk of code represented by a public parselet (e.g. an initializer).

    In addition to parsing a language, you can also take an language model - called a SemanticNode in the parselets framework - and re-generate the code corresponding to that model. In the parselets framework, this is implemented with the "generate" methods.

    Model generation lets you reconstruct the code for objects that are programmatically constructed (e.g. by manually constructing instances for each language type, setting the properties, etc.) or parsed, or parsed and modified. Each parsed set of SemanticNodes have a parallel parse node tree hanging off of them. As you change properties in a SemanticNode, by default the parselets framework will update or regenerate any affected code made by the change. It can do this in two modes - either by invalidating the parse-nodes that are affected and later doing the generation, or it can do the generation in real time as you make property changes to the SemanticNode objects.

    For the most part, this generation comes for free when you write a grammar in Parselets that builds a SemanticNode tree. You do need to ensure you retain enough information in your SemanticNode trees so that parselets can undo what it parsed. Spacing, comments, etc. are generally not preserved in the semantic tree. Instead, parselets use special parse nodes which represent spacing. Parse nodes support a "format" phase, that runs after generation. This phase can use the data in the semantic model to add back in the necessary spacing, newlines, etc.

    • Field Detail

      • extensions

        public java.lang.String[] extensions
      • patterns

        public java.lang.String[] patterns
      • classLoader

        public java.lang.ClassLoader classLoader
      • debug

        public boolean debug
      • debugReparse

        public boolean debugReparse
      • globalParseCt

        public int globalParseCt
      • globalReparseCt

        public int globalReparseCt
      • initialized

        public boolean initialized
      • trackChanges

        public boolean trackChanges
        The generation scheme works in two modes: when trackTranges=true, each property change automatically updates the generated result. When it is false, we invalidate changed nodes and only revalidate them as needed. tracking changes is better for debugging but a little slower
      • debugSuccessOnly

        public boolean debugSuccessOnly
      • srcPathTypes

        public java.lang.String[] srcPathTypes
        The optional name you can set to specify which types of files this language operates on - when the extension is used by different implementations for different parts of your file tree. You can register with the layer, a set of srcPathTypes. The default of 'null' matches all src files in the layer which are not explicitly marked with a different type.
      • exportProcessing

        public boolean exportProcessing
      • pathPrefix

        public java.lang.String pathPrefix
      • useCommonBuildDir

        public boolean useCommonBuildDir
      • defaultExtension

        public java.lang.String defaultExtension
      • parseletsByName

        public java.util.Map<java.lang.String,​Parselet> parseletsByName
      • parseletsList

        public java.util.ArrayList<Parselet> parseletsList
      • languages

        public static java.util.Map<java.lang.String,​Language> languages
      • extraLanguagesByExtension

        public static java.util.Map<java.lang.String,​java.util.List<Language>> extraLanguagesByExtension
      • languageName

        public java.lang.String languageName
    • Constructor Detail

      • Language

        public Language()
      • Language

        public Language​(Layer layer)
    • Method Detail

      • parse

        public java.lang.Object parse​(java.io.File file)
      • parse

        public java.lang.Object parse​(java.lang.String fileName,
                                      boolean enablePartialValues)
      • parse

        public java.lang.Object parse​(java.lang.String fileName,
                                      Parselet parselet,
                                      boolean enablePartialValues)
      • parseIntoInstance

        public java.lang.Object parseIntoInstance​(java.lang.String string,
                                                  Parselet start,
                                                  java.lang.Object populateInst)
        Parses the string using the given parselet but instead of creating a new instance, populates the instance specified - which must match the type used to produce the grammar
      • matchString

        public boolean matchString​(java.lang.String string,
                                   Parselet start)
      • parseString

        public java.lang.Object parseString​(java.lang.String string)
      • parseString

        public java.lang.Object parseString​(java.lang.String string,
                                            boolean enablePartialValues)
      • parseString

        public java.lang.Object parseString​(java.lang.String fileName,
                                            java.lang.String string,
                                            boolean enablePartialValues)
      • parseString

        public java.lang.Object parseString​(java.lang.String fileName,
                                            java.lang.String string,
                                            Parselet start,
                                            boolean enablePartialValues)
      • parseString

        public java.lang.Object parseString​(java.lang.String string,
                                            Parselet start)
      • parse

        public java.lang.Object parse​(java.lang.String fileName,
                                      java.io.Reader reader)
      • parse

        public java.lang.Object parse​(java.io.Reader reader)
      • parse

        public java.lang.Object parse​(java.lang.String fileName,
                                      java.io.Reader reader,
                                      Parselet start,
                                      boolean enablePartialValues)
        This is the most flexible form of the parse method. It takes an input reader which supplies the input stream. It also takes a starting parselet... i.e. a node in the grammar which we should expect at the first character of the input. You can use the variant of parse which does not take a parselet to parse the default grammar.

        Parameters:
        reader - the input reader.
        start - the parselet which defines the grammar we expect in the reader
        enablePartialValues - - if true, return the longest matched grammar
        Returns:
        This method returns the parse tree generated If your grammar defines a semantic value, you can retrieve that using the @see ParseUtil.nodeToSemanticValue method passing in the return value if the parse method.
      • parse

        public java.lang.Object parse​(java.lang.String fileName,
                                      java.io.Reader reader,
                                      Parselet start,
                                      boolean enablePartialValues,
                                      boolean matchOnly,
                                      java.lang.Object toPopulateInst,
                                      int bufSize)
        This is the variant you can safely override because all calls go through here. The fileName argument may be null but helps the TemplateLanguage do its processing of the file. The matchOnly just returns a true/false if this is parseable without doing all of the work of parsing it. You can provide an instance in the rare case you want to use it as the default semantic node object for the parser to work on. The bufSize is a tunable parameter to choose the size of the buffer, useful if you are parsing a small string or something.
      • incompleteParse

        protected java.lang.Object incompleteParse​(Parser p)
      • reparse

        public java.lang.Object reparse​(IParseNode pnode,
                                        DiffContext dctx,
                                        java.lang.String newText,
                                        boolean enablePartialValues)
      • restore

        public java.lang.Object restore​(java.lang.String fileName,
                                        ISemanticNode oldModel,
                                        sc.binf.ParseInStream pIn,
                                        boolean enablePartialValues)
      • restore

        public java.lang.Object restore​(java.lang.String fileName,
                                        java.io.Reader reader,
                                        ISemanticNode oldModel,
                                        Parselet start,
                                        sc.binf.ParseInStream pIn,
                                        boolean enablePartialValues,
                                        int bufSize)
        Given a deserialized semantic node (usually a JavaModel) and an unchanged file, this method quickly restores the parseNode tree. It basically does a parse but where we know the result ahead of time and don't need to parse anything that's a string - because that's not represented in the parse node tree anyway. TODO: how do we deal with error models? Maybe not cache them at all? If so we can remove a lot of code and replace it with an assertion because should never hit the error code paths in those cases.
      • postProcessResult

        public void postProcessResult​(java.lang.Object res,
                                      java.lang.String fileName)
      • postProcessSemanticValue

        public void postProcessSemanticValue​(java.lang.Object semValue,
                                             java.lang.String fileName)
        Hook to do language specific processing on the semantic values returned from this method
      • parse

        public java.lang.Object parse​(java.io.Reader reader,
                                      Parselet start)
      • style

        public java.lang.Object style​(java.lang.String input)
      • styleLayer

        public java.lang.Object styleLayer​(java.lang.String input)
      • styleNoTypeErrors

        public java.lang.Object styleNoTypeErrors​(java.lang.String layerName,
                                                  java.lang.String fileName,
                                                  java.lang.String input)
      • styleNoTypeErrors

        public java.lang.Object styleNoTypeErrors​(java.lang.String input)
      • style

        public java.lang.Object style​(java.lang.String layerName,
                                      java.lang.String fileName,
                                      java.lang.String input)
        Uses a normalized file name.
      • styleDemoFile

        public java.lang.Object styleDemoFile​(java.lang.String layerName,
                                              java.lang.String dispLayerName,
                                              java.lang.String fileName,
                                              java.lang.String input)
        Uses one file to parse, a separate to display so we can use the samples to build a new sample interactive sample
      • styleFileNoTypeErrors

        public java.lang.Object styleFileNoTypeErrors​(java.lang.String layerName,
                                                      java.lang.String fileName,
                                                      boolean isLayer)
      • styleFileNoTypeErrors

        public java.lang.Object styleFileNoTypeErrors​(java.lang.String layerName,
                                                      java.lang.String fileName)
        Uses a normalized file name.
      • styleFileNoTypeErrors

        public java.lang.Object styleFileNoTypeErrors​(java.lang.String fileName)
        Uses a normalized file name.
      • styleFile

        public java.lang.Object styleFile​(java.lang.String layerName,
                                          java.lang.String fileName)
        Uses a normalized file name.
      • styleFile

        public java.lang.Object styleFile​(java.lang.String fileName)
        Takes a normalized file name - / instead of the OS dependent path
      • styleFile

        public java.lang.Object styleFile​(java.lang.String layerName,
                                          java.lang.String fileName,
                                          boolean displayError,
                                          boolean isLayer)
      • styleBuildFile

        public java.lang.Object styleBuildFile​(java.lang.String layerName,
                                               java.lang.String buildPathPrefix,
                                               java.lang.String fileName,
                                               boolean displayError,
                                               boolean isLayer,
                                               boolean layerEnabled)
        Styles a file in the build directory. If layerName is specified, the file name is taken from the layer's build directory. The buildPathPrefix specifies a prefix to add onto the file which is not displayed in the file's path in the UI. fileName is also added onto the path and is displayed as the file name in the display of the file.
      • styleFile

        public java.lang.Object styleFile​(java.lang.String layerName,
                                          java.lang.String fileName,
                                          boolean displayError,
                                          boolean isLayer,
                                          boolean layerEnabled)
      • generate

        public java.lang.Object generate​(java.lang.Object model,
                                         boolean finalGenerate)
        Generates a textual representation from the model that could be later parsed. This takes the semantic value from the result and generates a textual description from it.
        Parameters:
        model -
        finalGenerate -
        Returns:
        the ParseNode or an error if the textual representation cannot be generated.
      • saveSemanticValue

        public void saveSemanticValue​(java.lang.Object model,
                                      java.io.File file)
      • setStartParselet

        public void setStartParselet​(Parselet sp)
      • getStartParselet

        public Parselet getStartParselet()
      • setSemanticValueClassPath

        public void setSemanticValueClassPath​(java.lang.String pkg)
        This is the package for this language's model classes.
        Parameters:
        pkg -
      • addToSemanticValueClassPath

        public void addToSemanticValueClassPath​(java.lang.String newPkg)
      • lookupModelClass

        public java.lang.Class lookupModelClass​(java.lang.String className)
      • getJavaFileName

        public java.lang.String getJavaFileName​(java.lang.String fileName)
      • registerLanguage

        public static void registerLanguage​(Language l,
                                            java.lang.String extension)
      • removeLanguage

        public static void removeLanguage​(java.lang.String extension)
      • getLanguageByExtension

        public static Language getLanguageByExtension​(java.lang.String type)
      • isParseable

        public static boolean isParseable​(java.lang.String fileName)
      • getLanguageExtensions

        public static java.util.Collection<java.lang.String> getLanguageExtensions()
      • findSrcFile

        public static java.io.File findSrcFile​(java.lang.String dir,
                                               java.lang.String srcName)
      • process

        public java.lang.Object process​(SrcEntry file,
                                        boolean enablePartialValues)
        Used by the LayeredSystem from the IFileProcessor interface to parse the file.
      • getNeedsCompile

        public boolean getNeedsCompile()
      • getProducesTypes

        public boolean getProducesTypes()
        By default, this language produces objects that go into the type system
      • getLayerPosition

        public int getLayerPosition()
        Description copied from interface: IFileProcessor
        Returns the layer position used to sort this entry to resolve conflicts when more than one is registered for a suffix
      • setDefinedInLayer

        public void setDefinedInLayer​(Layer l)
      • getDefinedInLayer

        public Layer getDefinedInLayer()
      • cleanupLanguages

        public static void cleanupLanguages()
      • getBuildPhase

        public BuildPhase getBuildPhase()
        Description copied from interface: IFileProcessor
        Returns the phase in which this processor's files get processed
      • getPrependLayerPackage

        public boolean getPrependLayerPackage()
        If true, returns the layer package
      • getUseCommonBuildDir

        public boolean getUseCommonBuildDir()
        Description copied from interface: IFileProcessor
        Returns true if the commonBuildDir (i.e. one per system) is used instead of the regular buildDir (one per build layer)
      • initialize

        public void initialize()
      • convertNameToKey

        public static java.lang.String convertNameToKey​(java.lang.String name)
      • getParselet

        public Parselet getParselet​(java.lang.String name)
      • getParseletById

        public Parselet getParseletById​(int id)
      • newSemanticContext

        public SemanticContext newSemanticContext​(Parselet parselet,
                                                  java.lang.Object semanticValue)
        Hook for languages to store and manage state used to help guide the parsing process. For example, in HTML keeping track of the current tag name stack so the grammar can properly assemble the tag-tree.
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • resetBuild

        public void resetBuild()
      • isParsed

        public boolean isParsed()
        Description copied from interface: IFileProcessor
        Returns true for Language implementations that are parseable and processable - TODO: rename to isProcessed() or isTransformed(). We have scr files that can be parsed but are not transformed
      • getSrcPathTypes

        public java.lang.String[] getSrcPathTypes()
      • init

        public void init()
        Description copied from interface: IComponent
        You can optionally implement this method to receive a initialization hook
        Specified by:
        init in interface IComponent
        Overrides:
        init in class LayerComponent
      • getCustomizedLanguages

        public java.util.List<Language> getCustomizedLanguages()