Package sc.parser
Class Parselet
- java.lang.Object
-
- sc.parser.Parselet
-
- All Implemented Interfaces:
java.lang.Cloneable
,ILifecycle
,IParserConstants
- Direct Known Subclasses:
NestedParselet
,Symbol
,SymbolChoice
public abstract class Parselet extends java.lang.Object implements java.lang.Cloneable, IParserConstants, ILifecycle
This is the abstract class for all parselets. It contains common functionality common to all parselets.
-
-
Field Summary
Fields Modifier and Type Field Description boolean
alwaysReparse
Set to true for parselets such as spacing that tend to glue together other parse nodes.static java.lang.Class
ARRAY_LIST_CLASS
int
attemptCount
boolean
cacheResults
Set to true for parselets which benefit from caching - i.e.int
failedProgressBytes
boolean
fieldNamed
int
generatedBytes
IParseNode
generateParseNode
This is a ParseNode which if set, is returned during the generation phase.int
id
boolean
ignoreCase
For Symbols only - match the symbol ignoring the caseboolean
ignoreEmptyList
boolean
initialized
Language
language
boolean
lookahead
boolean
negated
boolean
optional
boolean
partialValuesOnly
Match only in partial values mode - use for 'IncompleteStatement' - that you want in the IDE but not messing up the grammar for the compiled parse.boolean
recordTime
boolean
reparseable
Set to false for parselets where the new parse-result during the reparse operation may not be contained in the same region of the file - e.g.boolean
repeat
boolean
reportError
java.lang.Class
resultClass
The class of the result.protected java.lang.String
resultClassName
BodyTypeDeclaration
resultDynType
In the rare case a parselet is used to populate a dynamic type, this stores that dyn type.boolean
semanticValueMultiTypedArray
boolean
skip
boolean
skipOnError
boolean
started
java.lang.String
styleName
Assign a default style for semantic values produced by this parse node.int
successCount
boolean
trace
boolean
treatValueAsScalar
-
Fields inherited from interface sc.parser.IParserConstants
EOF, IGNORE_CASE, LOOKAHEAD, NOERROR, NOT, OPTIONAL, PARTIAL_VALUES_ONLY, REPEAT, SKIP, SKIP_ON_ERROR, TRACE
-
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description protected java.lang.String
accept(SemanticContext ctx, java.lang.Object value, int startIx, int endIx)
Lets subclasses add additional acceptance criteria to a rule.protected java.lang.String
acceptSemanticValue(java.lang.Object value)
Lets subclasses accept or modify the semantic value.protected java.lang.String
acceptTree(SemanticContext ctx, java.lang.Object value, int startIx, int endIx)
Like accept but for hierarchical nodes, performs the 'accept' operation on children in the tree.boolean
addReparseResultToParent(java.lang.Object node, ParentParseNode parent, int svIndex, int childIndex, int slotIndex, Parser parser, java.lang.Object oldChildParseNode, DiffContext dctx, boolean removeExtraNodes, boolean parseArray)
boolean
addResultToParent(java.lang.Object node, ParentParseNode parent, int index, Parser parser)
protected void
advancePointer(Parser parser, java.lang.Object oldParseNode, DiffContext dctx)
protected boolean
anyReparseChanges(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean forceReparse)
void
changeParseletName(java.lang.String newName)
Changes the internal name of the parselet while preserving the semantic parametersprotected void
checkForSameAgainRegion(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean beforeMatch, boolean forceReparse)
protected void
clearChangedRegion(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean beforeMatch, boolean forceReparse)
java.lang.Object
clone()
Parselet
copy()
Parselet
copyWithOptions(int options)
boolean
dataTypeMatches(java.lang.Object other)
boolean
elementTypeMatches(java.lang.Object other)
boolean
emptyValue(GenerateContext ctx, java.lang.Object value)
The empty value for a base parselet is determined if we have an empty string, an empty list, or a "false" boolean.void
format(FormatContext ctx, IParseNode node)
The generate phase generate parse nodes in the output.void
formatStyled(FormatContext ctx, IParseNode node, IStyleAdapter adapter)
abstract java.lang.Object
generate(GenerateContext ctx, java.lang.Object semanticValue)
java.lang.Object
generateResult(GenerateContext ctx, java.lang.Object result)
IParseNode
getBeforeFirstNode(IParseNode beforeFirstNode)
boolean
getDiscard()
IParseNode[]
getFormattingParseNodes(java.util.Set<Parselet> visited)
Language
getLanguage()
boolean
getLookahead()
java.lang.String
getName()
boolean
getNegated()
boolean
getNewSemanticValueIsArray()
protected java.lang.String
getPrefixSymbol()
boolean
getReportError()
java.lang.Class
getSemanticValueClass()
java.lang.String
getSemanticValueClassName()
java.lang.Class
getSemanticValueComponentClass()
boolean
getSemanticValueIsArray()
boolean
getSemanticValueMultiTypedArray()
java.lang.Class
getSemanticValueSlotClass()
This is a hook for subclasses to override if there is any conditionality in how the slot class is used.boolean
getSkip()
protected java.lang.String
getSuffixSymbol()
boolean
getTrace()
void
init()
boolean
isComplexStringType()
From the IDE's perspective,boolean
isInitialized()
boolean
isNullValid()
For most parselets (all but EOF), null is not a real value.boolean
isProcessed()
boolean
isStarted()
boolean
isValidated()
boolean
needsChildren()
IParseNode
newGeneratedParseNode(java.lang.Object value)
IParseNode
newParseNode()
IParseNode
newParseNode(int ix)
abstract java.lang.Object
parse(Parser p)
The main parse method for the parselet.ParseError
parseEOFError(Parser parser, java.lang.Object partialValue, java.lang.String errorCode, java.lang.Object... args)
ParseError
parseError(Parser parser, java.lang.Object partialValue, ParseError chain, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
ParseError
parseError(Parser parser, java.lang.String errorCode, java.lang.Object... args)
ParseError
parseError(Parser parser, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
java.lang.Object
parseExtendedErrors(Parser p, Parselet exitParselet)
An optional method for parsing a repeating parselet.ParseError
parsePartialError(Parser parser, java.lang.Object partialValue, ParseError chain, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
java.lang.Object
parseResult(Parser parser, java.lang.Object value, boolean skipSemanticValue)
boolean
peek(Parser p)
void
process()
boolean
producesParselet(Parselet other)
boolean
producesParseletId(int otherId)
java.lang.Object
propagateResult(java.lang.Object node)
Overridden in ChainResultSequence to do post-processing that would ordinarily be done in addResultToParent for a nested sequencejava.lang.Object
reparse(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean forceReparse, Parselet exitParselet)
ParseError
reparseError(Parser parser, DiffContext dctx, java.lang.String errorCode, java.lang.Object... args)
ParseError
reparseError(Parser parser, Parselet childParselet, DiffContext dctx, java.lang.String errorCode, java.lang.Object... args)
java.lang.Object
reparseExtendedErrors(Parser p, Parselet exitParselet, java.lang.Object oldChildNode, DiffContext dctx, boolean forceReparse)
ParseError
reparsePartialError(Parser parser, DiffContext dctx, java.lang.Object partialValue, ParseError chain, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
abstract java.lang.Object
restore(Parser p, ISemanticNode oldModel, RestoreCtx ctx, boolean inherited)
The parselet method to re-create the parse node tree given the resulting model.void
restoreOldNode(IParseNode pn, ISemanticNode oldModel)
java.lang.Object
restoreResult(Parser parser, ISemanticNode oldModel, java.lang.Object value, boolean skipSemanticValue, boolean updateModelParseNode)
abstract void
saveParse(IParseNode pn, ISemanticNode oldNode, SaveParseCtx spctx)
The parselet method to save parselet ids, offsets, string-lengths etc.void
setLanguage(Language l)
void
setName(java.lang.String n)
void
setResultClass(java.lang.Class c)
void
setResultClassName(java.lang.String rcName)
boolean
setResultOnParent(java.lang.Object node, ParentParseNode parent, int index, Parser parser)
This method replicates the functionality of addResultToParent for the special case where we are parsing errors and we've reparsed the value of a node and need to replace it in it's parent before returning.void
setSemanticValueClass(java.lang.Class c)
void
start()
void
stop()
java.lang.String
toHeaderString(java.util.Map<Parselet,java.lang.Integer> visited)
boolean
typeMatches(java.lang.Object sv, boolean isElement)
void
updateCachedResult(java.lang.Object res)
This is called when we are parsing using the result cache.int
updateParseNodes(java.lang.Object semanticValue, IParseNode node)
void
validate()
-
-
-
Field Detail
-
ARRAY_LIST_CLASS
public static final java.lang.Class ARRAY_LIST_CLASS
-
id
public int id
-
trace
public boolean trace
-
language
public Language language
-
ignoreEmptyList
public boolean ignoreEmptyList
-
optional
public boolean optional
-
repeat
public boolean repeat
-
negated
public boolean negated
-
lookahead
public boolean lookahead
-
skip
public boolean skip
-
skipOnError
public boolean skipOnError
-
reportError
public boolean reportError
-
treatValueAsScalar
public boolean treatValueAsScalar
-
initialized
public boolean initialized
-
started
public boolean started
-
cacheResults
public boolean cacheResults
Set to true for parselets which benefit from caching - i.e. commonly are contained in parent regions of different types e.g. children of a tree-tag or simple-tag in an HTML element which is parsed both ways when a matching close cannot be found.
-
reparseable
public boolean reparseable
Set to false for parselets where the new parse-result during the reparse operation may not be contained in the same region of the file - e.g. a simpleTag which turns into a treeTag during a reparse. The containing anyTag parselet is marked not reparseable so we try the 'treeTag' option even during reparse where that simpleTag's contents did not change. This catches the case where say the end tag changes or some region in the middle changes. Unfortunately we will parse a lot more nodes when simpleTags are used... ideally we'd have some optimization to avoid the treeTag unless there is a possible close tag for that tree.
-
partialValuesOnly
public boolean partialValuesOnly
Match only in partial values mode - use for 'IncompleteStatement' - that you want in the IDE but not messing up the grammar for the compiled parse. Parse nodes parsed by a partialValuesOnly parselet are marked as 'error nodes' so we know to extend the region for reparsing
-
ignoreCase
public boolean ignoreCase
For Symbols only - match the symbol ignoring the case
-
attemptCount
public int attemptCount
-
successCount
public int successCount
-
generatedBytes
public int generatedBytes
-
failedProgressBytes
public int failedProgressBytes
-
recordTime
public boolean recordTime
-
resultClass
public java.lang.Class resultClass
The class of the result. Must implement IParseNode. Usually a subclass of either ParseNode for primitive values constructed from a String or ParentParseNode for complex objects.
-
resultDynType
public BodyTypeDeclaration resultDynType
In the rare case a parselet is used to populate a dynamic type, this stores that dyn type.
-
resultClassName
protected java.lang.String resultClassName
-
generateParseNode
public IParseNode generateParseNode
This is a ParseNode which if set, is returned during the generation phase. For procedurally generated elements such as spacing and newlines which need the context of the parse tree in which they are embedded to determine their behavior, you can just set this property to a fixed instance.
-
alwaysReparse
public boolean alwaysReparse
Set to true for parselets such as spacing that tend to glue together other parse nodes. Those need to be reparsed always.
-
styleName
public java.lang.String styleName
Assign a default style for semantic values produced by this parse node. Often, but not always the style for a text node is associated with it's grammar node. If you need to customize the style for a given node, override the styleNode method.
-
fieldNamed
public boolean fieldNamed
-
semanticValueMultiTypedArray
public boolean semanticValueMultiTypedArray
-
-
Method Detail
-
init
public void init()
- Specified by:
init
in interfaceILifecycle
-
setResultClassName
public void setResultClassName(java.lang.String rcName)
-
setResultClass
public void setResultClass(java.lang.Class c)
-
start
public void start()
- Specified by:
start
in interfaceILifecycle
-
stop
public void stop()
- Specified by:
stop
in interfaceILifecycle
-
isInitialized
public boolean isInitialized()
- Specified by:
isInitialized
in interfaceILifecycle
-
isStarted
public boolean isStarted()
- Specified by:
isStarted
in interfaceILifecycle
-
isValidated
public boolean isValidated()
- Specified by:
isValidated
in interfaceILifecycle
-
validate
public void validate()
- Specified by:
validate
in interfaceILifecycle
-
process
public void process()
- Specified by:
process
in interfaceILifecycle
-
isProcessed
public boolean isProcessed()
- Specified by:
isProcessed
in interfaceILifecycle
-
parseResult
public java.lang.Object parseResult(Parser parser, java.lang.Object value, boolean skipSemanticValue)
-
restoreResult
public java.lang.Object restoreResult(Parser parser, ISemanticNode oldModel, java.lang.Object value, boolean skipSemanticValue, boolean updateModelParseNode)
-
restoreOldNode
public void restoreOldNode(IParseNode pn, ISemanticNode oldModel)
-
addResultToParent
public boolean addResultToParent(java.lang.Object node, ParentParseNode parent, int index, Parser parser)
-
addReparseResultToParent
public boolean addReparseResultToParent(java.lang.Object node, ParentParseNode parent, int svIndex, int childIndex, int slotIndex, Parser parser, java.lang.Object oldChildParseNode, DiffContext dctx, boolean removeExtraNodes, boolean parseArray)
-
setResultOnParent
public boolean setResultOnParent(java.lang.Object node, ParentParseNode parent, int index, Parser parser)
This method replicates the functionality of addResultToParent for the special case where we are parsing errors and we've reparsed the value of a node and need to replace it in it's parent before returning. It only handles a few of the semantic value mappings so far... another approach would be to throw away and rebuild the parent's value entirely. The key here is that when we update the parse node for a child, we also update the semantic value so the two are consistent.
-
propagateResult
public java.lang.Object propagateResult(java.lang.Object node)
Overridden in ChainResultSequence to do post-processing that would ordinarily be done in addResultToParent for a nested sequence
-
getPrefixSymbol
protected java.lang.String getPrefixSymbol()
-
getSuffixSymbol
protected java.lang.String getSuffixSymbol()
-
getSkip
public boolean getSkip()
-
getDiscard
public boolean getDiscard()
-
getLookahead
public boolean getLookahead()
-
getNegated
public boolean getNegated()
-
getReportError
public boolean getReportError()
-
setSemanticValueClass
public void setSemanticValueClass(java.lang.Class c)
-
getSemanticValueClass
public java.lang.Class getSemanticValueClass()
-
getSemanticValueClassName
public java.lang.String getSemanticValueClassName()
-
getSemanticValueIsArray
public boolean getSemanticValueIsArray()
-
getNewSemanticValueIsArray
public boolean getNewSemanticValueIsArray()
-
getSemanticValueComponentClass
public java.lang.Class getSemanticValueComponentClass()
-
generateResult
public java.lang.Object generateResult(GenerateContext ctx, java.lang.Object result)
-
setName
public void setName(java.lang.String n)
-
getName
public java.lang.String getName()
-
changeParseletName
public void changeParseletName(java.lang.String newName)
Changes the internal name of the parselet while preserving the semantic parameters
-
getLanguage
public Language getLanguage()
-
setLanguage
public void setLanguage(Language l)
-
reparseError
public ParseError reparseError(Parser parser, Parselet childParselet, DiffContext dctx, java.lang.String errorCode, java.lang.Object... args)
-
parseError
public ParseError parseError(Parser parser, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
-
parseError
public ParseError parseError(Parser parser, java.lang.Object partialValue, ParseError chain, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
-
parsePartialError
public ParseError parsePartialError(Parser parser, java.lang.Object partialValue, ParseError chain, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
-
reparsePartialError
public ParseError reparsePartialError(Parser parser, DiffContext dctx, java.lang.Object partialValue, ParseError chain, Parselet childParselet, java.lang.String errorCode, java.lang.Object... args)
-
reparseError
public ParseError reparseError(Parser parser, DiffContext dctx, java.lang.String errorCode, java.lang.Object... args)
-
parseError
public ParseError parseError(Parser parser, java.lang.String errorCode, java.lang.Object... args)
-
parseEOFError
public ParseError parseEOFError(Parser parser, java.lang.Object partialValue, java.lang.String errorCode, java.lang.Object... args)
-
toHeaderString
public java.lang.String toHeaderString(java.util.Map<Parselet,java.lang.Integer> visited)
-
accept
protected java.lang.String accept(SemanticContext ctx, java.lang.Object value, int startIx, int endIx)
Lets subclasses add additional acceptance criteria to a rule. Returns an error string or null if all is ok. Passed the semanticContext, an optional object the language can use for keeping state during the parse process. For these languages, the startIx and endIx from the input stream are provided.
-
acceptSemanticValue
protected java.lang.String acceptSemanticValue(java.lang.Object value)
Lets subclasses accept or modify the semantic value. Returns an error string or null if all is ok.
-
acceptTree
protected java.lang.String acceptTree(SemanticContext ctx, java.lang.Object value, int startIx, int endIx)
Like accept but for hierarchical nodes, performs the 'accept' operation on children in the tree. We use this when accepting a cached or reparsed value, or generating a value. But during the parse operation, the accept is run on each node as part of the parse so we do not have to recurse to children.
-
getTrace
public boolean getTrace()
-
newParseNode
public IParseNode newParseNode()
-
newGeneratedParseNode
public IParseNode newGeneratedParseNode(java.lang.Object value)
-
newParseNode
public IParseNode newParseNode(int ix)
-
needsChildren
public boolean needsChildren()
-
emptyValue
public boolean emptyValue(GenerateContext ctx, java.lang.Object value)
The empty value for a base parselet is determined if we have an empty string, an empty list, or a "false" boolean. The last one occurs because we turn a null value into a false and a non-null into a true in the semantic model. So if we see a false come through it means there is no parsed value for that slot. For an empty list, there are some cases where we need to treat an empty
-
clone
public java.lang.Object clone()
- Overrides:
clone
in classjava.lang.Object
-
copy
public Parselet copy()
-
copyWithOptions
public Parselet copyWithOptions(int options)
-
format
public void format(FormatContext ctx, IParseNode node)
The generate phase generate parse nodes in the output. The formatting phase turns these gene into string values. The parselet steers both of these operations through its generate and fo
-
formatStyled
public void formatStyled(FormatContext ctx, IParseNode node, IStyleAdapter adapter)
-
parse
public abstract java.lang.Object parse(Parser p)
The main parse method for the parselet. Use the supplied parser and generate a 'result' either a CharSequence or an IParseNode.
-
restore
public abstract java.lang.Object restore(Parser p, ISemanticNode oldModel, RestoreCtx ctx, boolean inherited)
The parselet method to re-create the parse node tree given the resulting model.
-
saveParse
public abstract void saveParse(IParseNode pn, ISemanticNode oldNode, SaveParseCtx spctx)
The parselet method to save parselet ids, offsets, string-lengths etc. The info that when augmented by a de-serialized semantic node quickly reproduces the parse-node tree and attaches it to the model
-
anyReparseChanges
protected boolean anyReparseChanges(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean forceReparse)
-
checkForSameAgainRegion
protected void checkForSameAgainRegion(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean beforeMatch, boolean forceReparse)
-
clearChangedRegion
protected void clearChangedRegion(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean beforeMatch, boolean forceReparse)
-
advancePointer
protected void advancePointer(Parser parser, java.lang.Object oldParseNode, DiffContext dctx)
-
reparse
public java.lang.Object reparse(Parser parser, java.lang.Object oldParseNode, DiffContext dctx, boolean forceReparse, Parselet exitParselet)
-
parseExtendedErrors
public java.lang.Object parseExtendedErrors(Parser p, Parselet exitParselet)
An optional method for parsing a repeating parselet. Used in the context when we know there are errors in the file and we know the parent parselet failed to parse the exit parselet in the current context. The parselet previous to exit parselet is called with this method. It will basically reparse itself, trying to skip over error text, making sure not to skip over the exit parselet. If we do match the exit parselet, we return the newly extended node.- Returns:
- null if we cannot extend our value. In this case, the parent parselet just uses the value it already has for this match.
-
reparseExtendedErrors
public java.lang.Object reparseExtendedErrors(Parser p, Parselet exitParselet, java.lang.Object oldChildNode, DiffContext dctx, boolean forceReparse)
-
peek
public boolean peek(Parser p)
-
generate
public abstract java.lang.Object generate(GenerateContext ctx, java.lang.Object semanticValue)
-
updateParseNodes
public int updateParseNodes(java.lang.Object semanticValue, IParseNode node)
-
isNullValid
public boolean isNullValid()
For most parselets (all but EOF), null is not a real value.
-
getFormattingParseNodes
public IParseNode[] getFormattingParseNodes(java.util.Set<Parselet> visited)
-
producesParselet
public boolean producesParselet(Parselet other)
-
producesParseletId
public boolean producesParseletId(int otherId)
-
typeMatches
public boolean typeMatches(java.lang.Object sv, boolean isElement)
-
elementTypeMatches
public boolean elementTypeMatches(java.lang.Object other)
-
dataTypeMatches
public boolean dataTypeMatches(java.lang.Object other)
-
getSemanticValueSlotClass
public java.lang.Class getSemanticValueSlotClass()
This is a hook for subclasses to override if there is any conditionality in how the slot class is used. For example, if the slot mappings are only applied to one of the children, and another child's value is forwarded directly the slot mappings are never applied to the directly forwarded element. So the type of the sequence will be a superclass of the type of the slot object.- Returns:
- the class for the semantic value for this slot.
-
isComplexStringType
public boolean isComplexStringType()
From the IDE's perspective,
-
getBeforeFirstNode
public IParseNode getBeforeFirstNode(IParseNode beforeFirstNode)
-
updateCachedResult
public void updateCachedResult(java.lang.Object res)
This is called when we are parsing using the result cache. This node found a match in the cache, but the semantic value's parse-node from the cached value might not be right - e.g. we parsed an expression as part of a methodPrefix, then did not find the :: and so stashed the result, but where the semantic value has been updated to point to the methodPrefix.2 - we need to reset it back to the parse-node which we are returning - probably lower down on the chain.
-
getSemanticValueMultiTypedArray
public boolean getSemanticValueMultiTypedArray()
-
-