Class: TokenType

TokenType

TokenType is an enum. A TokenType specifies the type of a token. The different token types are the differnt terminal tokens in the syntactic grammar defined in LEMParser

Each TokenType has one or more texual representations. TokenTypes such as TokenType.PIPE (aka the | character) have only one texual representation. Other token types such as string literals have multiple or even an inifinite number of possible texual representations.

Tokens can be either text-based or symbolic. For example, TokenType.AND ("and") is text-based whereas TokenType.EQUAL ("=") is symbolic

Constructor

new TokenType(text, isTextopt, nullable, prettyNameopt, nullable)

The constructor for this enum should never be used outside of the TokenType class
Parameters:
Name Type Attributes Description
text String The texual representation of the token
isText Boolean <optional>
<nullable>
prettyName String <optional>
<nullable>
Source:

Members

(static, constant) AMP

Source:

(static, constant) CARET

Source:

(static, constant) CHAR_SEQ

CHAR_SEQ = / [a-zA-Z0-9]* /
Source:

(static, constant) DOLLAR

Source:

(static, constant) EQUAL

Source:

(static, constant) EXCLAMATION_EQUAL

Source:

(static, constant) L_PAREN

Source:

(static, constant) PIPE

Source:

(static, constant) R_PAREN

Source:

(static, constant) REGEX_LITERAL

RegexLiteral = / \/[^\/]*\/ /
Source:

(static, constant) STRING_LITERAL

StringLiteral = / "[^"]*" /
Source:

Methods

(static) getSymbolicTokenStringSet() → {Object.<String, TokenType>}

Returns a mapping of the texual representation of symbolic tokens to the corresponding TokenType enum.
Source:
Returns:
Type
Object.<String, TokenType>

(static) getTextTokenStringSet() → {Object.<String, TokenType>}

Returns a mapping of the texual representation of text-based tokens to the corresponding TokenType enum. TokenTypes that have multiple texual representations are not included.
Source:
Returns:
Type
Object.<String, TokenType>

(static) getTokenTypes() → {Array.<TokenType>}

Gets all valid token types
Source:
Returns:
All valid token types as an array
Type
Array.<TokenType>

getIsText() → {Boolean}

Returns true if this token type is text-based as opposed to symbolic
Source:
Returns:
True if token type is text based
Type
Boolean

getPrettyName() → {String}

Returns the pretty name or display name of this token type. The pretty name is one that would make sense to the end-user
Source:
Returns:
The pretty name
Type
String

getText() → {String}

Gets the texual representation of this token type. If this token type has multiple permissible texual representations (such as a string literal token type), this method returns null
Source:
Returns:
The texual representation of this type of token, or null if multiple exist
Type
String