The string represented by this token (i.e. the lexeme)
The category that this token falls into for the sake of syntax highlighting.
This defaults to null, which indicates that the token needs no highlighting.
Whether this token is ignored by the parser (default: false). This will generally
be true for comments and spaces and false for anything else. The purpose of
having ignored tokens is that they're still accessible when the tokenizer is used
by something other than the parser (like a syntax highlighter for example).
The category that this token falls into for the sake of syntax highlighting.
The category that this token falls into for the sake of syntax highlighting.
This defaults to null, which indicates that the token needs no highlighting.
Whether this token is ignored by the parser (default: false).
Whether this token is ignored by the parser (default: false). This will generally
be true for comments and spaces and false for anything else. The purpose of
having ignored tokens is that they're still accessible when the tokenizer is used
by something other than the parser (like a syntax highlighter for example).
The string represented by this token (i.e.
The string represented by this token (i.e. the lexeme)