Uses of Interface
org.antlr.runtime.Token

Packages that use Token

Package	Description
org.antlr.analysis
org.antlr.codegen
org.antlr.grammar.v3
org.antlr.runtime
org.antlr.runtime.debug
org.antlr.runtime.tree
org.antlr.tool

Uses of Token in org.antlr.analysis

Fields in org.antlr.analysis with type parameters of type Token

Modifier and Type	Field	Description
protected java.util.Map<DFAState,java.util.Map<java.lang.Integer,java.util.Set<Token>>>	DecisionProbe.stateToIncompletelyCoveredAltsMap	Tracks alts insufficiently covered.

Methods in org.antlr.analysis that return types with arguments of type Token

Modifier and Type	Method	Description
java.util.List<Token>	MachineProbe.getGrammarLocationsForInputSequence(java.util.List<java.util.Set<NFAState>> nfaStates, java.util.List<IntSet> labels)	Given an alternative associated with a DFA state, return the list of tokens (from grammar) associated with path through NFA following the labels sequence.
java.util.Map<java.lang.Integer,java.util.Set<Token>>	DecisionProbe.getIncompletelyCoveredAlts(DFAState d)	Return a list of alts whose predicate context was insufficient to resolve a nondeterminism for state d.

Method parameters in org.antlr.analysis with type arguments of type Token

Modifier and Type	Method	Description
void	DecisionProbe.reportIncompletelyCoveredAlts(DFAState d, java.util.Map<java.lang.Integer,java.util.Set<Token>> altToLocationsReachableWithoutPredicate)

Uses of Token in org.antlr.codegen

Methods in org.antlr.codegen with parameters of type Token

Modifier and Type	Method	Description
void	CodeGenerator.issueInvalidAttributeError(java.lang.String x, java.lang.String y, Rule enclosingRule, Token actionToken, int outerAltNum)
void	CodeGenerator.issueInvalidAttributeError(java.lang.String x, Rule enclosingRule, Token actionToken, int outerAltNum)
void	CodeGenerator.issueInvalidScopeError(java.lang.String x, java.lang.String y, Rule enclosingRule, Token actionToken, int outerAltNum)
java.util.List<java.lang.Object>	Python3Target.postProcessAction(java.util.List<java.lang.Object> chunks, Token actionToken)
java.util.List<java.lang.Object>	PythonTarget.postProcessAction(java.util.List<java.lang.Object> chunks, Token actionToken)
java.util.List<java.lang.Object>	Target.postProcessAction(java.util.List<java.lang.Object> chunks, Token actionToken)	Give target a chance to do some postprocessing on actions.
org.stringtemplate.v4.ST	CodeGenerator.translateTemplateConstructor(java.lang.String ruleName, int outerAltNum, Token actionToken, java.lang.String templateActionText)	Given a template constructor action like %foo(a={...}) in an action, translate it to the appropriate template constructor from the templateLib.

Uses of Token in org.antlr.grammar.v3

Fields in org.antlr.grammar.v3 declared as Token

Modifier and Type	Field	Description
(package private) Token	ActionAnalysis.actionToken
(package private) Token	ActionTranslator.actionToken
Token	ANTLRParser.GrammarASTErrorNode.start
Token	ANTLRParser.GrammarASTErrorNode.stop

Methods in org.antlr.grammar.v3 that return Token

Modifier and Type	Method	Description
private Token	ANTLRParser.LT(int k)
Token	ActionAnalysis.nextToken()
Token	ActionTranslator.nextToken()
Token	ANTLRLexer.nextToken()

Methods in org.antlr.grammar.v3 with parameters of type Token

Modifier and Type	Method	Description
java.lang.Object	ANTLRParser.grammar_Adaptor.create(Token payload)
java.lang.Object	ANTLRParser.grammar_Adaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e)

Constructors in org.antlr.grammar.v3 with parameters of type Token

Constructor	Description
ActionTranslator(CodeGenerator generator, java.lang.String ruleName, Token actionToken, int outerAltNum)
GrammarASTErrorNode(TokenStream input, Token start, Token stop, RecognitionException e)

Uses of Token in org.antlr.runtime

Classes in org.antlr.runtime that implement Token

Modifier and Type	Class	Description
class	ClassicToken	A Token object like we'd use in ANTLR 2.x; has an actual string created and associated with this object.
class	CommonToken

Fields in org.antlr.runtime declared as Token

Modifier and Type	Field	Description
static Token	Token.EOF_TOKEN
static Token	Token.INVALID_TOKEN
static Token	Token.SKIP_TOKEN	In an action, a lexer rule can set token to this SKIP_TOKEN and ANTLR will avoid creating a token for this symbol and try to fetch another.
Token	ParserRuleReturnScope.start
Token	ParserRuleReturnScope.stop
Token	RecognitionException.token	The current Token when an error occurred.
Token	RecognizerSharedState.token	The goal of all lexer rules/methods is to create a token object.

Fields in org.antlr.runtime with type parameters of type Token

Modifier and Type	Field	Description
protected java.util.List<Token>	BufferedTokenStream.tokens	Record every single token pulled from the source so we can reproduce chunks of it later.
protected java.util.List<Token>	LegacyCommonTokenStream.tokens	Record every single token pulled from the source so we can reproduce chunks of it later.

Methods in org.antlr.runtime that return Token

Modifier and Type	Method	Description
Token	Lexer.emit()	The standard method called to automatically emit a token at the outermost lexical rule.
Token	BufferedTokenStream.get(int i)
Token	LegacyCommonTokenStream.get(int i)	Return absolute token i; ignore which channel the tokens are on; that is, count all tokens not just on-channel tokens.
Token	TokenStream.get(int i)	Get a token at an absolute index i; 0..n-1.
Token	UnbufferedTokenStream.get(int i)
Token	Lexer.getEOFToken()	Returns the EOF token (default), if you need to return a custom token instead override this method.
Token	UnwantedTokenException.getUnexpectedToken()
protected Token	BufferedTokenStream.LB(int k)
protected Token	CommonTokenStream.LB(int k)
protected Token	LegacyCommonTokenStream.LB(int k)	Look backwards k tokens on-channel tokens
Token	BufferedTokenStream.LT(int k)
Token	CommonTokenStream.LT(int k)
Token	LegacyCommonTokenStream.LT(int k)	Get the ith token from the current position 1..n where k=1 is the first symbol of lookahead.
Token	TokenStream.LT(int k)	Get Token at current input pointer + i ahead where i=1 is next Token.
Token	UnbufferedTokenStream.nextElement()
Token	Lexer.nextToken()	Return a token from this source; i.e., match a token on the char stream.
Token	TokenSource.nextToken()	Return a Token object from your input stream (usually a CharStream).

Methods in org.antlr.runtime that return types with arguments of type Token

Modifier and Type	Method	Description
java.util.List<? extends Token>	BufferedTokenStream.get(int start, int stop)	Get all tokens from start..stop inclusively
java.util.List<? extends Token>	LegacyCommonTokenStream.get(int start, int stop)	Get all tokens from start..stop inclusively
java.util.List<? extends Token>	BufferedTokenStream.getTokens()
java.util.List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop)
java.util.List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop, int ttype)
java.util.List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop, java.util.List<java.lang.Integer> types)
java.util.List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop, BitSet types)	Given a start and stop index, return a List of all tokens in the token type BitSet.
java.util.List<? extends Token>	LegacyCommonTokenStream.getTokens()
java.util.List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop)
java.util.List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop, int ttype)
java.util.List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop, java.util.List<java.lang.Integer> types)
java.util.List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop, BitSet types)	Given a start and stop index, return a List of all tokens in the token type BitSet.

Methods in org.antlr.runtime with parameters of type Token

Modifier and Type	Method	Description
void	TokenRewriteStream.delete(java.lang.String programName, Token from, Token to)
void	TokenRewriteStream.delete(Token indexT)
void	TokenRewriteStream.delete(Token from, Token to)
void	Lexer.emit(Token token)	Currently does not support multiple emits per nextToken invocation for efficiency reasons.
java.lang.String	BaseRecognizer.getTokenErrorDisplay(Token t)	How should a token be displayed in an error message? The default is to display just the text, but during development you might want to have a lot of information spit out.
void	TokenRewriteStream.insertAfter(java.lang.String programName, Token t, java.lang.Object text)
void	TokenRewriteStream.insertAfter(Token t, java.lang.Object text)
void	TokenRewriteStream.insertBefore(java.lang.String programName, Token t, java.lang.Object text)
void	TokenRewriteStream.insertBefore(Token t, java.lang.Object text)
boolean	UnbufferedTokenStream.isEOF(Token o)
void	TokenRewriteStream.replace(java.lang.String programName, Token from, Token to, java.lang.Object text)
void	TokenRewriteStream.replace(Token indexT, java.lang.Object text)
void	TokenRewriteStream.replace(Token from, Token to, java.lang.Object text)
java.lang.String	BufferedTokenStream.toString(Token start, Token stop)
java.lang.String	LegacyCommonTokenStream.toString(Token start, Token stop)
java.lang.String	TokenStream.toString(Token start, Token stop)	Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location.
java.lang.String	UnbufferedTokenStream.toString(Token start, Token stop)

Method parameters in org.antlr.runtime with type arguments of type Token

Modifier and Type	Method	Description
java.util.List<java.lang.String>	BaseRecognizer.toStrings(java.util.List<? extends Token> tokens)	A convenience method for use most often with template rewrites.

Constructors in org.antlr.runtime with parameters of type Token

Constructor	Description
ClassicToken(Token oldToken)
CommonToken(Token oldToken)

Uses of Token in org.antlr.runtime.debug

Classes in org.antlr.runtime.debug that implement Token

Modifier and Type	Class	Description
static class	RemoteDebugEventSocketListener.ProxyToken

Fields in org.antlr.runtime.debug declared as Token

Modifier and Type	Field	Description
protected Token	Profiler.lastRealTokenTouchedInDecision

Fields in org.antlr.runtime.debug with type parameters of type Token

Modifier and Type	Field	Description
(package private) java.util.List<Token>	ParseTreeBuilder.hiddenTokens

Methods in org.antlr.runtime.debug that return Token

Modifier and Type	Method	Description
Token	DebugTokenStream.get(int i)
Token	DebugTreeAdaptor.getToken(java.lang.Object t)
Token	DebugTokenStream.LT(int i)

Methods in org.antlr.runtime.debug with parameters of type Token

Modifier and Type	Method	Description
void	DebugTreeAdaptor.addChild(java.lang.Object t, Token child)
java.lang.Object	DebugTreeAdaptor.becomeRoot(Token newRoot, java.lang.Object oldRoot)
void	BlankDebugEventListener.consumeHiddenToken(Token token)
void	DebugEventHub.consumeHiddenToken(Token token)
void	DebugEventListener.consumeHiddenToken(Token t)	An off-channel input token was consumed.
void	DebugEventRepeater.consumeHiddenToken(Token token)
void	DebugEventSocketProxy.consumeHiddenToken(Token t)
void	ParseTreeBuilder.consumeHiddenToken(Token token)
void	Profiler.consumeHiddenToken(Token token)
void	BlankDebugEventListener.consumeToken(Token token)
void	DebugEventHub.consumeToken(Token token)
void	DebugEventListener.consumeToken(Token t)	An input token was consumed; matched by any kind of element.
void	DebugEventRepeater.consumeToken(Token token)
void	DebugEventSocketProxy.consumeToken(Token t)
void	ParseTreeBuilder.consumeToken(Token token)
void	Profiler.consumeToken(Token token)
java.lang.Object	DebugTreeAdaptor.create(int tokenType, Token fromToken)
java.lang.Object	DebugTreeAdaptor.create(int tokenType, Token fromToken, java.lang.String text)
java.lang.Object	DebugTreeAdaptor.create(Token payload)
void	BlankDebugEventListener.createNode(java.lang.Object node, Token token)
void	DebugEventHub.createNode(java.lang.Object node, Token token)
void	DebugEventListener.createNode(java.lang.Object node, Token token)	Announce a new node built from an existing token.
void	DebugEventRepeater.createNode(java.lang.Object node, Token token)
void	DebugEventSocketProxy.createNode(java.lang.Object node, Token token)
void	TraceDebugEventListener.createNode(java.lang.Object node, Token token)
java.lang.Object	DebugTreeAdaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e)
void	BlankDebugEventListener.LT(int i, Token t)
void	DebugEventHub.LT(int index, Token t)
void	DebugEventListener.LT(int i, Token t)	Somebody (anybody) looked ahead.
void	DebugEventRepeater.LT(int i, Token t)
void	DebugEventSocketProxy.LT(int i, Token t)
void	Profiler.LT(int i, Token t)	Track refs to lookahead if in a fixed/nonfixed decision.
protected java.lang.String	DebugEventSocketProxy.serializeToken(Token t)
void	DebugTreeAdaptor.setTokenBoundaries(java.lang.Object t, Token startToken, Token stopToken)
java.lang.String	DebugTokenStream.toString(Token start, Token stop)

Uses of Token in org.antlr.runtime.tree

Fields in org.antlr.runtime.tree declared as Token

Modifier and Type	Field	Description
Token	CommonErrorNode.start
Token	CommonErrorNode.stop
Token	CommonTree.token	A single token is the payload

Fields in org.antlr.runtime.tree with type parameters of type Token

Modifier and Type	Field	Description
java.util.List<Token>	ParseTree.hiddenTokens

Methods in org.antlr.runtime.tree that return Token

Modifier and Type	Method	Description
abstract Token	BaseTreeAdaptor.createToken(int tokenType, java.lang.String text)	Tell me how to create a token for use with imaginary token nodes.
abstract Token	BaseTreeAdaptor.createToken(Token fromToken)	Tell me how to create a token for use with imaginary token nodes.
Token	CommonTreeAdaptor.createToken(int tokenType, java.lang.String text)	Tell me how to create a token for use with imaginary token nodes.
Token	CommonTreeAdaptor.createToken(Token fromToken)	Tell me how to create a token for use with imaginary token nodes.
Token	CommonTree.getToken()
Token	CommonTreeAdaptor.getToken(java.lang.Object t)	What is the Token associated with this node? If you are not using CommonTree, then you must override this in your own adaptor.
Token	TreeAdaptor.getToken(java.lang.Object t)	Return the token object from which this node was created.
Token	RewriteRuleTokenStream.nextToken()

Methods in org.antlr.runtime.tree with parameters of type Token

Modifier and Type	Method	Description
java.lang.Object	BaseTreeAdaptor.becomeRoot(Token newRoot, java.lang.Object oldRoot)
java.lang.Object	TreeAdaptor.becomeRoot(Token newRoot, java.lang.Object oldRoot)	Create a node for newRoot make it the root of oldRoot.
java.lang.Object	BaseTreeAdaptor.create(int tokenType, Token fromToken)
java.lang.Object	BaseTreeAdaptor.create(int tokenType, Token fromToken, java.lang.String text)
java.lang.Object	CommonTreeAdaptor.create(Token payload)
java.lang.Object	TreeAdaptor.create(int tokenType, Token fromToken)	Create a new node derived from a token, with a new token type.
java.lang.Object	TreeAdaptor.create(int tokenType, Token fromToken, java.lang.String text)	Same as create(tokenType,fromToken) except set the text too.
java.lang.Object	TreeAdaptor.create(Token payload)	Create a tree node from Token object; for CommonTree type trees, then the token just becomes the payload.
java.lang.Object	TreeWizard.TreePatternTreeAdaptor.create(Token payload)
abstract Token	BaseTreeAdaptor.createToken(Token fromToken)	Tell me how to create a token for use with imaginary token nodes.
Token	CommonTreeAdaptor.createToken(Token fromToken)	Tell me how to create a token for use with imaginary token nodes.
java.lang.Object	BaseTreeAdaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e)	create tree node that holds the start and stop tokens associated with an error.
java.lang.Object	TreeAdaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e)	Return a tree node representing an error.
void	CommonTreeAdaptor.setTokenBoundaries(java.lang.Object t, Token startToken, Token stopToken)	Track start/stop token for subtree root created for a rule.
void	TreeAdaptor.setTokenBoundaries(java.lang.Object t, Token startToken, Token stopToken)	Where are the bounds in the input token stream for this node and all children? Each rule that creates AST nodes will call this method right before returning.

Constructors in org.antlr.runtime.tree with parameters of type Token

Constructor	Description
CommonErrorNode(TokenStream input, Token start, Token stop, RecognitionException e)
CommonTree(Token t)
TreePattern(Token payload)
WildcardTreePattern(Token payload)

Uses of Token in org.antlr.tool

Fields in org.antlr.tool declared as Token

Modifier and Type	Field	Description
Token	AttributeScope.derivedFromToken	This scope is associated with which input token (for error handling)?
Token	Grammar.LabelElementPair.label
Token	GrammarSemanticsMessage.offendingToken	Most of the time, we'll have a token such as an undefined rule ref and so this will be set.
Token	GrammarSyntaxMessage.offendingToken	Most of the time, we'll have a token and so this will be set.

Fields in org.antlr.tool with type parameters of type Token

Modifier and Type	Field	Description
java.util.Map<java.lang.Integer,java.util.Set<Token>>	GrammarInsufficientPredicatesMessage.altToLocations
protected java.util.Set<Token>	Grammar.tokenIDRefs	The unique set of all token ID references in any rule

Methods in org.antlr.tool that return Token

Modifier and Type	Method	Description
Token	Interpreter.nextToken()

Methods in org.antlr.tool with parameters of type Token

Modifier and Type	Method	Description
protected void	NameSpaceChecker.checkForLabelConflict(Rule r, Token label)	Make sure a label doesn't conflict with another symbol.
boolean	NameSpaceChecker.checkForLabelTypeMismatch(Rule r, Token label, int type)	If type of previous label differs from new label's type, that's an error.
AttributeScope	Grammar.createParameterScope(java.lang.String ruleName, Token argAction)
AttributeScope	Grammar.createReturnScope(java.lang.String ruleName, Token retAction)
AttributeScope	Grammar.createRuleScope(java.lang.String ruleName, Token scopeAction)
AttributeScope	Grammar.defineGlobalScope(java.lang.String name, Token scopeAction)
protected void	Grammar.defineLabel(Rule r, Token label, GrammarAST element, int type)	Define a label defined in a rule r; check the validity then ask the Rule object to actually define it.
void	Rule.defineLabel(Token label, GrammarAST elementRef, int type)
void	Grammar.defineLexerRuleFoundInParser(Token ruleToken, GrammarAST ruleAST)
void	Grammar.defineRule(Token ruleToken, java.lang.String modifier, java.util.Map<java.lang.String,java.lang.Object> options, GrammarAST tree, GrammarAST argActionAST, int numAlts)	Define a new rule.
void	Grammar.defineRuleListLabel(java.lang.String ruleName, Token label, GrammarAST element)
void	Grammar.defineRuleRefLabel(java.lang.String ruleName, Token label, GrammarAST ruleRef)
void	Grammar.defineTokenListLabel(java.lang.String ruleName, Token label, GrammarAST element)
void	Grammar.defineTokenRefLabel(java.lang.String ruleName, Token label, GrammarAST tokenRef)
void	Grammar.defineWildcardTreeLabel(java.lang.String ruleName, Token label, GrammarAST tokenRef)
void	Grammar.defineWildcardTreeListLabel(java.lang.String ruleName, Token label, GrammarAST tokenRef)
java.util.Collection<java.lang.String>	LeftRecursiveRuleAnalyzer.getNamesFromArgAction(Token t)
static void	ErrorManager.grammarError(int msgID, Grammar g, Token token)
static void	ErrorManager.grammarError(int msgID, Grammar g, Token token, java.lang.Object arg)
static void	ErrorManager.grammarError(int msgID, Grammar g, Token token, java.lang.Object arg, java.lang.Object arg2)
static void	ErrorManager.grammarWarning(int msgID, Grammar g, Token token)
static void	ErrorManager.grammarWarning(int msgID, Grammar g, Token token, java.lang.Object arg)
static void	ErrorManager.grammarWarning(int msgID, Grammar g, Token token, java.lang.Object arg, java.lang.Object arg2)
void	GrammarAST.initialize(Token token)
java.lang.String	Grammar.setOption(java.lang.String key, java.lang.Object value, Token optionsStartToken)	Save the option key/value pair and process it; return the key or null if invalid option.
java.lang.String	Rule.setOption(java.lang.String key, java.lang.Object value, Token optionsStartToken)	Save the option key/value pair and process it; return the key or null if invalid option.
void	Grammar.setOptions(java.util.Map<java.lang.String,java.lang.Object> options, Token optionsStartToken)
void	Rule.setOptions(java.util.Map<java.lang.String,java.lang.Object> options, Token optionsStartToken)
void	GrammarAST.setTokenBoundaries(Token startToken, Token stopToken)	Track start/stop token for subtree root created for a rule.
static void	ErrorManager.syntaxError(int msgID, Grammar grammar, Token token, java.lang.Object arg, RecognitionException re)

Method parameters in org.antlr.tool with type arguments of type Token

Modifier and Type	Method	Description
static void	ErrorManager.insufficientPredicates(DecisionProbe probe, DFAState d, java.util.Map<java.lang.Integer,java.util.Set<Token>> altToUncoveredLocations)

Constructors in org.antlr.tool with parameters of type Token

Constructor	Description
AttributeScope(java.lang.String name, Token derivedFromToken)
AttributeScope(Grammar grammar, java.lang.String name, Token derivedFromToken)
GrammarAST(Token token)
GrammarSemanticsMessage(int msgID, Grammar g, Token offendingToken)
GrammarSemanticsMessage(int msgID, Grammar g, Token offendingToken, java.lang.Object arg)
GrammarSemanticsMessage(int msgID, Grammar g, Token offendingToken, java.lang.Object arg, java.lang.Object arg2)
GrammarSyntaxMessage(int msgID, Grammar grammar, Token offendingToken, java.lang.Object arg, RecognitionException exception)
GrammarSyntaxMessage(int msgID, Grammar grammar, Token offendingToken, RecognitionException exception)
LabelElementPair(Token label, GrammarAST elementRef)
RuleLabelScope(Rule referencedRule, Token actionToken)

Constructor parameters in org.antlr.tool with type arguments of type Token

Constructor	Description
GrammarInsufficientPredicatesMessage(DecisionProbe probe, DFAState problemState, java.util.Map<java.lang.Integer,java.util.Set<Token>> altToLocations)