|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object net.sf.saxon.expr.parser.Tokenizer
public final class Tokenizer
Tokenizer for expressions and inputs. This code was originally derived from James Clark's xt, though it has been greatly modified since. See copyright notice at end of file.
Field Summary | |
---|---|
static int |
BARE_NAME_STATE
State in which a name is NOT to be merged with what comes next, for example "(" |
int |
currentToken
The number identifying the most recently read token |
int |
currentTokenStartOffset
The position in the input expression where the current token starts |
String |
currentTokenValue
The string value of the most recently read token |
static int |
DEFAULT_STATE
Initial default state of the Tokenizer |
boolean |
disallowUnionKeyword
Flag to disallow "union" as a synonym for "|" when parsing XSLT 2.0 patterns |
String |
input
The string being parsed |
int |
inputOffset
The current position within the input string |
boolean |
isXQuery
Flag to indicate that this is XQuery as distinct from XPath |
static int |
OPERATOR_STATE
State in which the next thing to be read is an operator |
static int |
SEQUENCE_TYPE_STATE
State in which the next thing to be read is a SequenceType |
int |
startLineNumber
The starting line number (for XPath in XSLT, the line number in the stylesheet) |
Constructor Summary | |
---|---|
Tokenizer()
|
Method Summary | |
---|---|
int |
getColumnNumber()
Get the column number of the current token |
int |
getColumnNumber(int offset)
Return the column number corresponding to a given offset in the expression |
long |
getLineAndColumn(int offset)
Get the line and column number corresponding to a given offset in the input expression, as a long value with the line number in the top half and the column number in the lower half |
int |
getLineNumber()
Get the line number of the current token |
int |
getLineNumber(int offset)
Return the line number corresponding to a given offset in the expression |
int |
getState()
Get the current tokenizer state |
void |
incrementLineNumber(int offset)
Increment the line number, making a record of where in the input string the newline character occurred. |
void |
lookAhead()
Look ahead by one token. |
void |
next()
Get the next token from the input expression. |
char |
nextChar()
Read next character directly. |
String |
recentText(int offset)
Get the most recently read text (for use in an error message) |
void |
setState(int state)
Set the tokenizer into a special state |
void |
tokenize(String input,
int start,
int end,
int lineNumber)
Prepare a string for tokenization. |
void |
treatCurrentAsOperator()
Force the current token to be treated as an operator if possible |
void |
unreadChar()
Step back one character. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final int DEFAULT_STATE
public static final int BARE_NAME_STATE
public static final int SEQUENCE_TYPE_STATE
public static final int OPERATOR_STATE
public int startLineNumber
public int currentToken
public String currentTokenValue
public int currentTokenStartOffset
public String input
public int inputOffset
public boolean disallowUnionKeyword
public boolean isXQuery
Constructor Detail |
---|
public Tokenizer()
Method Detail |
---|
public int getState()
public void setState(int state)
state
- the new statepublic void tokenize(String input, int start, int end, int lineNumber) throws XPathException
input
- the string to be tokenizedstart
- start point within the stringend
- end point within the string (last character not read):
-1 means end of stringlineNumber
- the linenumber in the source where the expression appears
XPathException
- if a lexical error occurs, e.g. unmatched
string quotespublic void next() throws XPathException
XPathException
- if a lexical error is detectedpublic void treatCurrentAsOperator()
public void lookAhead() throws XPathException
XPathException
- if a lexical error occurspublic char nextChar() throws StringIndexOutOfBoundsException
StringIndexOutOfBoundsException
- if an attempt is made to read beyond
the end of the string. This will only occur in the event of a syntax error in the
input.public void incrementLineNumber(int offset)
offset
- the place in the input string where the newline occurredpublic void unreadChar()
public String recentText(int offset)
offset
- the offset of the offending token, if known, or -1 to use the current offset
public int getLineNumber()
public int getColumnNumber()
public long getLineAndColumn(int offset)
offset
- the byte offset in the expression
public int getLineNumber(int offset)
offset
- the byte offset in the expression
public int getColumnNumber(int offset)
offset
- the byte offset in the expression
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |