java.io
Class StreamTokenizer

java.lang.Object
  |
  +--java.io.StreamTokenizer

public class StreamTokenizer
extends Object

A class to turn an input stream into a stream of tokens. There are a number of methods that define the lexical syntax of tokens.


Field Summary
 double nval
          The number value.
 String sval
          The Stream value.
static int TT_EOF
          The End-of-file token.
static int TT_EOL
          The End-of-line token.
static int TT_NUMBER
          The number token.
static int TT_WORD
          The word token.
 int ttype
          The type of the last token returned.
 
Constructor Summary
StreamTokenizer(InputStream I)
          Creates a stream tokenizer that parses the specified input stream.
 
Method Summary
 void commentChar(int ch)
          Specifies that this character starts a single line comment.
 void eolIsSignificant(boolean flag)
          If the flag is true, end-of-lines are significant (TT_EOL will be returned by nexttoken).
 int lineno()
          Return the current line number.
 void lowerCaseMode(boolean fl)
          Examines a boolean to decide whether TT_WORD tokens are forced to be lower case.
 int nextToken()
          Parses a token from the input stream.
 void ordinaryChar(int ch)
          Specifies that this character is 'ordinary': it removes any significance as a word, comment, string, whitespace or number character.
 void ordinaryChars(int low, int hi)
          Specifies that characters in this range are 'ordinary'.
 void parseNumbers()
          Specifies that numbers should be parsed.
 void pushBack()
          Pushes back a stream token.
 void quoteChar(int ch)
          Specifies that matching pairs of this character delimit String constants.
 void resetSyntax()
          Resets the syntax table so that all characters are special.
 void slashSlashComments(boolean flag)
          If the flag is true, recognize C++ style( // ) comments.
 void slashStarComments(boolean flag)
          If the flag is true, recognize C style( /* ) comments.
 String toString()
          Returns the String representation of the stream token.
 void whitespaceChars(int low, int hi)
          Specifies that characters in this range are whitespace characters.
 void wordChars(int low, int hi)
          Specifies that characters in this range are word characters.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

ttype

public int ttype
The type of the last token returned. It's value will either be one of the following TT_* constants, or a single character. For example, if '+' is encountered and is not a valid word character, ttype will be '+'


TT_EOF

public static final int TT_EOF
The End-of-file token.

See Also:
Constant Field Values

TT_EOL

public static final int TT_EOL
The End-of-line token.

See Also:
Constant Field Values

TT_NUMBER

public static final int TT_NUMBER
The number token. This value is in nval.

See Also:
Constant Field Values

TT_WORD

public static final int TT_WORD
The word token. This value is in sval.

See Also:
Constant Field Values

sval

public String sval
The Stream value.


nval

public double nval
The number value.

Constructor Detail

StreamTokenizer

public StreamTokenizer(InputStream I)
Creates a stream tokenizer that parses the specified input stream. By default, it recognizes numbers, Strings quoted with single and double quotes, and all the alphabetics.

Parameters:
I - the input stream
Method Detail

resetSyntax

public void resetSyntax()
Resets the syntax table so that all characters are special.


wordChars

public void wordChars(int low,
                      int hi)
Specifies that characters in this range are word characters.

Parameters:
low - the low end of the range
hi - the high end of the range

whitespaceChars

public void whitespaceChars(int low,
                            int hi)
Specifies that characters in this range are whitespace characters.

Parameters:
low - the low end of the range
hi - the high end of the range

ordinaryChars

public void ordinaryChars(int low,
                          int hi)
Specifies that characters in this range are 'ordinary'. Ordinary characters mean that any significance as words, comments, strings, whitespaces or number characters are removed. When these characters are encountered by the parser, they return a ttype equal to the character.

Parameters:
low - the low end of the range
hi - the high end of the range

ordinaryChar

public void ordinaryChar(int ch)
Specifies that this character is 'ordinary': it removes any significance as a word, comment, string, whitespace or number character. When encountered by the parser, it returns a ttype equal to the character.

Parameters:
ch - the character

commentChar

public void commentChar(int ch)
Specifies that this character starts a single line comment.

Parameters:
ch - the character

quoteChar

public void quoteChar(int ch)
Specifies that matching pairs of this character delimit String constants. When a String constant is recognized, ttype will be the character that delimits the String, and sval will have the body of the String.

Parameters:
ch - the character

parseNumbers

public void parseNumbers()
Specifies that numbers should be parsed. This method accepts double precision floating point numbers and returns a ttype of TT_NUMBER with the value in nval.


eolIsSignificant

public void eolIsSignificant(boolean flag)
If the flag is true, end-of-lines are significant (TT_EOL will be returned by nexttoken). If false, they will be treated as whitespace.


slashStarComments

public void slashStarComments(boolean flag)
If the flag is true, recognize C style( /* ) comments.


slashSlashComments

public void slashSlashComments(boolean flag)
If the flag is true, recognize C++ style( // ) comments.


lowerCaseMode

public void lowerCaseMode(boolean fl)
Examines a boolean to decide whether TT_WORD tokens are forced to be lower case.

Parameters:
fl - the boolean flag

nextToken

public int nextToken()
              throws IOException
Parses a token from the input stream. The return value is the same as the value of ttype. Typical clients of this class first set up the syntax tables and then sit in a loop calling nextToken to parse successive tokens until TT_EOF is returned.

IOException

pushBack

public void pushBack()
Pushes back a stream token.


lineno

public int lineno()
Return the current line number.


toString

public String toString()
Returns the String representation of the stream token.

Overrides:
toString in class Object