java.lang.Object | |
↳ | java.io.StreamTokenizer |
Parses a stream into a set of defined tokens, one at a time. The different types of tokens that can be found are numbers, identifiers, quoted strings, and different comment styles. The class can be used for limited processing of source code of programming languages like Java, although it is nowhere near a full parser.
Constants | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
int | TT_EOF | The constant representing the end of the stream. | |||||||||
int | TT_EOL | The constant representing the end of the line. | |||||||||
int | TT_NUMBER | The constant representing a number token. | |||||||||
int | TT_WORD | The constant representing a word token. |
Fields | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
nval |
Contains a number if the current token is a number (
ttype
==
TT_NUMBER
).
|
|||||||||
|
sval |
Contains a string if the current token is a word (
ttype
==
TT_WORD
).
|
|||||||||
|
ttype |
After calling
nextToken()
,
ttype
contains the type of
token that has been read.
|
Public Constructors | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
This constructor was deprecated
in API level 1.
Use
StreamTokenizer(Reader)
instead.
|
||||||||||
|
Constructs a new
StreamTokenizer
with
r
as source reader.
|
Public Methods | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
Specifies that the character
ch
shall be treated as a comment
character.
|
||||||||||
|
Specifies whether the end of a line is significant and should be returned
as
TT_EOF
in
ttype
by this tokenizer.
|
||||||||||
|
Returns the current line number.
|
||||||||||
|
Specifies whether word tokens should be converted to lower case when they
are stored in
sval
.
|
||||||||||
|
Parses the next token from this tokenizer's source stream or reader.
|
||||||||||
|
Specifies that the character
ch
shall be treated as an ordinary
character by this tokenizer.
|
||||||||||
|
Specifies that the characters in the range from
low
to
hi
shall be treated as an ordinary character by this tokenizer.
|
||||||||||
|
Specifies that this tokenizer shall parse numbers.
|
||||||||||
|
Indicates that the current token should be pushed back and returned again
the next time
nextToken()
is called.
|
||||||||||
|
Specifies that the character
ch
shall be treated as a quote
character.
|
||||||||||
|
Specifies that all characters shall be treated as ordinary characters.
|
||||||||||
|
Specifies whether "slash-slash" (C++-style) comments shall be recognized.
|
||||||||||
|
Specifies whether "slash-star" (C-style) comments shall be recognized.
|
||||||||||
|
Returns the state of this tokenizer in a readable format.
|
||||||||||
|
Specifies that the characters in the range from
low
to
hi
shall be treated as whitespace characters by this tokenizer.
|
||||||||||
|
Specifies that the characters in the range from
low
to
hi
shall be treated as word characters by this tokenizer.
|
[Expand]
Inherited Methods
|
|||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
From class
java.lang.Object
|
The constant representing the end of the stream.
The constant representing the end of the line.
The constant representing a number token.
The constant representing a word token.
Contains a number if the current token is a number (
ttype
==
TT_NUMBER
).
Contains a string if the current token is a word (
ttype
==
TT_WORD
).
After calling
nextToken()
,
ttype
contains the type of
token that has been read. When a single character is read, its value
converted to an integer is stored in
ttype
. For a quoted string,
the value is the quoted character. Otherwise, its value is one of the
following:
TT_WORD
- the token is a word.
TT_NUMBER
- the token is a number.
TT_EOL
- the end of line has been reached. Depends on
whether
eolIsSignificant
is
true
.
TT_EOF
- the end of the stream has been reached.
This constructor was deprecated
in API level 1.
Use
StreamTokenizer(Reader)
instead.
Constructs a new
StreamTokenizer
with
is
as source input
stream. This constructor is deprecated; instead, the constructor that
takes a
Reader
as an argument should be used.
is | the source stream from which to parse tokens. |
---|
NullPointerException |
if
is
is
null
.
|
---|
Constructs a new
StreamTokenizer
with
r
as source reader.
The tokenizer's initial state is as follows:
r | the source reader from which to parse tokens. |
---|
Specifies that the character
ch
shall be treated as a comment
character.
ch | the character to be considered a comment character. |
---|
Specifies whether the end of a line is significant and should be returned
as
TT_EOF
in
ttype
by this tokenizer.
flag |
true
if EOL is significant,
false
otherwise.
|
---|
Returns the current line number.
Specifies whether word tokens should be converted to lower case when they
are stored in
sval
.
flag |
true
if
sval
should be converted to lower
case,
false
otherwise.
|
---|
Parses the next token from this tokenizer's source stream or reader. The
type of the token is stored in the
ttype
field, additional
information may be stored in the
nval
or
sval
fields.
ttype
.
IOException | if an I/O error occurs while parsing the next token. |
---|
Specifies that the character
ch
shall be treated as an ordinary
character by this tokenizer. That is, it has no special meaning as a
comment character, word component, white space, string delimiter or
number.
ch | the character to be considered an ordinary character. |
---|
Specifies that the characters in the range from
low
to
hi
shall be treated as an ordinary character by this tokenizer. That is,
they have no special meaning as a comment character, word component,
white space, string delimiter or number.
low | the first character in the range of ordinary characters. |
---|---|
hi | the last character in the range of ordinary characters. |
Indicates that the current token should be pushed back and returned again
the next time
nextToken()
is called.
Specifies that the character
ch
shall be treated as a quote
character.
ch | the character to be considered a quote character. |
---|
Specifies that all characters shall be treated as ordinary characters.
Specifies whether "slash-slash" (C++-style) comments shall be recognized. This kind of comment ends at the end of the line.
flag |
true
if
//
should be recognized as the start
of a comment,
false
otherwise.
|
---|
Specifies whether "slash-star" (C-style) comments shall be recognized. Slash-star comments cannot be nested and end when a star-slash combination is found.
flag |
true
if
/*
should be recognized as the start
of a comment,
false
otherwise.
|
---|
Returns the state of this tokenizer in a readable format.
Specifies that the characters in the range from
low
to
hi
shall be treated as whitespace characters by this tokenizer.
low | the first character in the range of whitespace characters. |
---|---|
hi | the last character in the range of whitespace characters. |
Specifies that the characters in the range from
low
to
hi
shall be treated as word characters by this tokenizer. A word consists of
a word character followed by zero or more word or number characters.
low | the first character in the range of word characters. |
---|---|
hi | the last character in the range of word characters. |