com.fathzer.soft.javaluator

Class Tokenizer

java.lang.Object

com.fathzer.soft.javaluator.Tokenizer

public class Tokenizer
extends Object

A String tokenizer that accepts delimiters that are greater than one character.

Author:

Jean-Marc Astesana

See Also:

License information (Apache 2)

Constructor Summary

Constructors

Constructor and Description
Tokenizer(List<String> delimiters) Constructor.

Method Summary

Modifier and Type	Method and Description
boolean	isTrimTokens() Tests whether this tokens trims the tokens returned by tokenize(String) method.
void	setTrimTokens(boolean trimTokens) Sets the trimTokens attribute.
Iterator<String>	tokenize(String string) Converts a string into tokens.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(List<String> delimiters)

Constructor.
By default, this tokenizer trims all the tokens.

Parameters:

delimiters - the delimiters of the tokenizer, usually, the operators symbols, the brackets and the function argument separator are used as delimiter in the string.

Method Detail

isTrimTokens

public boolean isTrimTokens()

Tests whether this tokens trims the tokens returned by tokenize(String) method.

Returns:

true if tokens are trimmed.

setTrimTokens

public void setTrimTokens(boolean trimTokens)

Sets the trimTokens attribute.

Parameters:

trimTokens - true to have the tokens returned by tokenize(String) method trimmed.
Note that empty tokens are always omitted by this class.

tokenize

public Iterator<String> tokenize(String string)

Converts a string into tokens.
Example: The result for the expression "-1+min(10,3)" evaluated for a DoubleEvaluator is an iterator on "-", "1", "+", "min", "(", "10", ",", "3", ")".

Parameters:

string - The string to be split into tokens

Returns:

The tokens