Package com.fathzer.soft.javaluator

Class Tokenizer

java.lang.Object
com.fathzer.soft.javaluator.Tokenizer
public class Tokenizer extends Object
A String tokenizer that accepts delimiters that are greater than one character.
Author:
Jean-Marc Astesana
See Also:

  • Constructor Details

    • Tokenizer

      public Tokenizer(List<String> delimiters)
      Constructor.
      By default, this tokenizer trims all the tokens.
      Parameters:
      delimiters - the delimiters of the tokenizer, usually, the operators symbols, the brackets and the function argument separator are used as delimiter in the string.

  • Method Details

    • isTrimTokens

      public boolean isTrimTokens()
      Tests whether this tokens trims the tokens returned by tokenize(String) method.
      Returns:
      true if tokens are trimmed.

    • setTrimTokens

      public void setTrimTokens(boolean trimTokens)
      Sets the trimTokens attribute.
      Parameters:
      trimTokens - true to have the tokens returned by tokenize(String) method trimmed.
      Note that empty tokens are always omitted by this class.

    • tokenize

      public Iterator<String> tokenize(String string)
      Converts a string into tokens.
      Example: The result for the expression "-1+min(10,3)" evaluated for a DoubleEvaluator is an iterator on "-", "1", "+", "min", "(", "10", ",", "3", ")".
      Parameters:
      string - The string to be split into tokens
      Returns:
      The tokens