StringTokenizer in java- Constructor, method with example

1 What is StringTokenizer?

A StringTokenizer is a class in Java. It is used to divide strings in form of token based on some delimiters.

It is defined in java.util.stringtokenizer

2 StringTokenizer Constructors

Sr No	Constructor and Description
1	StringTokenizer(String str) Construct a string tokenizer for given string
2	StringTokenizer(String str, String delim) Construct a string tokenizer for given string and generate tokens based on delimiters
3	StringTokenizer(String str, String delim, boolean returnDelims) Construct a string tokenizer for given string and generate tokens based on delimiters the returnDelims specify whether to include delim in token or not

Creating StringTokenizer object using first constructor

StringTokenizer st = new StringTokenizer("Arise awake and do not stop until the goal is reached");

1	StringTokenizer st = new StringTokenizer("Arise awake and do not stop until the goal is reached");

Creating StringTokenizer object using second constructor

StringTokenizer st = new StringTokenizer("Hello, how, are, you",",");

1	StringTokenizer st = new StringTokenizer("Hello, how, are, you",",");

3 StringTokenizer methods

A Java StringTokenizer countTokens() Method

This method is used to count the number of the tokens from current token position to the last token

Signature

public int countTokens()

Parameters

Return

Return the number of tokens available from specific token position

Example

package ebhor;

import java.util.StringTokenizer;

public class JavaStringTokenCount {

    public static void main(String[] args) {
        StringTokenizer st = new StringTokenizer("Arise awake and do not stop until the goal is reached");
        int tokenCount = st.countTokens();
        System.out.println("Total Numbers of tokens are " + tokenCount);
        st.nextToken();
        tokenCount = st.countTokens();
        System.out.println("Total Numbers of tokens are " + tokenCount);
    }

}

package ebhor;

import java.util.StringTokenizer;

public class JavaStringTokenCount {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Arise awake and do not stop until the goal is reached");

int tokenCount = st.countTokens();

System.out.println("Total Numbers of tokens are " + tokenCount);

st.nextToken();

tokenCount = st.countTokens();

System.out.println("Total Numbers of tokens are " + tokenCount);

}

Total Numbers of tokens are 11
Total Numbers of tokens are 10

1 2	Total Numbers of tokens are 11 Total Numbers of tokens are 10

Initially token position was before the first word “Arise” so on calling st.countToken() returns 11.
on calling st.nextToken() position of token is incresed by one and it is on “Arise” from this position new token count is 10.

import java.util.StringTokenizer;

public class TokenCount1 {

    public static void main(String[] args) {
        StringTokenizer st = new StringTokenizer("Arise awake and do not stop until the goal is reached");
        int tokenCount = st.countTokens();
        String token;
        System.out.println("Total Numbers of tokens are " + tokenCount);
        while (st.hasMoreTokens()) {
            token = st.nextToken();
            tokenCount = st.countTokens();
            System.out.println("Next Token " + token + " token count from this position " + tokenCount);
        }
    }

}

import java.util.StringTokenizer;

public class TokenCount1 {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Arise awake and do not stop until the goal is reached");

int tokenCount = st.countTokens();

String token;

System.out.println("Total Numbers of tokens are " + tokenCount);

while (st.hasMoreTokens()) {

token = st.nextToken();

tokenCount = st.countTokens();

System.out.println("Next Token " + token + " token count from this position " + tokenCount);

}

Total Numbers of tokens are 11
Next Token Arise token count from this position 10
Next Token awake token count from this position 9
Next Token and token count from this position 8
Next Token do token count from this position 7
Next Token not token count from this position 6
Next Token stop token count from this position 5
Next Token until token count from this position 4
Next Token the token count from this position 3
Next Token goal token count from this position 2
Next Token is token count from this position 1
Next Token reached token count from this position 0

Total Numbers of tokens are 11

Next Token Arise token count from this position 10

Next Token awake token count from this position 9

Next Token and token count from this position 8

Next Token do token count from this position 7

Next Token not token count from this position 6

Next Token stop token count from this position 5

Next Token until token count from this position 4

Next Token the token count from this position 3

Next Token goal token count from this position 2

Next Token is token count from this position 1

Next Token reached token count from this position 0

B Java StringTokenizer hasMoreElements() Method

This method checks for next token in StringTokenizer if the token exists then return true else return false. Work similar as
hasMoreToken()

Parameters

Return

Return true if next token exists else return false

Example

package ebhor;

import java.util.StringTokenizer;

public class StringTokenizer5 {
    public static void main(String[] args) {
        StringTokenizer st=new StringTokenizer("Java is secure language");
        boolean b=st.hasMoreElements();
        System.out.println("Has More Element :"+b);
   }
 }

package ebhor;

import java.util.StringTokenizer;

public class StringTokenizer5 {

public static void main(String[] args) {

StringTokenizer st=new StringTokenizer("Java is secure language");

boolean b=st.hasMoreElements();

System.out.println("Has More Element :"+b);

}

Result

Has More Element :true

1	Has More Element :true

At starting object st is pointing before the first word(Java). on calling st.hasMoreElements it is checkin that the first word java is there so it is returning true.

C Java StringTokenizer hasMoreToken() Method

This method is used with StringTokenizer object to check whether there is more token or not
For example
StringTokenizer st=new StringTokenizer(“Hello Friends”);
There are two tokens Hello and Friends. At starting hasMoreToken() point before the Hello Token so calling this method will return true at first.
To get next token we use nextToken().
After getting token Friends, again calling hasMoreToken() it return false.

Signature

public boolean hasMoreToken()

Parameters

Return

Return true if there is more token else return false

Example

public class StringTokenizerExample3 {

    public static void main(String[] args) {
        StringTokenizer st = new StringTokenizer("Hello Friends");
        boolean b = st.hasMoreTokens();
        System.out.println("Has more token " + b);
    }

}

public class StringTokenizerExample3 {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Hello Friends");

boolean b = st.hasMoreTokens();

System.out.println("Has more token " + b);

}

Result

Has more token true

1	Has more token true

St object is pointing before the Hello so calling st.hasMoreTokens() will always return true.

D Java StringTokenizer nextElement() Method

This method returns next token from stringTokenizer object this method work same as nextToken() only the return type of this method is object instead of String

Signature

public Object nextElement()

Parameters

Return

Return next token from StringTokenizer object

Throws

NoSuchElementException – If there is no token in StringTokenizer object then it throw this exception

Example

Lets see how to tokenize a string in java

package ebhor;
import java.util.StringTokenizer;

public class StringTokenizerExample2 {

    public static void main(String[] args) {
        StringTokenizer st = new StringTokenizer("Hello Friends");
        boolean hasMoreElement;
        Object nextElement;
        hasMoreElement = st.hasMoreElements();
        System.out.println("Has more element " + hasMoreElement);
        nextElement = st.nextElement();
        System.out.println("next element is " + nextElement);
        hasMoreElement = st.hasMoreElements();
        System.out.println("Has more element " + hasMoreElement);
        nextElement = st.nextElement();
        System.out.println("next element is " + nextElement);
          hasMoreElement = st.hasMoreElements();
        System.out.println("Has more element " + hasMoreElement);
    }
}

package ebhor;

import java.util.StringTokenizer;

public class StringTokenizerExample2 {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Hello Friends");

boolean hasMoreElement;

Object nextElement;

hasMoreElement = st.hasMoreElements();

System.out.println("Has more element " + hasMoreElement);

nextElement = st.nextElement();

System.out.println("next element is " + nextElement);

hasMoreElement = st.hasMoreElements();

System.out.println("Has more element " + hasMoreElement);

nextElement = st.nextElement();

System.out.println("next element is " + nextElement);

hasMoreElement = st.hasMoreElements();

System.out.println("Has more element " + hasMoreElement);

}

Has more element true
next element is Hello
Has more element true
next element is Friends
Has more element false

Has more element true

next element is Hello

Has more element true

next element is Friends

Has more element false

Another Example

package ebhor;
import java.util.StringTokenizer;
public class JavaStringTokenizer4 {

    public static void main(String[] args) {
        StringTokenizer st = new StringTokenizer("Hello Friends How Are You");
        while(st.hasMoreElements())
        {
            System.out.println("Next Element is " + st.nextElement());
        }
    }

}

package ebhor;

import java.util.StringTokenizer;

public class JavaStringTokenizer4 {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Hello Friends How Are You");

while(st.hasMoreElements())

{

System.out.println("Next Element is " + st.nextElement());

}

Next Element is Hello
Next Element is Friends
Next Element is How
Next Element is Are
Next Element is You

Next Element is Hello

Next Element is Friends

Next Element is How

Next Element is Are

Next Element is You

E Java StringTokenizer nextToken() Method

This method returns next token from stringTokenizer object

Signature

public String nextToken()

Parameters

Return

Return next token from StringTokenizer object

Throws

NoSuchElementException – If there is no token in StringTokenizer object then it throw this exception

Example

public class StringTokenizerExample1 {

    public static void main(String[] args) {

        StringTokenizer st = new StringTokenizer("Hello Friends");
        boolean moreToken;
        String nextToken;
        moreToken = st.hasMoreTokens();
        nextToken = st.nextToken();
        System.out.println("Has more Token " + moreToken + " next token is " + nextToken);
        moreToken = st.hasMoreTokens();
        nextToken = st.nextToken();
        System.out.println("Has more Token " + moreToken + " next token is " + nextToken);
        moreToken = st.hasMoreTokens();
        System.out.println("Has more Token " + moreToken);
    }

}

public class StringTokenizerExample1 {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Hello Friends");

boolean moreToken;

String nextToken;

moreToken = st.hasMoreTokens();

nextToken = st.nextToken();

System.out.println("Has more Token " + moreToken + " next token is " + nextToken);

moreToken = st.hasMoreTokens();

nextToken = st.nextToken();

System.out.println("Has more Token " + moreToken + " next token is " + nextToken);

moreToken = st.hasMoreTokens();

System.out.println("Has more Token " + moreToken);

}

public class StringTokenizerExample1 {

    public static void main(String[] args) {

        StringTokenizer st = new StringTokenizer("Hello Friends");
        boolean moreToken;
        String nextToken;
        while (moreToken = st.hasMoreTokens()) {
            nextToken = st.nextToken();
            System.out.println("Has more Token " + moreToken + " next token is " + nextToken);
        }

    }

public class StringTokenizerExample1 {

public static void main(String[] args) {

StringTokenizer st = new StringTokenizer("Hello Friends");

boolean moreToken;

String nextToken;

while (moreToken = st.hasMoreTokens()) {

nextToken = st.nextToken();

System.out.println("Has more Token " + moreToken + " next token is " + nextToken);

}

Result

Has more Token true next token is Hello
Has more Token true next token is Friends
Has more Token false

Has more Token true next token is Hello

Has more Token true next token is Friends

Has more Token false

Has more Token true next token is Hello
Has more Token true next token is Friends

1 2	Has more Token true next token is Hello Has more Token true next token is Friends