3.6 Tokenizer

You can use the class RWCTokenizer to break up a string into tokens separated by arbitrary white spaces. Here's an example:

#include <rw/ctoken.h>
#include <rw/cstring.h>
#include <rw/rstream.h>

main(){
  RWCString a("a string with five tokens");

  RWCTokenizer next(a);

  int i = 0;

  // Advance until the null string is returned:
  while( !next().isNull() ) i++;

  cout << i << endl;
  return 0;
}

Program Output:

5

This program counts the number of tokens in the string. The function call operator for class RWCTokenizer has been overloaded to mean "advance to the next token and return it as an RWCSubString," much like other Tools.h++ iterators. When there are no more tokens, it returns the null substring. Class RWCSubString has a member function isNull() which returns TRUE if the substring is the null substring. Hence, the loop is broken. See the Class Reference under RWCTokenizer for details.