utl::Tokenizer Class Reference

Stream tokenizer. More...

#include <Tokenizer.h>

Inheritance diagram for utl::Tokenizer:

Public Member Functions
virtual void	copy (const Object &rhs)
	Copy another instance. More...
void	clear ()
	Re-set to initial configuration. More...
void	addTerminal (uint_t id, const String &regex)
	Add a terminal. More...
void	scan (TokenizerTokens &tokens, Stream *is, bool owner=true) const
	Scan an input stream. More...
Public Member Functions inherited from utl::Object
void	clear ()
	Revert to initial state. More...
virtual int	compare (const Object &rhs) const
	Compare with another object. More...
virtual void	vclone (const Object &rhs)
	Make an exact copy of another instance. More...
virtual void	steal (Object &rhs)
	"Steal" the internal representation from another instance. More...
virtual void	dump (Stream &os, uint_t level=uint_t_max) const
	Dump a human-readable representation of self to the given output stream. More...
void	dumpWithClassName (Stream &os, uint_t indent=4, uint_t level=uint_t_max) const
	Front-end for dump() that prints the object's class name. More...
virtual const Object &	getKey () const
	Get the key for this object. More...
bool	hasKey () const
	Determine whether or not the object has a key. More...
virtual const Object &	getProxiedObject () const
	Get the proxied object (= self if none). More...
virtual Object &	getProxiedObject ()
	Get the proxied object (= self if none). More...
virtual size_t	hash (size_t size) const
	Get the hash code for the object. More...
bool	_isA (const RunTimeClass *runTimeClass) const
	Determine whether self's class is a descendent of the given class. More...
virtual String	toString () const
	Return a string representation of self. More...
	operator String () const
	Conversion to String. More...
size_t	allocatedSize () const
	Get the total allocated size of this object. More...
virtual size_t	innerAllocatedSize () const
	Get the "inner" allocated size. More...
virtual void	addOwnedIt (const class FwdIt *it) const
	Notify self that it owns the given iterator. More...
virtual void	removeOwnedIt (const class FwdIt *it) const
	Notify self that the given owned iterator has been destroyed. More...
bool	operator< (const Object &rhs) const
	Less-than operator. More...
bool	operator<= (const Object &rhs) const
	Less-than-or-equal-to operator. More...
bool	operator> (const Object &rhs) const
	Greater-than operator. More...
bool	operator>= (const Object &rhs) const
	Greater-than-or-equal-to operator. More...
bool	operator== (const Object &rhs) const
	Equal-to operator. More...
bool	operator!= (const Object &rhs) const
	Unequal-to operator. More...
void	serializeIn (Stream &is, uint_t mode=ser_default)
	Serialize from an input stream. More...
void	serializeOut (Stream &os, uint_t mode=ser_default) const
	Serialize to an output stream. More...
virtual void	serialize (Stream &stream, uint_t io, uint_t mode=ser_default)
	Serialize to or from a stream. More...
void	serializeOutBoxed (Stream &os, uint_t mode=ser_default) const
	Serialize a boxed object to an output stream. More...

Additional Inherited Members
Static Public Member Functions inherited from utl::Object
static Object *	serializeInNullable (Stream &is, uint_t mode=ser_default)
	Serialize a nullptr-able object from an input stream. More...
static void	serializeOutNullable (const Object *object, Stream &os, uint_t mode=ser_default)
	Serialize a nullptr-able object to an output stream. More...
static void	serializeNullable (Object *&object, Stream &stream, uint_t io, uint_t mode=ser_default)
	Serialize a nullptr-able object to or from a stream. More...
static Object *	serializeInBoxed (Stream &is, uint_t mode=ser_default)
	Serialize a boxed object from an input stream. More...
static void	serializeBoxed (Object *&object, Stream &stream, uint_t io, uint_t mode=ser_default)
	Serialize a boxed object to or from a stream. More...

Detailed Description

Stream tokenizer.

A Tokenizer is configured by defining a set of "terminals" described by regular expressions. Tokenizer can then scan an input stream for strings that match one of these expressions (ignoring any intervening characters).

Author

Adam McKee

Definition at line 35 of file Tokenizer.h.

Member Function Documentation

copy()

virtual void utl::Tokenizer::copy ( const Object &  rhs )

virtual

Copy another instance.

When you override copy(), you should usually call the superclass's copy().

Parameters

rhs	object to copy

Reimplemented from utl::Object.

clear()

void utl::Tokenizer::clear

(

)

Re-set to initial configuration.

addTerminal()

void utl::Tokenizer::addTerminal	(	uint_t	id,
		const String &	regex
	)

Add a terminal.

Parameters

id	id
regex	regular expression

scan()

void utl::Tokenizer::scan	(	TokenizerTokens &	tokens,
		Stream *	is,
		bool	owner = true
	)		const

Scan an input stream.

Parameters

tokens	output tokens
is	input stream
owner	input stream ownership flag

---

The documentation for this class was generated from the following file:

• Tokenizer.h