Text Processing

Detailed Description

This set of classes and functions extends the string and localization support of the C++ standard library to work with unicode characters and strings. A unicode character type and string class (a specialization of std::basic_string) can be used to hold unicode text. A set of functions allows to transform and classify individual characters. Text can be converted e.g. between different encodings using i/o streams and text codecs. A regular expression class allows to search and match patterns in unicode strings. Localization facets are available for the systems which support standard C++ locales.

One of the most common standards for character encoding is the ASCII standard. Each character is encoded using 7 bits of a byte, so 128 different characters can be addressed. Reading and writing ASCII characters is straightforward, because each character is stored in exactly one byte. The builtin C++ type char can be used to represent ASCII characters. The draw-back of ASCII, of course, is the small character set of only 128 characters. There are a lot more characters than that in the languages all around the world.

This problem was addressed by the Unicode standard, which was created to make every known character of the world available in a single character table. Each character has a defined position in the table, a so-called code point. The unicode table contains 0x10FFFF entries at the moment, so a 32 bit type is required to represent a raw unicode character.

The UTF-8 encoding was introduced to store unicode characters in byte sequences, which are compatible to classic null-terminated C strings. One unicode character is encoded into a byte sequence of 1 or more bytes. Further, the characters are encoded such that a character in 7-bit ASCII has the exact same value as in UTF-8, so any valid ASCII text is valid UTF-8 encoded text. This demonstrates the difference between encodings and character types. ASCII and UTF-8 can both be represented by sequences of the character type char, but their values are interpreted according to the encoding. Besides UTF-8 encoding, many more encodings have been developed, for example Latin-1, UTF-16 or, in the broadest sense, Base64.

Classes
class	String
	Unicode capable basic_string. More...
class	TextCodec< CharT, ByteT >
	Converts between character encodings. More...
class	Base64Codec
	A codec for base-64 encoding. More...
class	InvalidRegex
	Invalid regular expression. More...
class	Regex
	Regular Expressions for Unicode Strings. More...
class	RegexSMatch
	Result of a regular expression match. More...
struct	Char
	Unicode character type. More...
class	StringBuffer
	Unicode string stream buffer. More...
class	IStringStream
	Unicode string input stream. More...
class	OStringStream
	Unicode string output stream. More...
class	StringStream
	Unicode string stream. More...
class	BasicTextBuffer< CharT, ByteT >
	Converts character sequences with different encodings. More...
class	BasicTextIStream< CharT, ByteT >
	Converts character sequences using a codec. More...
class	BasicTextOStream< CharT, ByteT >
	Converts character sequences using a codec. More...
class	BasicTextStream< CharT, ByteT >
	Converts character sequences using a codec. More...
class	Utf16BECodec
	UTF-16 big endian codec. More...
class	Utf16LECodec
	UTF-16 little endian codec. More...
class	Utf32BECodec
	UTF-32 big endian codec. More...
class	Utf32LECodec
	UTF-32 little endian codec. More...
class	Utf8Appender
	UTF-8 string output iterator. More...
class	Utf8Convert
	UTF-8 string converter. More...
class	Utf8Codec
	Convert between unicode and UTF-8. More...
class	TextBuffer
	Converts character sequences with different encodings. More...
class	TextIStream
	Text input stream for unicode character conversion. More...
class	TextOStream
	Text output stream for unicode character conversion. More...
class	TextStream
	Text stream for unicode character conversion. More...