I have a few comments regarding the tokenizer library.
1. The documentation says that char_delimiters_separator is default parameter
to 'tokenizer' template, and at the same time says that
'char_delimiters_separator' is deprecated. I think that's confusing and
default parameter should be changed to 'char-separator'.
2. The token interator description is very brief. Specifically, it does not
say what that iterator is usefull for, or when it's preferrable to direct use
of tokenizer. The only way to construct the iterator is via
make_token_iterator function which takes two interators as arguments. The
meaning of those arguments is not documented.
Lastly, the usage example
typedef token_iterator_generator<offset_separator>::type Iter;
Iter beg = make_token_iterator<string>(s.begin(),s.end(),f);
Iter end = make_token_iterator<string>(s.end(),s.end(),f);
for(;beg!=end;++beg){
appears to be just longer than tokenizer use:
typedef tokenizer< offset_separator > tok_t;
tok_t tok(s, f);
for(tok_t::iterator i = tok.begin(); i != tok.end(): ++i) {
so I *really* wonder what this iterator is for. OTOH, if it could be used
like:
for(token_iterator< offset_separator > i(s, f), e; i != e; ++i) {
}
it would be definitely simpler and easier. Is something like this possible?
3. The 'escaped_list_separator' template could have default argument for the
first parameter, "Char".
4. I almost always try to use tokenizer when values are separated by commas.
Believe me or not, I'm always confused as to which tokenizer function to use.
This time, I read all docs for char_separator and only then used escaped_list
separator -- which does the work out of the box. Maybe, a different name,
like "csv_with_escapes_separator" or "extended_csv_separator" would help?
It would make immediately clear what this separator is for.
- Volodya
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
