Steffen Daode Nurpmeso <[email protected]> added the comment:
I wonder what this normalize_encoding() does! Here is a pretty standard
version of mine which is a bit more expensive but catches match more cases!
This is stripped, of course, and can be rewritten very easily to Python's needs
(i.e. using char[32] instead of char[11].
* @@li If a character is either ::s_char_is_space() or ::s_char_is_punct():
* @@li Replace with ASCII space (0x20).
* @@li Squeeze adjacent spaces to a single one.
* @@li Else if a character is ::s_char_is_alnum():
* @@li ::s_char_to_lower() characters.
* @@li Separate groups of alphas and digits with ASCII space (0x20).
* @@li Else discard character.
* E.g. "ISO_8859---1" becomes "iso 8859 1"
* and "ISO8859-1" also becomes "iso 8859 1".
s_textcodec_normalize_name(s_CString *_name) {
enum { C_NONE, C_WS, C_ALPHA, C_DIGIT } c_type = C_NONE;
char *name, c;
auto s_CString input;
s_cstring_swap(s_cstring_init(&input), _name);
_name = s_cstring_reserve(_name, 31, s_FAL0);
name = s_cstring_cstr(&input);
while ((c = *(name++)) != s_NUL) {
s_si8 sep = s_FAL0;
if (s_char_is_space(c) || s_char_is_punct(c)) {
if (c_type == C_WS)
continue;
c_type = C_WS;
c = ' ';
} else if (s_char_is_alpha(c)) {
sep = (c_type == C_DIGIT);
c_type = C_ALPHA;
c = s_char_to_lower(c);
} else if (s_char_is_digit(c)) {
sep = (c_type == C_ALPHA);
c_type = C_DIGIT;
} else
continue;
do
_name = s_cstring_append_char(_name, (sep ? ' ' : c));
while (--sep >= s_FAL0);
}
s_cstring_destroy(&input);
return _name;
}
----------
nosy: +sdaoden
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue11303>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
