Change 14900 by jhi@alpha on 2002/02/27 19:06:39
Update the UTF-8 explanation table.
Affected files ...
.... //depot/perl/pod/perlunicode.pod#76 edit
.... //depot/perl/utf8.h#55 edit
Differences ...
==== //depot/perl/pod/perlunicode.pod#76 (text) ====
Index: perl/pod/perlunicode.pod
--- perl/pod/perlunicode.pod.~1~ Wed Feb 27 12:15:05 2002
+++ perl/pod/perlunicode.pod Wed Feb 27 12:15:05 2002
@@ -700,18 +700,23 @@
require 4 bytes), byteorder independent encoding. For ASCII, UTF-8 is
transparent (and we really do mean 7-bit ASCII, not another 8-bit encoding).
-The following table is from Unicode 3.1.
+The following table is from Unicode 3.2.
Code Points 1st Byte 2nd Byte 3rd Byte 4th Byte
- U+0000..U+007F 00..7F���
- U+0080..U+07FF C2..DF 80..BF���
+ U+0000..U+007F 00..7F
+ U+0080..U+07FF C2..DF 80..BF
U+0800..U+0FFF E0 A0..BF 80..BF��
- U+1000..U+FFFF E1..EF 80..BF 80..BF��
+ U+1000..U+CFFF E1..EC 80..BF 80..BF��
+ U+D000..U+D7FF ED 80..9F 80..BF��
+ U+D800..U+DFFF ******* ill-formed *******
+ U+E000..U+FFFF EE..EF 80..BF 80..BF��
U+10000..U+3FFFF F0 90..BF 80..BF 80..BF
U+40000..U+FFFFF F1..F3 80..BF 80..BF 80..BF
U+100000..U+10FFFF F4 80..8F 80..BF 80..BF
+Note the A0..BF in U+0800..U+0FFF, the 80..9F in U+D000...U+D7FF,
+the 90..BF in U+10000..U+3FFFF, and the 80...8F in U+100000..U+10FFFF.
Or, another way to look at it, as bits:
Code Points 1st Byte 2nd Byte 3rd Byte 4th Byte
@@ -722,7 +727,7 @@
00000dddccccccbbbbbbaaaaaa 11110ddd 10cccccc 10bbbbbb 10aaaaaa
As you can see, the continuation bytes all begin with C<10>, and the
-leading bits of the start byte tells how many bytes the are in the
+leading bits of the start byte tell how many bytes the are in the
encoded character.
=item
==== //depot/perl/utf8.h#55 (text) ====
Index: perl/utf8.h
--- perl/utf8.h.~1~ Wed Feb 27 12:15:05 2002
+++ perl/utf8.h Wed Feb 27 12:15:05 2002
@@ -63,20 +63,43 @@
/*
- The following table is from Unicode 3.1.
+ The following table is from Unicode 3.2.
Code Points 1st Byte 2nd Byte 3rd Byte 4th Byte
U+0000..U+007F 00..7F���
U+0080..U+07FF C2..DF 80..BF���
U+0800..U+0FFF E0 A0..BF 80..BF��
- U+1000..U+FFFF E1..EF 80..BF 80..BF��
+ U+1000..U+CFFF E1..EC 80..BF 80..BF��
+ U+D000..U+D7FF ED 80..9F 80..BF��
+ U+D800..U+DFFF ******* ill-formed *******
+ U+E000..U+FFFF EE..EF 80..BF 80..BF��
U+10000..U+3FFFF F0 90..BF 80..BF 80..BF
U+40000..U+FFFFF F1..F3 80..BF 80..BF 80..BF
U+100000..U+10FFFF F4 80..8F 80..BF 80..BF
+Note the A0..BF in U+0800..U+0FFF, the 80..9F in U+D000...U+D7FF,
+the 90..BF in U+10000..U+3FFFF, and the 80...8F in U+100000..U+10FFFF.
+
*/
+/*
+ Another way to look at it, as bits:
+
+ Code Points 1st Byte 2nd Byte 3rd Byte 4th Byte
+
+ 0aaaaaaa 0aaaaaaa
+ 00000bbbbbaaaaaa 110bbbbb 10aaaaaa
+ ccccbbbbbbaaaaaa 1110cccc 10bbbbbb 10aaaaaa
+ 00000dddccccccbbbbbbaaaaaa 11110ddd 10cccccc 10bbbbbb 10aaaaaa
+
+As you can see, the continuation bytes all begin with C<10>, and the
+leading bits of the start byte tell how many bytes the are in the
+encoded character.
+
+*/
+
+
#define UNI_IS_INVARIANT(c) (((UV)c) < 0x80)
#define UTF8_IS_INVARIANT(c) UNI_IS_INVARIANT(NATIVE_TO_UTF(c))
#define NATIVE_IS_INVARIANT(c) UNI_IS_INVARIANT(NATIVE_TO_ASCII(c))
End of Patch.
