Change 17286 by jhi@alpha on 2002/06/18 20:19:25
Document the "Unicode in package/sub names" problem;
a microfix in toke.c towards the final goal (the fix
removes the need to have quotes around Unicode package
names when calling a method on them)
Affected files ...
.... //depot/perl/lib/utf8.pm#29 edit
.... //depot/perl/pod/perldelta.pod#460 edit
.... //depot/perl/pod/perltodo.pod#70 edit
.... //depot/perl/pod/perlunicode.pod#107 edit
.... //depot/perl/pod/perluniintro.pod#40 edit
.... //depot/perl/toke.c#435 edit
Differences ...
==== //depot/perl/lib/utf8.pm#29 (text) ====
Index: perl/lib/utf8.pm
--- perl/lib/utf8.pm#28~15947~ Tue Apr 16 05:34:36 2002
+++ perl/lib/utf8.pm Tue Jun 18 13:19:25 2002
@@ -57,9 +57,10 @@
Bytes in the source text that have their high-bit set will be treated
as being part of a literal UTF-8 character. This includes most
literals such as identifier names, string constants, and constant
-regular expression patterns. On EBCDIC platforms characters in
-the Latin 1 character set are treated as being part of a literal
-UTF-EBCDIC character.
+regular expression patterns.
+
+On EBCDIC platforms characters in the Latin 1 character set are
+treated as being part of a literal UTF-EBCDIC character.
=back
@@ -130,6 +131,19 @@
functions utf8::valid, utf8::encode, utf8::decode, utf8::upgrade,
and utf8::downgrade are always available, without a C<require utf8>
statement-- this may change in future releases.
+
+=head1 BUGS
+
+One can have Unicode in identifier names, but not in package/class or
+subroutine names. While some limited functionality towards this does
+exist as of Perl 5.8.0, that is more accidental than designed; use of
+Unicode for the said purposes is unsupported.
+
+One reason of this unfinishedness is its (currently) inherent
+unportability: since both package names and subroutine names may need
+to be mapped to file and directory names, the Unicode capability of
+the filesystem becomes important-- and there unfortunately aren't
+portable answers.
=head1 SEE ALSO
==== //depot/perl/pod/perldelta.pod#460 (text) ====
Index: perl/pod/perldelta.pod
--- perl/pod/perldelta.pod#459~17277~ Mon Jun 17 19:31:26 2002
+++ perl/pod/perldelta.pod Tue Jun 18 13:19:25 2002
@@ -3284,6 +3284,19 @@
./perl -Ilib ext/Time/HiRes/HiRes.t
+=head2 Unicode in package/class and subroutine names does not work
+
+One can have Unicode in identifier names, but not in package/class or
+subroutine names. While some limited functionality towards this does
+exist as of Perl 5.8.0, that is more accidental than designed; use of
+Unicode for the said purposes is unsupported.
+
+One reason of this unfinishedness is its (currently) inherent
+unportability: since both package names and subroutine names may
+need to be mapped to file and directory names, the Unicode capability
+of the filesystem becomes important-- and there unfortunately aren't
+portable answers.
+
=head2 UNICOS/mk
=over 4
==== //depot/perl/pod/perltodo.pod#70 (text) ====
Index: perl/pod/perltodo.pod
--- perl/pod/perltodo.pod#69~16824~ Mon May 27 13:58:48 2002
+++ perl/pod/perltodo.pod Tue Jun 18 13:19:25 2002
@@ -98,7 +98,25 @@
UTF-8 in package names and sub names? The first is problematic
because of the mapping to pathnames, ditto for the second one if
-one does autosplitting, for example.
+one does autosplitting, for example. Some of this works already
+in 5.8.0, but essentially it is unsupported. Constructs to consider,
+at the very least:
+
+ use utf8;
+ package UnicodePackage;
+ sub new { bless {}, shift };
+ sub UnicodeMethod1 { ... $_[0]->UnicodeMethod2(...) ... }
+ sub UnicodeMethod2 { ... } # in here caller(0) should contain Unicode
+ ...
+ package main;
+ my $x = UnicodePackage->new;
+ print ref $x, "\n"; # should be Unicode
+ $x->UnicodeMethod1(...);
+ my $y = UnicodeMethod3 UnicodePackage ...;
+
+In the above all I<UnicodeXxx> contain (identifier-worthy) characters
+beyond the code point 255, for example 256. Wherever package/class or
+subroutine names can be returned needs to be checked for Unicodeness.
=back
==== //depot/perl/pod/perlunicode.pod#107 (text) ====
Index: perl/pod/perlunicode.pod
--- perl/pod/perlunicode.pod#106~16849~ Tue May 28 18:16:23 2002
+++ perl/pod/perlunicode.pod Tue Jun 18 13:19:25 2002
@@ -37,7 +37,7 @@
(in string or regular expression literals, or in identifier names) on
ASCII-based machines or to recognize UTF-EBCDIC on EBCDIC-based
machines. B<These are the only times when an explicit C<use utf8>
-is needed.>
+is needed.> See L<utf8>.
You can also use the C<encoding> pragma to change the default encoding
of the data in your script; see L<encoding>.
==== //depot/perl/pod/perluniintro.pod#40 (text) ====
Index: perl/pod/perluniintro.pod
--- perl/pod/perluniintro.pod#39~17180~ Mon Jun 10 20:21:53 2002
+++ perl/pod/perluniintro.pod Tue Jun 18 13:19:25 2002
@@ -132,7 +132,7 @@
needed: if your Perl script itself is encoded in UTF-8, you can use
UTF-8 in your identifier names, and in string and regular expression
literals, by saying C<use utf8>. This is not the default because
-scripts with legacy 8-bit data in them would break.
+scripts with legacy 8-bit data in them would break. See L<utf8>.
=head2 Perl's Unicode Model
==== //depot/perl/toke.c#435 (text) ====
Index: perl/toke.c
--- perl/toke.c#434~16922~ Fri May 31 06:12:51 2002
+++ perl/toke.c Tue Jun 18 13:19:25 2002
@@ -1870,7 +1870,7 @@
* Method if it's "foo $bar"
* Not a method if it's really "print foo $bar"
* Method if it's really "foo package::" (interpreted as package->foo)
- * Not a method if bar is known to be a subroutne ("sub bar; foo bar")
+ * Not a method if bar is known to be a subroutine ("sub bar; foo bar")
* Not a method if bar is a filehandle or package, but is quoted with
* =>
*/
@@ -3913,6 +3913,10 @@
CLINE;
yylval.opval = (OP*)newSVOP(OP_CONST, 0, sv);
yylval.opval->op_private = OPpCONST_BARE;
+ /* UTF-8 package name? */
+ if (UTF && !IN_BYTES &&
+ is_utf8_string((U8*)SvPVX(sv), SvCUR(sv)))
+ SvUTF8_on(sv);
/* And if "Foo::", then that's what it certainly is. */
End of Patch.
