Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-19 Thread Aaron Crane
David Green david.gr...@telus.net wrote:
 Maybe setting $*CWD just calls chdir() under the hood?  Same implementation,
 brand new shiny Perl-style interface!

I don't think that's a good idea.  Suppose you have code like this:

  $*CWD = '/some/absolute/path';
  $*CWD = '../relative/path';
  my $cwd = $*CWD;

Assuming that no exception was thrown, should $cwd now be
'/some/absolute/relative/path' or '../relative/path' ?  If the former,
you've broken the user's reasonable expectation that a value assigned
to a variable can be retrieved by looking in the same variable.  If
the latter, $*CWD is a relative path name that doesn't exist in the
current directory.

Now that I think about it, I'm having doubts about whether it makes
sense to have a $*CWD variable at all.  Suppose the current directory
is /a/b/c, and a different process renames /a/b to /surprise.  Should
reading $*CWD reflect that?  Calling a POSIX-like getcwd() would do,
but that's surprising in just the same way.  But if $*CWD doesn't
reflect external changes, you can't expect to be able to opendir its
value; at that point, it doesn't seem very useful.

If the motivation for $*CWD is simply so that it can be temporized,
perhaps it would be better to provide a more direct method of doing
that (preferably using fchdir where available).

-- 
Aaron Crane ** http://aaroncrane.co.uk/


r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread pugs-commits
Author: wayland
Date: 2009-08-18 09:24:07 +0200 (Tue, 18 Aug 2009)
New Revision: 28017

Modified:
   docs/Perl6/Spec/S02-bits.pod
   docs/Perl6/Spec/S16-io.pod
   docs/Perl6/Spec/S28-special-names.pod
   docs/Perl6/Spec/S32-setting-library/IO.pod
Log:
[S02] Changed :io to :p and :path
[S16] Documented p{/path/to/file}
[S28] Made $*CWD have type Path instead of Str
[S32/IO] Many changes, including:
* Changed IO::FSNode into Path
* Merged most of IO::DirectoryNode and all of IO::LinkNode into Path (but still 
need to 
  merge IO::FileNode)
* Moved remnants of IO::DirectoryNode to IO::Directory
It's not finished yet, but I thought I'd commit anyway.  I'll keep working on 
it.  


Modified: docs/Perl6/Spec/S02-bits.pod
===
--- docs/Perl6/Spec/S02-bits.pod2009-08-17 21:09:32 UTC (rev 28016)
+++ docs/Perl6/Spec/S02-bits.pod2009-08-18 07:24:07 UTC (rev 28017)
@@ -2834,7 +2834,7 @@
 :subst  Parse as substitution
 :trans  Parse as transliteration
 :code   Quasiquoting
-:io Return an IO::FSNode
+:p  :path   Return a Path object
 
 You may omit the first colon by joining an initial CQ, Cq, or Cqq with
 a single short form adverb, which produces forms like:

Modified: docs/Perl6/Spec/S16-io.pod
===
--- docs/Perl6/Spec/S16-io.pod  2009-08-17 21:09:32 UTC (rev 28016)
+++ docs/Perl6/Spec/S16-io.pod  2009-08-18 07:24:07 UTC (rev 28017)
@@ -82,17 +82,55 @@
 
 The use of filenames requires a special quoting syntax.  It works as follows:
 
-q:io{/path/to/file}
+p{/path/to/file}
+q:p{/path/to/file}
 
+Both of the above result in the same thing.  
+
 The quote characters can be any of the usual ones, although / is probably a 
bad choice
 for filenames.  
 
-The code shown above returns an IO::FSNode object (or a descendant thereof).  
+The code shown above returns a Path object (or a descendant thereof).  
 
 Naturally you can also ask for interpolation in filenames:
 
-qq:io{$directory/$file}
+p:qq{$directory/$file}
+qq:p{$directory/$file}
 
+There are a number of special adverbs that can be applied to the file quoting 
+operator.  Most of these are filesystem-specific.  They confine what can be
+included in a filename.  
+
+=head3 Default constraints
+
+The default p{} only allows / as separator and does not allow path elements 
+to contain
+characters that won't work on modern Windows and Unix like \ / ? % * : |   ,
+etc. The reason for this is that portable paths are the default. If
+platform/filesystem specific behavior is really needed it should be shown in 
+the code by applying different sets of constraints (see below).  
+
+=head3 Windows-style constraints
+
+We allow windows style paths so converting and maintaining code on this
+platform is not a pain.
+
+my Path $path = p:win{C:\Program Files\MS Access\file.file};
+
+Note that this specifically excludes the backslash quoting usually used with 
+q{}.  
+
+=head3 Unix-style constraints
+
+For Unix specific behavior we have a p:unix{} literal.  Here the only
+limits are what is defined by the locale and the filesystem type. So we won't 
+be able to write full Unicode if locale is set to Latin1. 
+
+my Path $path = p:unix{/usr/src/bla/myfile?:%.file};
+
+And for cases where this is a problem p:bin{} can be used as no checking is
+done here, other than assuming that / is the separator.  
+
 =head1 Name Services
 
 =head2 User role

Modified: docs/Perl6/Spec/S28-special-names.pod
===
--- docs/Perl6/Spec/S28-special-names.pod   2009-08-17 21:09:32 UTC (rev 
28016)
+++ docs/Perl6/Spec/S28-special-names.pod   2009-08-18 07:24:07 UTC (rev 
28017)
@@ -68,7 +68,7 @@
  Variable  Spec  Type Description
       ---
 
- @_   # ???
+ @_   # ??? (FIX)
  $!S04# Current Exception object
  $/S05   Match# Last match
  $0, $1, $2S05   Str  # First captured value from match: $/[0]
@@ -81,7 +81,7 @@
  $?CLASS   S02   Class# current class
  @=COMMENT(S26)   # All the comment blocks in the file
  %?CONFIGHash of XXX  # configuration hash XXX What does this 
do?
- $*CWD   Str  # current working directory
+ $*CWD   Path # current working directory
  $=DATA   (S26)  IO   # data block handle (=begin DATA ... 
=end)
  @=DATA   (S26)  Array# Same as above, but array
  %?DEEPMAGIC   S13   Hash of XXX  # Controls the mappings of magical names 
to sub definitions

Modified: docs/Perl6/Spec/S32-setting-library/IO.pod

Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Nicholas Clark
On Tue, Aug 18, 2009 at 09:24:08AM +0200, pugs-comm...@feather.perl6.nl wrote:

 +=head3 Default constraints
 +
 +The default p{} only allows / as separator and does not allow path 
 elements 
 +to contain
 +characters that won't work on modern Windows and Unix like \ / ? % * : |   
 ,
 +etc. The reason for this is that portable paths are the default. If
 +platform/filesystem specific behavior is really needed it should be shown in 
 +the code by applying different sets of constraints (see below).  

Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy
of enforced portable filenames by default, it should (at least) also exclude
- as the first character, and forbid more than one . in a filename.

(Also, I suspect, ; and . are illegal in directory names.
 And on VMS catfile() and catdir() *are* distinct operations, so it's important
 to know if the right-most argument is a file or a directory)

Nicholas Clark


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Carlin Bingham
2009/8/18  pugs-comm...@feather.perl6.nl:
 Author: wayland
 Date: 2009-08-18 09:24:07 +0200 (Tue, 18 Aug 2009)
 New Revision: 28017

 +=item chdir FILENAME
 +Xchdir Xcd
 +
 +=item chdir
 +
 +Gone, just set $*CWD (which throws an exception if it fails).
 +

chdir provides functionality that would be quite convoluted to mimic
through manually setting $*CWD, such as changing to a relative
directory.

Unless there are ways to do it that I'm missing, this doesn't seem
like a good idea.


--
Carlin


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Jan Ingvoldstad
On Tue, Aug 18, 2009 at 10:01 AM, Nicholas Clarkn...@ccl4.org wrote:
 On Tue, Aug 18, 2009 at 09:24:08AM +0200, pugs-comm...@feather.perl6.nl wrote:

 +=head3 Default constraints
 +
 +The default p{} only allows / as separator and does not allow path 
 elements
 +to contain
 +characters that won't work on modern Windows and Unix like \ / ? % * : |  
  ,
 +etc. The reason for this is that portable paths are the default. If
 +platform/filesystem specific behavior is really needed it should be shown in
 +the code by applying different sets of constraints (see below).

 Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy
 of enforced portable filenames by default, it should (at least) also exclude
 - as the first character, and forbid more than one . in a filename.

And, as I mentioned in an earlier post during the discussion, the
restrictions for Windows are numerous:

http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx

Enforcing truly portable filenames is unrealistic, I think, but
having a POSIX-checking default is a good thing.
-- 
Jan


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Nicholas Clark
On Tue, Aug 18, 2009 at 10:36:45AM +0200, Jan Ingvoldstad wrote:
 On Tue, Aug 18, 2009 at 10:01 AM, Nicholas Clarkn...@ccl4.org wrote:
  On Tue, Aug 18, 2009 at 09:24:08AM +0200, pugs-comm...@feather.perl6.nl 
  wrote:
 
  +=head3 Default constraints
  +
  +The default p{} only allows / as separator and does not allow path 
  elements
  +to contain
  +characters that won't work on modern Windows and Unix like \ / ? % * : | 
,
  +etc. The reason for this is that portable paths are the default. If
  +platform/filesystem specific behavior is really needed it should be shown 
  in
  +the code by applying different sets of constraints (see below).
 
  Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a 
  policy
  of enforced portable filenames by default, it should (at least) also exclude
  - as the first character, and forbid more than one . in a filename.
 
 And, as I mentioned in an earlier post during the discussion, the
 restrictions for Windows are numerous:
 
 http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx

Oh gosh yes. I forgot. AUX.TXT
And all the the other CP/M device file names, with our without extensions...


[And of course, IIRC, DOS filenames can't be more than 64 characters. Which
means that your code thinks that it knows what it's doing by stripping of
the drive letter and allowing 62 in the rest of the pathname. Except that
I remember university servers with drives UX: and UY:, which always made me
wonder if there was a buffer overflow waiting to happen...]

 Enforcing truly portable filenames is unrealistic, I think, but
 having a POSIX-checking default is a good thing.

What does POSIX enforce? ASCII NUL terminated, multiple adjacent / characters
fold to 1, except at the start, where // is special, but /// etc aren't?
And anything else goes?

Nicholas Clark


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread David Green

On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote:

chdir provides functionality that would be quite convoluted to mimic
through manually setting $*CWD, such as changing to a relative
directory.


Maybe setting $*CWD just calls chdir() under the hood?  Same  
implementation, brand new shiny Perl-style interface!



-David



Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Jan Ingvoldstad
On Tue, Aug 18, 2009 at 10:58 AM, Nicholas Clarkn...@ccl4.org wrote:
 Oh gosh yes. I forgot. AUX.TXT
 And all the the other CP/M device file names, with our without extensions...


 [And of course, IIRC, DOS filenames can't be more than 64 characters. Which
 means that your code thinks that it knows what it's doing by stripping of
 the drive letter and allowing 62 in the rest of the pathname. Except that
 I remember university servers with drives UX: and UY:, which always made me
 wonder if there was a buffer overflow waiting to happen...]

Yes, lovely, isn't it?

 What does POSIX enforce? ASCII NUL terminated, multiple adjacent / characters
 fold to 1, except at the start, where // is special, but /// etc aren't?
 And anything else goes?

POSIX doesn't enforce much. I used our friendly do-no-evil empire, and
found this:

http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html#standards

And particularly from «2.2. Standards permit the exclusion of bad filenames»:

«I then examined the Portable Filename Character Set, defined in 3.276
(“Portable Filename Character Set”); this turns out to be just A-Z,
a-z, 0-9, period, underscore, and hyphen (aka the dash
character). So it’s perfectly okay for a POSIX system to reject a
non-portable filename due to it having “odd” characters or a leading
hyphen.»
-- 
Jan


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Jan Ingvoldstad
On Tue, Aug 18, 2009 at 11:04 AM, David Greendavid.gr...@telus.net wrote:
 On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote:

 chdir provides functionality that would be quite convoluted to mimic
 through manually setting $*CWD, such as changing to a relative
 directory.

 Maybe setting $*CWD just calls chdir() under the hood?  Same implementation,
 brand new shiny Perl-style interface!

It may seem cool, but I don't like secondary effects like that. They
break the principle of least surprise.
-- 
Jan


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Timothy S. Nelson

On Tue, 18 Aug 2009, David Green wrote:


On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote:

chdir provides functionality that would be quite convoluted to mimic
through manually setting $*CWD, such as changing to a relative
directory.


Maybe setting $*CWD just calls chdir() under the hood?  Same implementation, 
brand new shiny Perl-style interface!


	That was my intent, but we had some discussions on IRC about the whys 
and wherefores, and it will return as soon as I do my next commit (hopefully 
sometime within the next 4-5 hours).  Unless you can think of a good way to do 
relative paths.


:)


-
| Name: Tim Nelson | Because the Creator is,|
| E-mail: wayl...@wayland.id.au| I am   |
-

BEGIN GEEK CODE BLOCK
Version 3.12
GCS d+++ s+: a- C++$ U+++$ P+++$ L+++ E- W+ N+ w--- V- 
PE(+) Y+++ PGP-+++ R(+) !tv b++ DI D G+ e++ h! y-

-END GEEK CODE BLOCK-



Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Mark J. Reed
 On Tue, Aug 18, 2009 at 11:04 AM, David Greendavid.gr...@telus.net wrote:
 On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote:

 chdir provides functionality that would be quite convoluted to mimic
 through manually setting $*CWD, such as changing to a relative
 directory.

 Maybe setting $*CWD just calls chdir() under the hood?  Same implementation,
 brand new shiny Perl-style interface!

If $*CWD is really a Path object and not a Str, then it should be easy
to use mutator methods to change to a relative directory and do other
chdir()ish things.  Say, concatenation works in terms of path
components, for instance:

$*CWD ~= $subdir;   # chdir($subdir)

With a method for getting the parent:

given $*CWD { $_ = $_.up ~ $sibling }   # chdir(../$sibling)

and so on.  My favorite kshism is cd old new which does a
search/replace on the current working directory; the bash equivalent
cd ${PWD/old/new} which is not quite as handy.  $*CWD could make
that simple, too.

-- 
Mark J. Reed markjr...@gmail.com


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Timothy S. Nelson

On Tue, 18 Aug 2009, Mark J. Reed wrote:


On Tue, Aug 18, 2009 at 11:04 AM, David Greendavid.gr...@telus.net wrote:

On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote:


chdir provides functionality that would be quite convoluted to mimic
through manually setting $*CWD, such as changing to a relative
directory.


Maybe setting $*CWD just calls chdir() under the hood?  Same implementation,
brand new shiny Perl-style interface!


If $*CWD is really a Path object and not a Str, then it should be easy
to use mutator methods to change to a relative directory and do other
chdir()ish things.  Say, concatenation works in terms of path
components, for instance:

$*CWD ~= $subdir;   # chdir($subdir)

With a method for getting the parent:

given $*CWD { $_ = $_.up ~ $sibling }   # chdir(../$sibling)

and so on.  My favorite kshism is cd old new which does a
search/replace on the current working directory; the bash equivalent
cd ${PWD/old/new} which is not quite as handy.  $*CWD could make
that simple, too.


	It's not in the revised spec, but I think that, even though 
we've revived chdir, we should still have it so that changing $*CWD will do a 
chdir under the hood.


	My intent is that Path should be able to act as a string, at least to 
a certain extent, and should also be able to act as an array of Str.  So 
Mark's second example could be done as:


pop $*CWD; push $*CWD $sibling

HTH,


-
| Name: Tim Nelson | Because the Creator is,|
| E-mail: wayl...@wayland.id.au| I am   |
-

BEGIN GEEK CODE BLOCK
Version 3.12
GCS d+++ s+: a- C++$ U+++$ P+++$ L+++ E- W+ N+ w--- V- 
PE(+) Y+++ PGP-+++ R(+) !tv b++ DI D G+ e++ h! y-

-END GEEK CODE BLOCK-


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Troels Liebe Bentsen
 Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy
 of enforced portable filenames by default, it should (at least) also exclude
 - as the first character, and forbid more than one . in a filename.

 And, as I mentioned in an earlier post during the discussion, the
 restrictions for Windows are numerous:

 http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx

 Enforcing truly portable filenames is unrealistic, I think, but
 having a POSIX-checking default is a good thing.

My idea with portable by default was only portability for modern Unix and
modern Windows. So DOS and VMS limitations would not apply. The problem of
enforcing truly portable filenames is that the files names get too
restrictive and for most applications targeting 98% of systems out there would
be enough.

With modern Unix/Windows I'm thinking about systems that support and use UCS2
or UTF8 and where . or other common characters does not have special meaning
for the filesystem.

Using POSIX as the basis would properly not be enough as it's either to lax or
to strict.

I think Windows NTFS/VFAT32 + Kernel limits on characters might be a good basis:

http://en.wikipedia.org/wiki/Filename

 * So paths can be up to 259 characters long
 * No characters in the range 0x01-0x1F
 * No control characters
 * Non of the following characters:  * :   ? \ / |.
 * Directory or filename up to 255 characters long
 * Only Unicode that fits in UCS-2(the 16bit subset)

Any system specific restrictions would not apply like AUX, CLOCK$, COM, etc.
would not apply and the programmer would be allow to shoot him/herself in the
foot.

Besides that, a simple check on Unix for what the locale is set to might also be
nice, so we don't write UTF8 files on a filesystem where the rest for the files
are in Latin1.

Regards Troels.


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Carlin Bingham
2009/8/18 Timothy S. Nelson wayl...@wayland.id.au:
 On Tue, 18 Aug 2009, Mark J. Reed wrote:

       It's not in the revised spec, but I think that, even though we've
 revived chdir, we should still have it so that changing $*CWD will do a
 chdir under the hood.


While in the spirit of TIMTOWTDI, having a magic variable that
behaves differently from other variables when it's being set would be
rather odd.

--
Carlin


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Jan Ingvoldstad
On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk wrote:

 My idea with portable by default was only portability for modern Unix and
 modern Windows. So DOS and VMS limitations would not apply. The problem of
 enforcing truly portable filenames is that the files names get too
 restrictive and for most applications targeting 98% of systems out there would
 be enough.

That's a decent enough point, but it may be unwise to ignore legacy
systems that where Perl 5 is in common use, unless we want to shed
that userbase. (Mark this down as a I don't know, and I don't have a
stake in it, but…)

 With modern Unix/Windows I'm thinking about systems that support and use UCS2
 or UTF8 and where . or other common characters does not have special meaning
 for the filesystem.

We also need to keep in mind the Unicode problems between certain
unixy platforms (i.e. MacOS X vs. most if not all the rest).

If I recall correctly, the internal Unicode format chosen for Perl 6
is incompatible with MacOS X, because MacOS X implemented Unicode
support at a time when the standard as we know it today wasn't
finalized.

This has bearing on filenames, and MacOS X isn't a small enough
platform that it can simply be ignored, either.

 Besides that, a simple check on Unix for what the locale is set to might also 
 be
 nice, so we don't write UTF8 files on a filesystem where the rest for the 
 files
 are in Latin1.

The locale doesn't say what format the filenames are on the
filesystem, though, merely the current user's language preferences may
be.

That can't really be trusted, at least not for filename comparison
purposes. But I agree that it's sensible to default to the locale's
settings regarding character encoding.
-- 
Jan


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Troels Liebe Bentsen
On Tue, Aug 18, 2009 at 13:10, Jan Ingvoldstadfrett...@gmail.com wrote:
 On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk 
 wrote:

 My idea with portable by default was only portability for modern Unix and
 modern Windows. So DOS and VMS limitations would not apply. The problem of
 enforcing truly portable filenames is that the files names get too
 restrictive and for most applications targeting 98% of systems out there 
 would
 be enough.

 That's a decent enough point, but it may be unwise to ignore legacy
 systems that where Perl 5 is in common use, unless we want to shed
 that userbase. (Mark this down as a I don't know, and I don't have a
 stake in it, but…)

I completely agree and we might make a p:posix{} or p:strict{} to handle that.

But one thing to remember is that having the default p{} allow characters and
formats that is not supported on VMS or DOS, only means the programmer won't
get a compiler error. In other words he is only a little better off than using
a normal string or Q{}. Also in cases where systems has special requirements
the local version should be used so, fx. VMS would have p:vms{} and DOS
p:dos{}.

The defaults should handle most cases and limiting path's to only ASCII might
be a bit much. For the default path literal to be useful it should work in most
normal cases, and I would say international characters would be a normal case.

What I would like to go for is the lowest reasonable common denominator for the
default p{}, for me at least this is MacOSX, Windows XP, Linux, *BSD, etc.


 With modern Unix/Windows I'm thinking about systems that support and use UCS2
 or UTF8 and where . or other common characters does not have special 
 meaning
 for the filesystem.

 We also need to keep in mind the Unicode problems between certain
 unixy platforms (i.e. MacOS X vs. most if not all the rest).

 If I recall correctly, the internal Unicode format chosen for Perl 6
 is incompatible with MacOS X, because MacOS X implemented Unicode
 support at a time when the standard as we know it today wasn't
 finalized.

 This has bearing on filenames, and MacOS X isn't a small enough
 platform that it can simply be ignored, either.

I guess if this puts limits of what characters can be in file names it should
also go in the default limits. But how Perl 6 stores the Path internally is not
really important, so long as it can be automatically converted with out
changing meaning before it is parsed to the OS. So what we should limit is what
can not be automatically converted.

Regards Troels.


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Mark J. Reed
On Tue, Aug 18, 2009 at 6:59 AM, Carlin Binghamcar...@theintersect.org wrote:
 2009/8/18 Timothy S. Nelson wayl...@wayland.id.au:
 On Tue, 18 Aug 2009, Mark J. Reed wrote:

       It's not in the revised spec, but I think that, even though we've
 revived chdir, we should still have it so that changing $*CWD will do a
 chdir under the hood.

You're quoting the wrong person.  That wasn't me.


-- 
Mark J. Reed markjr...@gmail.com


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Nicholas Clark
On Tue, Aug 18, 2009 at 01:10:58PM +0200, Jan Ingvoldstad wrote:
 On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk 
 wrote:

  Besides that, a simple check on Unix for what the locale is set to might 
  also be
  nice, so we don't write UTF8 files on a filesystem where the rest for the 
  files
  are in Latin1.
 
 The locale doesn't say what format the filenames are on the
 filesystem, though, merely the current user's language preferences may
 be.

We don't want to make the same mistakes as Python 3:

http://mail.python.org/pipermail/python-dev/2008-December/083856.html

The summary is that different file names in the same directory might be
in different encodings, and your programming language runtime sucks big time
if it doesn't offer you a way to iterate over all of them somehow, even if
you can't render their names.

[Consider a security critical program scanning using glob('*'), which gives
a clean bill of health because it opened all files and found no problems.]

I don't know how Python 3 resolved this.

Nicholas Clark


Re: r28017 - in docs/Perl6/Spec: . S32-setting-library

2009-08-18 Thread Troels Liebe Bentsen
I don't think python is the only one with that problem, try saving a file with
non utf8 chars in subversion and see what happens.

We should be liberal in what we accept and strict in what we send as we really
don't know the filesystem will return to us. I guess a file read from the
filesystem could have PathBinary as it's default object type. And the
programmer would have the option of converting it.

But you are right we have the same problem with Perl 5 today:

my $file = readdir $dir;

Should $file have the utf8 flag set if my locale is set to uft8?

or should i have to do a :

my $file = eval { decode(utf8, $file, Encode::FB_CROAK); };

every time i get a filename?

Regards Troels.

On Tue, Aug 18, 2009 at 16:37, Nicholas Clarkn...@ccl4.org wrote:
 On Tue, Aug 18, 2009 at 01:10:58PM +0200, Jan Ingvoldstad wrote:
 On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk 
 wrote:

  Besides that, a simple check on Unix for what the locale is set to might 
  also be
  nice, so we don't write UTF8 files on a filesystem where the rest for the 
  files
  are in Latin1.

 The locale doesn't say what format the filenames are on the
 filesystem, though, merely the current user's language preferences may
 be.

 We don't want to make the same mistakes as Python 3:

 http://mail.python.org/pipermail/python-dev/2008-December/083856.html

 The summary is that different file names in the same directory might be
 in different encodings, and your programming language runtime sucks big time
 if it doesn't offer you a way to iterate over all of them somehow, even if
 you can't render their names.

 [Consider a security critical program scanning using glob('*'), which gives
 a clean bill of health because it opened all files and found no problems.]

 I don't know how Python 3 resolved this.

 Nicholas Clark