Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
David Green david.gr...@telus.net wrote: Maybe setting $*CWD just calls chdir() under the hood? Same implementation, brand new shiny Perl-style interface! I don't think that's a good idea. Suppose you have code like this: $*CWD = '/some/absolute/path'; $*CWD = '../relative/path'; my $cwd = $*CWD; Assuming that no exception was thrown, should $cwd now be '/some/absolute/relative/path' or '../relative/path' ? If the former, you've broken the user's reasonable expectation that a value assigned to a variable can be retrieved by looking in the same variable. If the latter, $*CWD is a relative path name that doesn't exist in the current directory. Now that I think about it, I'm having doubts about whether it makes sense to have a $*CWD variable at all. Suppose the current directory is /a/b/c, and a different process renames /a/b to /surprise. Should reading $*CWD reflect that? Calling a POSIX-like getcwd() would do, but that's surprising in just the same way. But if $*CWD doesn't reflect external changes, you can't expect to be able to opendir its value; at that point, it doesn't seem very useful. If the motivation for $*CWD is simply so that it can be temporized, perhaps it would be better to provide a more direct method of doing that (preferably using fchdir where available). -- Aaron Crane ** http://aaroncrane.co.uk/
r28017 - in docs/Perl6/Spec: . S32-setting-library
Author: wayland Date: 2009-08-18 09:24:07 +0200 (Tue, 18 Aug 2009) New Revision: 28017 Modified: docs/Perl6/Spec/S02-bits.pod docs/Perl6/Spec/S16-io.pod docs/Perl6/Spec/S28-special-names.pod docs/Perl6/Spec/S32-setting-library/IO.pod Log: [S02] Changed :io to :p and :path [S16] Documented p{/path/to/file} [S28] Made $*CWD have type Path instead of Str [S32/IO] Many changes, including: * Changed IO::FSNode into Path * Merged most of IO::DirectoryNode and all of IO::LinkNode into Path (but still need to merge IO::FileNode) * Moved remnants of IO::DirectoryNode to IO::Directory It's not finished yet, but I thought I'd commit anyway. I'll keep working on it. Modified: docs/Perl6/Spec/S02-bits.pod === --- docs/Perl6/Spec/S02-bits.pod2009-08-17 21:09:32 UTC (rev 28016) +++ docs/Perl6/Spec/S02-bits.pod2009-08-18 07:24:07 UTC (rev 28017) @@ -2834,7 +2834,7 @@ :subst Parse as substitution :trans Parse as transliteration :code Quasiquoting -:io Return an IO::FSNode +:p :path Return a Path object You may omit the first colon by joining an initial CQ, Cq, or Cqq with a single short form adverb, which produces forms like: Modified: docs/Perl6/Spec/S16-io.pod === --- docs/Perl6/Spec/S16-io.pod 2009-08-17 21:09:32 UTC (rev 28016) +++ docs/Perl6/Spec/S16-io.pod 2009-08-18 07:24:07 UTC (rev 28017) @@ -82,17 +82,55 @@ The use of filenames requires a special quoting syntax. It works as follows: -q:io{/path/to/file} +p{/path/to/file} +q:p{/path/to/file} +Both of the above result in the same thing. + The quote characters can be any of the usual ones, although / is probably a bad choice for filenames. -The code shown above returns an IO::FSNode object (or a descendant thereof). +The code shown above returns a Path object (or a descendant thereof). Naturally you can also ask for interpolation in filenames: -qq:io{$directory/$file} +p:qq{$directory/$file} +qq:p{$directory/$file} +There are a number of special adverbs that can be applied to the file quoting +operator. Most of these are filesystem-specific. They confine what can be +included in a filename. + +=head3 Default constraints + +The default p{} only allows / as separator and does not allow path elements +to contain +characters that won't work on modern Windows and Unix like \ / ? % * : | , +etc. The reason for this is that portable paths are the default. If +platform/filesystem specific behavior is really needed it should be shown in +the code by applying different sets of constraints (see below). + +=head3 Windows-style constraints + +We allow windows style paths so converting and maintaining code on this +platform is not a pain. + +my Path $path = p:win{C:\Program Files\MS Access\file.file}; + +Note that this specifically excludes the backslash quoting usually used with +q{}. + +=head3 Unix-style constraints + +For Unix specific behavior we have a p:unix{} literal. Here the only +limits are what is defined by the locale and the filesystem type. So we won't +be able to write full Unicode if locale is set to Latin1. + +my Path $path = p:unix{/usr/src/bla/myfile?:%.file}; + +And for cases where this is a problem p:bin{} can be used as no checking is +done here, other than assuming that / is the separator. + =head1 Name Services =head2 User role Modified: docs/Perl6/Spec/S28-special-names.pod === --- docs/Perl6/Spec/S28-special-names.pod 2009-08-17 21:09:32 UTC (rev 28016) +++ docs/Perl6/Spec/S28-special-names.pod 2009-08-18 07:24:07 UTC (rev 28017) @@ -68,7 +68,7 @@ Variable Spec Type Description --- - @_ # ??? + @_ # ??? (FIX) $!S04# Current Exception object $/S05 Match# Last match $0, $1, $2S05 Str # First captured value from match: $/[0] @@ -81,7 +81,7 @@ $?CLASS S02 Class# current class @=COMMENT(S26) # All the comment blocks in the file %?CONFIGHash of XXX # configuration hash XXX What does this do? - $*CWD Str # current working directory + $*CWD Path # current working directory $=DATA (S26) IO # data block handle (=begin DATA ... =end) @=DATA (S26) Array# Same as above, but array %?DEEPMAGIC S13 Hash of XXX # Controls the mappings of magical names to sub definitions Modified: docs/Perl6/Spec/S32-setting-library/IO.pod
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 09:24:08AM +0200, pugs-comm...@feather.perl6.nl wrote: +=head3 Default constraints + +The default p{} only allows / as separator and does not allow path elements +to contain +characters that won't work on modern Windows and Unix like \ / ? % * : | , +etc. The reason for this is that portable paths are the default. If +platform/filesystem specific behavior is really needed it should be shown in +the code by applying different sets of constraints (see below). Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy of enforced portable filenames by default, it should (at least) also exclude - as the first character, and forbid more than one . in a filename. (Also, I suspect, ; and . are illegal in directory names. And on VMS catfile() and catdir() *are* distinct operations, so it's important to know if the right-most argument is a file or a directory) Nicholas Clark
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
2009/8/18 pugs-comm...@feather.perl6.nl: Author: wayland Date: 2009-08-18 09:24:07 +0200 (Tue, 18 Aug 2009) New Revision: 28017 +=item chdir FILENAME +Xchdir Xcd + +=item chdir + +Gone, just set $*CWD (which throws an exception if it fails). + chdir provides functionality that would be quite convoluted to mimic through manually setting $*CWD, such as changing to a relative directory. Unless there are ways to do it that I'm missing, this doesn't seem like a good idea. -- Carlin
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 10:01 AM, Nicholas Clarkn...@ccl4.org wrote: On Tue, Aug 18, 2009 at 09:24:08AM +0200, pugs-comm...@feather.perl6.nl wrote: +=head3 Default constraints + +The default p{} only allows / as separator and does not allow path elements +to contain +characters that won't work on modern Windows and Unix like \ / ? % * : | , +etc. The reason for this is that portable paths are the default. If +platform/filesystem specific behavior is really needed it should be shown in +the code by applying different sets of constraints (see below). Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy of enforced portable filenames by default, it should (at least) also exclude - as the first character, and forbid more than one . in a filename. And, as I mentioned in an earlier post during the discussion, the restrictions for Windows are numerous: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx Enforcing truly portable filenames is unrealistic, I think, but having a POSIX-checking default is a good thing. -- Jan
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 10:36:45AM +0200, Jan Ingvoldstad wrote: On Tue, Aug 18, 2009 at 10:01 AM, Nicholas Clarkn...@ccl4.org wrote: On Tue, Aug 18, 2009 at 09:24:08AM +0200, pugs-comm...@feather.perl6.nl wrote: +=head3 Default constraints + +The default p{} only allows / as separator and does not allow path elements +to contain +characters that won't work on modern Windows and Unix like \ / ? % * : | , +etc. The reason for this is that portable paths are the default. If +platform/filesystem specific behavior is really needed it should be shown in +the code by applying different sets of constraints (see below). Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy of enforced portable filenames by default, it should (at least) also exclude - as the first character, and forbid more than one . in a filename. And, as I mentioned in an earlier post during the discussion, the restrictions for Windows are numerous: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx Oh gosh yes. I forgot. AUX.TXT And all the the other CP/M device file names, with our without extensions... [And of course, IIRC, DOS filenames can't be more than 64 characters. Which means that your code thinks that it knows what it's doing by stripping of the drive letter and allowing 62 in the rest of the pathname. Except that I remember university servers with drives UX: and UY:, which always made me wonder if there was a buffer overflow waiting to happen...] Enforcing truly portable filenames is unrealistic, I think, but having a POSIX-checking default is a good thing. What does POSIX enforce? ASCII NUL terminated, multiple adjacent / characters fold to 1, except at the start, where // is special, but /// etc aren't? And anything else goes? Nicholas Clark
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote: chdir provides functionality that would be quite convoluted to mimic through manually setting $*CWD, such as changing to a relative directory. Maybe setting $*CWD just calls chdir() under the hood? Same implementation, brand new shiny Perl-style interface! -David
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 10:58 AM, Nicholas Clarkn...@ccl4.org wrote: Oh gosh yes. I forgot. AUX.TXT And all the the other CP/M device file names, with our without extensions... [And of course, IIRC, DOS filenames can't be more than 64 characters. Which means that your code thinks that it knows what it's doing by stripping of the drive letter and allowing 62 in the rest of the pathname. Except that I remember university servers with drives UX: and UY:, which always made me wonder if there was a buffer overflow waiting to happen...] Yes, lovely, isn't it? What does POSIX enforce? ASCII NUL terminated, multiple adjacent / characters fold to 1, except at the start, where // is special, but /// etc aren't? And anything else goes? POSIX doesn't enforce much. I used our friendly do-no-evil empire, and found this: http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html#standards And particularly from «2.2. Standards permit the exclusion of bad filenames»: «I then examined the Portable Filename Character Set, defined in 3.276 (“Portable Filename Character Set”); this turns out to be just A-Z, a-z, 0-9, period, underscore, and hyphen (aka the dash character). So it’s perfectly okay for a POSIX system to reject a non-portable filename due to it having “odd” characters or a leading hyphen.» -- Jan
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 11:04 AM, David Greendavid.gr...@telus.net wrote: On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote: chdir provides functionality that would be quite convoluted to mimic through manually setting $*CWD, such as changing to a relative directory. Maybe setting $*CWD just calls chdir() under the hood? Same implementation, brand new shiny Perl-style interface! It may seem cool, but I don't like secondary effects like that. They break the principle of least surprise. -- Jan
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, 18 Aug 2009, David Green wrote: On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote: chdir provides functionality that would be quite convoluted to mimic through manually setting $*CWD, such as changing to a relative directory. Maybe setting $*CWD just calls chdir() under the hood? Same implementation, brand new shiny Perl-style interface! That was my intent, but we had some discussions on IRC about the whys and wherefores, and it will return as soon as I do my next commit (hopefully sometime within the next 4-5 hours). Unless you can think of a good way to do relative paths. :) - | Name: Tim Nelson | Because the Creator is,| | E-mail: wayl...@wayland.id.au| I am | - BEGIN GEEK CODE BLOCK Version 3.12 GCS d+++ s+: a- C++$ U+++$ P+++$ L+++ E- W+ N+ w--- V- PE(+) Y+++ PGP-+++ R(+) !tv b++ DI D G+ e++ h! y- -END GEEK CODE BLOCK-
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 11:04 AM, David Greendavid.gr...@telus.net wrote: On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote: chdir provides functionality that would be quite convoluted to mimic through manually setting $*CWD, such as changing to a relative directory. Maybe setting $*CWD just calls chdir() under the hood? Same implementation, brand new shiny Perl-style interface! If $*CWD is really a Path object and not a Str, then it should be easy to use mutator methods to change to a relative directory and do other chdir()ish things. Say, concatenation works in terms of path components, for instance: $*CWD ~= $subdir; # chdir($subdir) With a method for getting the parent: given $*CWD { $_ = $_.up ~ $sibling } # chdir(../$sibling) and so on. My favorite kshism is cd old new which does a search/replace on the current working directory; the bash equivalent cd ${PWD/old/new} which is not quite as handy. $*CWD could make that simple, too. -- Mark J. Reed markjr...@gmail.com
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, 18 Aug 2009, Mark J. Reed wrote: On Tue, Aug 18, 2009 at 11:04 AM, David Greendavid.gr...@telus.net wrote: On 2009-Aug-18, at 2:29 am, Carlin Bingham wrote: chdir provides functionality that would be quite convoluted to mimic through manually setting $*CWD, such as changing to a relative directory. Maybe setting $*CWD just calls chdir() under the hood? Same implementation, brand new shiny Perl-style interface! If $*CWD is really a Path object and not a Str, then it should be easy to use mutator methods to change to a relative directory and do other chdir()ish things. Say, concatenation works in terms of path components, for instance: $*CWD ~= $subdir; # chdir($subdir) With a method for getting the parent: given $*CWD { $_ = $_.up ~ $sibling } # chdir(../$sibling) and so on. My favorite kshism is cd old new which does a search/replace on the current working directory; the bash equivalent cd ${PWD/old/new} which is not quite as handy. $*CWD could make that simple, too. It's not in the revised spec, but I think that, even though we've revived chdir, we should still have it so that changing $*CWD will do a chdir under the hood. My intent is that Path should be able to act as a string, at least to a certain extent, and should also be able to act as an array of Str. So Mark's second example could be done as: pop $*CWD; push $*CWD $sibling HTH, - | Name: Tim Nelson | Because the Creator is,| | E-mail: wayl...@wayland.id.au| I am | - BEGIN GEEK CODE BLOCK Version 3.12 GCS d+++ s+: a- C++$ U+++$ P+++$ L+++ E- W+ N+ w--- V- PE(+) Y+++ PGP-+++ R(+) !tv b++ DI D G+ e++ h! y- -END GEEK CODE BLOCK-
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
Perl 5 runs on (at least) VMS and VOS too. So, if Perl 6 is to adopt a policy of enforced portable filenames by default, it should (at least) also exclude - as the first character, and forbid more than one . in a filename. And, as I mentioned in an earlier post during the discussion, the restrictions for Windows are numerous: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx Enforcing truly portable filenames is unrealistic, I think, but having a POSIX-checking default is a good thing. My idea with portable by default was only portability for modern Unix and modern Windows. So DOS and VMS limitations would not apply. The problem of enforcing truly portable filenames is that the files names get too restrictive and for most applications targeting 98% of systems out there would be enough. With modern Unix/Windows I'm thinking about systems that support and use UCS2 or UTF8 and where . or other common characters does not have special meaning for the filesystem. Using POSIX as the basis would properly not be enough as it's either to lax or to strict. I think Windows NTFS/VFAT32 + Kernel limits on characters might be a good basis: http://en.wikipedia.org/wiki/Filename * So paths can be up to 259 characters long * No characters in the range 0x01-0x1F * No control characters * Non of the following characters: * : ? \ / |. * Directory or filename up to 255 characters long * Only Unicode that fits in UCS-2(the 16bit subset) Any system specific restrictions would not apply like AUX, CLOCK$, COM, etc. would not apply and the programmer would be allow to shoot him/herself in the foot. Besides that, a simple check on Unix for what the locale is set to might also be nice, so we don't write UTF8 files on a filesystem where the rest for the files are in Latin1. Regards Troels.
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
2009/8/18 Timothy S. Nelson wayl...@wayland.id.au: On Tue, 18 Aug 2009, Mark J. Reed wrote: It's not in the revised spec, but I think that, even though we've revived chdir, we should still have it so that changing $*CWD will do a chdir under the hood. While in the spirit of TIMTOWTDI, having a magic variable that behaves differently from other variables when it's being set would be rather odd. -- Carlin
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk wrote: My idea with portable by default was only portability for modern Unix and modern Windows. So DOS and VMS limitations would not apply. The problem of enforcing truly portable filenames is that the files names get too restrictive and for most applications targeting 98% of systems out there would be enough. That's a decent enough point, but it may be unwise to ignore legacy systems that where Perl 5 is in common use, unless we want to shed that userbase. (Mark this down as a I don't know, and I don't have a stake in it, but…) With modern Unix/Windows I'm thinking about systems that support and use UCS2 or UTF8 and where . or other common characters does not have special meaning for the filesystem. We also need to keep in mind the Unicode problems between certain unixy platforms (i.e. MacOS X vs. most if not all the rest). If I recall correctly, the internal Unicode format chosen for Perl 6 is incompatible with MacOS X, because MacOS X implemented Unicode support at a time when the standard as we know it today wasn't finalized. This has bearing on filenames, and MacOS X isn't a small enough platform that it can simply be ignored, either. Besides that, a simple check on Unix for what the locale is set to might also be nice, so we don't write UTF8 files on a filesystem where the rest for the files are in Latin1. The locale doesn't say what format the filenames are on the filesystem, though, merely the current user's language preferences may be. That can't really be trusted, at least not for filename comparison purposes. But I agree that it's sensible to default to the locale's settings regarding character encoding. -- Jan
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 13:10, Jan Ingvoldstadfrett...@gmail.com wrote: On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk wrote: My idea with portable by default was only portability for modern Unix and modern Windows. So DOS and VMS limitations would not apply. The problem of enforcing truly portable filenames is that the files names get too restrictive and for most applications targeting 98% of systems out there would be enough. That's a decent enough point, but it may be unwise to ignore legacy systems that where Perl 5 is in common use, unless we want to shed that userbase. (Mark this down as a I don't know, and I don't have a stake in it, but…) I completely agree and we might make a p:posix{} or p:strict{} to handle that. But one thing to remember is that having the default p{} allow characters and formats that is not supported on VMS or DOS, only means the programmer won't get a compiler error. In other words he is only a little better off than using a normal string or Q{}. Also in cases where systems has special requirements the local version should be used so, fx. VMS would have p:vms{} and DOS p:dos{}. The defaults should handle most cases and limiting path's to only ASCII might be a bit much. For the default path literal to be useful it should work in most normal cases, and I would say international characters would be a normal case. What I would like to go for is the lowest reasonable common denominator for the default p{}, for me at least this is MacOSX, Windows XP, Linux, *BSD, etc. With modern Unix/Windows I'm thinking about systems that support and use UCS2 or UTF8 and where . or other common characters does not have special meaning for the filesystem. We also need to keep in mind the Unicode problems between certain unixy platforms (i.e. MacOS X vs. most if not all the rest). If I recall correctly, the internal Unicode format chosen for Perl 6 is incompatible with MacOS X, because MacOS X implemented Unicode support at a time when the standard as we know it today wasn't finalized. This has bearing on filenames, and MacOS X isn't a small enough platform that it can simply be ignored, either. I guess if this puts limits of what characters can be in file names it should also go in the default limits. But how Perl 6 stores the Path internally is not really important, so long as it can be automatically converted with out changing meaning before it is parsed to the OS. So what we should limit is what can not be automatically converted. Regards Troels.
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 6:59 AM, Carlin Binghamcar...@theintersect.org wrote: 2009/8/18 Timothy S. Nelson wayl...@wayland.id.au: On Tue, 18 Aug 2009, Mark J. Reed wrote: It's not in the revised spec, but I think that, even though we've revived chdir, we should still have it so that changing $*CWD will do a chdir under the hood. You're quoting the wrong person. That wasn't me. -- Mark J. Reed markjr...@gmail.com
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
On Tue, Aug 18, 2009 at 01:10:58PM +0200, Jan Ingvoldstad wrote: On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk wrote: Besides that, a simple check on Unix for what the locale is set to might also be nice, so we don't write UTF8 files on a filesystem where the rest for the files are in Latin1. The locale doesn't say what format the filenames are on the filesystem, though, merely the current user's language preferences may be. We don't want to make the same mistakes as Python 3: http://mail.python.org/pipermail/python-dev/2008-December/083856.html The summary is that different file names in the same directory might be in different encodings, and your programming language runtime sucks big time if it doesn't offer you a way to iterate over all of them somehow, even if you can't render their names. [Consider a security critical program scanning using glob('*'), which gives a clean bill of health because it opened all files and found no problems.] I don't know how Python 3 resolved this. Nicholas Clark
Re: r28017 - in docs/Perl6/Spec: . S32-setting-library
I don't think python is the only one with that problem, try saving a file with non utf8 chars in subversion and see what happens. We should be liberal in what we accept and strict in what we send as we really don't know the filesystem will return to us. I guess a file read from the filesystem could have PathBinary as it's default object type. And the programmer would have the option of converting it. But you are right we have the same problem with Perl 5 today: my $file = readdir $dir; Should $file have the utf8 flag set if my locale is set to uft8? or should i have to do a : my $file = eval { decode(utf8, $file, Encode::FB_CROAK); }; every time i get a filename? Regards Troels. On Tue, Aug 18, 2009 at 16:37, Nicholas Clarkn...@ccl4.org wrote: On Tue, Aug 18, 2009 at 01:10:58PM +0200, Jan Ingvoldstad wrote: On Tue, Aug 18, 2009 at 12:54 PM, Troels Liebe Bentsent...@rapanden.dk wrote: Besides that, a simple check on Unix for what the locale is set to might also be nice, so we don't write UTF8 files on a filesystem where the rest for the files are in Latin1. The locale doesn't say what format the filenames are on the filesystem, though, merely the current user's language preferences may be. We don't want to make the same mistakes as Python 3: http://mail.python.org/pipermail/python-dev/2008-December/083856.html The summary is that different file names in the same directory might be in different encodings, and your programming language runtime sucks big time if it doesn't offer you a way to iterate over all of them somehow, even if you can't render their names. [Consider a security critical program scanning using glob('*'), which gives a clean bill of health because it opened all files and found no problems.] I don't know how Python 3 resolved this. Nicholas Clark