Re: VMS and Unix pathnames
On Mon, Aug 24, 2009 at 09:48:17PM -0500, Craig A. Berry wrote: > But the bottom line is, get the t/TEST change into blead -- it doesn't > significantly change the number of minor mysteries that need > investigating and fixing. Thanks for checking this. It's in blead as of 7ebf5c89e8918d77e1798303430ceccf, and I think that your smoker has already smoked it. Nicholas Clark
Re: VMS and Unix pathnames
On Aug 24, 2009, at 9:58 PM, John E. Malmberg wrote: Nicholas Clark wrote: On Mon, Aug 24, 2009 at 08:40:44AM -0500, John E. Malmberg wrote: Nicholas Clark wrote: Currently the core uses File::Spec inside the test running script, t/TEST, that VMS uses. If I understand perlvms.pod correctly, perl on VMS can understand Unix-like pathnames directly. I'd like to remove File::Spec from t/TEST, as it's something complex run too early in testing. Would the appended change work on VMS? It assumes that As the current t/TEST is used now, I think it is passed the starting file specification in VMS format. So on VMS you still need a VMS::Filespec::unixify to convert the input path parameter(s) to the test. If I understand vms/test.com enough: $ If PERL_TEST_DRIVER .eqs. "minitest" $ Then $ MCR Sys$Disk:[]Perl'exe' TEST. "-minitest" "base/*.t" "comp/ *.t" "cmd/*.t" "run/*.t" "io/*.t" "op/*.t" "uni/*.t" $ Else $ MCR Sys$Disk:[]Perl'exe' "-I[-.lib]" 'PERL_TEST_DRIVER' "''p3'" "''p4'" "''p5'" "''p6'" "''p7'" $ EndIf and vms/descrip_mms.template test : all [.t.lib]vmsfspec.t [.t.lib]vms_dclsym.t [.t.lib]vms_stdio.t unpack_files @ PERL_TEST_DRIVER == "TEST." - @[.vms]test.com "$(E)" "$(__DEBUG__)" @ $(MINIPERL) -e "print ""Ran tests"";" > [.t]rantests. then under minitest, it's being passed Unix-style globs, and under test, nothing. So it is "already" expecting Unix-style names. vms/test.com is also run manually and when that is done, it is generally passed a VMS style path name, which it would have a difficult time to translate to a Unix path name to pass to t/TEST. It doesn't need to. It really just needs a filename it can pass to Perl. I just ran the following in a build directory after Nicholas's patch: $ @[.vms]test .exe "" -"v" [.base]cond.t %DELETE-I-FILDEL, D0:[smoke.Test-Smoke.blead.t]PERL.EXE;1 deleted (16 blocks) %COPY-S-COPIED, D0:[smoke.Test-Smoke.blead]PERL.EXE;1 copied to D0: [SMOKE.TEST-SMOKE.BLEAD.t]Perl.EXE;1 (16 blocks) %DELETE-I-FILDEL, D0:[smoke.Test-Smoke.blead.t]vmspipe.com;1 deleted (16 blocks) %COPY-S-COPIED, D0:[smoke.Test-Smoke.blead]vmspipe.com;1 copied to D0: [SMOKE.TEST-SMOKE.BLEAD.t]vmspipe.com;1 (2 blocks) 24-AUG-2009 22:12:10.60 User: SMOKEProcess ID: 000130F1 Node: ALMA Process name: "SMOKE" Accounting information: Buffered I/O count: 17662 Peak working set size: 19360 Direct I/O count:3324 Peak virtual size: 187920 Page faults: 7500 Mounted volumes:0 Images activated: 40 Elapsed CPU time: 0 00:00:04.30 Connect time: 0 02:23:20.43 1..4 ok 1 ok 2 ok 3 ok 4 t/[.BASE]CONDok So that bit looks a little off. I'm not convinced we should worry too much about that. We were still able to run the test and see its result. I don't think anything that parses the output will pass in native-syntax filenames, so it won't see them on output either. All tests successful. u=4.38 s=0.00 cu=0.00 cs=0.00 scripts=1 tests=4 24-AUG-2009 22:12:10.74 User: SMOKEProcess ID: 000130F1 Craig A. Berry mailto:craigbe...@mac.com "... getting out of a sonnet is much more difficult than getting in." Brad Leithauser
Re: VMS and Unix pathnames
Nicholas Clark wrote: On Mon, Aug 24, 2009 at 08:40:44AM -0500, John E. Malmberg wrote: Nicholas Clark wrote: Currently the core uses File::Spec inside the test running script, t/TEST, that VMS uses. If I understand perlvms.pod correctly, perl on VMS can understand Unix-like pathnames directly. I'd like to remove File::Spec from t/TEST, as it's something complex run too early in testing. Would the appended change work on VMS? It assumes that As the current t/TEST is used now, I think it is passed the starting file specification in VMS format. So on VMS you still need a VMS::Filespec::unixify to convert the input path parameter(s) to the test. If I understand vms/test.com enough: $ If PERL_TEST_DRIVER .eqs. "minitest" $ Then $ MCR Sys$Disk:[]Perl'exe' TEST. "-minitest" "base/*.t" "comp/*.t" "cmd/*.t" "run/*.t" "io/*.t" "op/*.t" "uni/*.t" $ Else $ MCR Sys$Disk:[]Perl'exe' "-I[-.lib]" 'PERL_TEST_DRIVER' "''p3'" "''p4'" "''p5'" "''p6'" "''p7'" $ EndIf and vms/descrip_mms.template test : all [.t.lib]vmsfspec.t [.t.lib]vms_dclsym.t [.t.lib]vms_stdio.t unpack_files @ PERL_TEST_DRIVER == "TEST." - @[.vms]test.com "$(E)" "$(__DEBUG__)" @ $(MINIPERL) -e "print ""Ran tests"";" > [.t]rantests. then under minitest, it's being passed Unix-style globs, and under test, nothing. So it is "already" expecting Unix-style names. vms/test.com is also run manually and when that is done, it is generally passed a VMS style path name, which it would have a difficult time to translate to a Unix path name to pass to t/TEST. -John wb8...@qsl.net Personal Opinion Only
Re: VMS and Unix pathnames
On Aug 24, 2009, at 5:12 AM, Nicholas Clark wrote:
Currently the core uses File::Spec inside the test running script, t/
TEST,
that VMS uses. If I understand perlvms.pod correctly, perl on VMS can
understand Unix-like pathnames directly.
I'd like to remove File::Spec from t/TEST, as it's something complex
run too
early in testing. Would the appended change work on VMS?
Yes. The embedded patch was mangled, but I've tried out the t/TEST
attached in a later message in this thread with a fairly recent blead
and got:
Failed 8 tests out of 1613, 99.50% okay.
../ext/IO/t/io_tell.t
../ext/Time-HiRes/t/HiRes.t
../lib/Module/Build/t/debug.t
../lib/base/t/fields-5.6.0.t
../lib/base/t/fields-5.8.0.t
io/errno.t
op/ref.t
pod/diag.t
The ones that didn't fail before the patch were:
../ext/IO/t/io_tell.t
../lib/base/t/fields-5.6.0.t
../lib/base/t/fields-5.8.0.t
The first one actually succeeded and the other two were skipped before
the patch (probably something to do with locating a file called
'Makefile'). The ones that were skipped before but fail now have code
that looks like:
if( $] >= 5.009 ) {
print "1..0 # skip pseudo-hashes removed in 5.9.0\n";
exit;
}
so it is truly bizarre that these are now run under a perl that thinks
it is:
$ perl -v
This is perl, v5.11.0 (GitLive-blead-1923-g45ec913*) built for
VMS_IA64-ithread
...
$ perl -e "print $]"
5.011000
And it's even a bit strange that these tests are still in the
repository for something that's reporting itself as 5.11.x, i.e., why
drag around tests that have been skipped since before 5.10.0?
But the bottom line is, get the t/TEST change into blead -- it doesn't
significantly change the number of minor mysteries that need
investigating and fixing.
Craig A. Berry
mailto:craigbe...@mac.com
"... getting out of a sonnet is much more
difficult than getting in."
Brad Leithauser
Re: VMS and Unix pathnames
On Mon, Aug 24, 2009 at 08:40:44AM -0500, John E. Malmberg wrote:
> Nicholas Clark wrote:
> >Currently the core uses File::Spec inside the test running script, t/TEST,
> >that VMS uses. If I understand perlvms.pod correctly, perl on VMS can
> >understand Unix-like pathnames directly.
> >
> >I'd like to remove File::Spec from t/TEST, as it's something complex run
> >too
> >early in testing. Would the appended change work on VMS? It assumes that
>
> As the current t/TEST is used now, I think it is passed the starting
> file specification in VMS format. So on VMS you still need a
> VMS::Filespec::unixify to convert the input path parameter(s) to the test.
If I understand vms/test.com enough:
$ If PERL_TEST_DRIVER .eqs. "minitest"
$ Then
$ MCR Sys$Disk:[]Perl'exe' TEST. "-minitest" "base/*.t" "comp/*.t"
"cmd/*.t" "run/*.t" "io/*.t" "op/*.t" "uni/*.t"
$ Else
$ MCR Sys$Disk:[]Perl'exe' "-I[-.lib]" 'PERL_TEST_DRIVER' "''p3'" "''p4'"
"''p5'" "''p6'" "''p7'"
$ EndIf
and vms/descrip_mms.template
test : all [.t.lib]vmsfspec.t [.t.lib]vms_dclsym.t [.t.lib]vms_stdio.t
unpack_files
@ PERL_TEST_DRIVER == "TEST."
- @[.vms]test.com "$(E)" "$(__DEBUG__)"
@ $(MINIPERL) -e "print ""Ran tests"";" > [.t]rantests.
then under minitest, it's being passed Unix-style globs, and under test,
nothing. So it is "already" expecting Unix-style names.
> >1: relative pathnames, Unix style, work
>
> Those mostly work. Testing would be needed to verify that these work
> for the pathnames that show up for t/TEST.
OK
> >2: readdir returns '.', '..' and directories in Unix format
>
> I think we would need to verify that also. I have been mostly focused
> on making Perl work with Unix file specifications using the Extended
> Character set.
>
> In the default/traditional mode of Perl on VMS, it was deemed acceptable
> for routines to return VMS format paths on output even when given Unix
> format paths on input. With the new Unix compatible mode, that should
> not happen.
>
> >Is that assuming to much? Would a subdirectory "foo" actually be returned
> >as
> >"foo." or "foo.DIR"?
>
> In the default/traditional mode of Perl on VMS, subdirectory "foo" will
> be returned as "foo.dir" when readdir() or glob operation is given a
> UNIX format path.
>
> It is a bug that we can not remove because there may be existing perl
> modules that depend on it being present.
That's OK. Nothing in MANIFEST is named .dir, so an unconditional s/\.dir$//i
will solve it
> Also in the default/traditional mode of Perl on VMS, file specifications
> are converted to lower case. The older file system on VMS only stores
> file names in uppercase, and code in VMS.c would convert them to
> lowercase for better Unix compatibility.
I think that this doesn't matter, as all this is doing is building pathnames
to open them for reading.
> >(I've also attached the entire file TEST, if anyone wants to try, and
> >doesn't
> >have a patch tool on VMS. You can get a snapshot of blead from
> >http://perl5.git.perl.org/perl.git/snapshot/HEAD.tar.gz )
>
> No attachment showed up.
PEBKAC, I suspect.
> I will try to do a build of blead sometime early this week to see how it
> works on VMS/Alpha.
Thanks
I'm (still) not subscribed to the vmsperl list, so please Cc: me on responses.
Nicholas Clark
#!./perl
# This is written in a peculiar style, since we're trying to avoid
# most of the constructs we'll be testing for. (This comment is
# probably obsolete on the avoidance side, though still currrent
# on the peculiarity side.)
$| = 1;
# for testing TEST only
#BEGIN { require '../lib/strict.pm'; strict->import() };
#BEGIN { require '../lib/warnings.pm'; warnings->import() };
# Let tests know they're running in the perl core. Useful for modules
# which live dual lives on CPAN.
$ENV{PERL_CORE} = 1;
delete $ENV{PERL5LIB};
# remove empty elements due to insertion of empty symbols via "''p1'" syntax
@ARGV = grep($_,@ARGV) if $^O eq 'VMS';
our $show_elapsed_time = $ENV{HARNESS_TIMER} || 0;
# Cheesy version of Getopt::Std. Maybe we should replace it with that.
{
my @argv = ();
foreach my $idx (0..$#ARGV) {
push( @argv, $ARGV[$idx] ), next unless $ARGV[$idx] =~ /^-(\S+)$/;
$::benchmark = 1 if $1 eq 'benchmark';
$::core= 1 if $1 eq 'core';
$::verbose = 1 if $1 eq 'v';
$::torture = 1 if $1 eq 'torture';
$::with_utf8 = 1 if $1 eq 'utf8';
$::with_utf16 = 1 if $1 eq 'utf16';
$::taintwarn = 1 if $1 eq 'taintwarn';
$ENV{PERL_CORE_MINITEST} = 1 if $1 eq 'minitest';
if ($1 =~ /^deparse(,.+)?$/) {
$::deparse = 1;
$::deparse_opts = $1;
}
}
@ARGV = @argv;
}
chdir 't' if -f 't/TEST';
die "You need to run \"make test\" first to set things up.\n"
unless -e 'perl' or -e 'perl.exe' or -e 'perl.pm';
if ($ENV{PERL_3LOG}) { # Tru64 third(1) tool, see perlhack
unless (-x 'perl.third')
Re: VMS and Unix pathnames
Nicholas Clark wrote: Currently the core uses File::Spec inside the test running script, t/TEST, that VMS uses. If I understand perlvms.pod correctly, perl on VMS can understand Unix-like pathnames directly. I'd like to remove File::Spec from t/TEST, as it's something complex run too early in testing. Would the appended change work on VMS? It assumes that As the current t/TEST is used now, I think it is passed the starting file specification in VMS format. So on VMS you still need a VMS::Filespec::unixify to convert the input path parameter(s) to the test. 1: relative pathnames, Unix style, work Those mostly work. Testing would be needed to verify that these work for the pathnames that show up for t/TEST. 2: readdir returns '.', '..' and directories in Unix format I think we would need to verify that also. I have been mostly focused on making Perl work with Unix file specifications using the Extended Character set. In the default/traditional mode of Perl on VMS, it was deemed acceptable for routines to return VMS format paths on output even when given Unix format paths on input. With the new Unix compatible mode, that should not happen. Is that assuming to much? Would a subdirectory "foo" actually be returned as "foo." or "foo.DIR"? In the default/traditional mode of Perl on VMS, subdirectory "foo" will be returned as "foo.dir" when readdir() or glob operation is given a UNIX format path. It is a bug that we can not remove because there may be existing perl modules that depend on it being present. Also in the default/traditional mode of Perl on VMS, file specifications are converted to lower case. The older file system on VMS only stores file names in uppercase, and code in VMS.c would convert them to lowercase for better Unix compatibility. When Perl on VMS is operating in the Extended Character Set mode with case preservered, then subdirectory "foo" will return "foo", and "FOOBAR" will be returned as "FOOBAR", provided that the Perl directory is on a disk with the newer file system and that the directory was created with the correct case. I have noticed that some of the files and directories created by the Perl build procedure on VMS are in the wrong case. VMS by default does not pay attention to the case when looking up a file, but if something is doing a directory lookup, they will get either the old behavior of all lowercase names, or the new behavior of exactly the case that is on the disk. When I am doing the tests, I set the mode that Perl is in before running all the tests. (I've also attached the entire file TEST, if anyone wants to try, and doesn't have a patch tool on VMS. You can get a snapshot of blead from http://perl5.git.perl.org/perl.git/snapshot/HEAD.tar.gz ) No attachment showed up. I will try to do a build of blead sometime early this week to see how it works on VMS/Alpha. I'm not subscribed to the vmsperl list, so please Cc: me on responses. Nicholas Clark -John wb8...@qsl.net Personal Opinion Only
VMS and Unix pathnames
Currently the core uses File::Spec inside the test running script, t/TEST,
that VMS uses. If I understand perlvms.pod correctly, perl on VMS can
understand Unix-like pathnames directly.
I'd like to remove File::Spec from t/TEST, as it's something complex run too
early in testing. Would the appended change work on VMS? It assumes that
1: relative pathnames, Unix style, work
2: readdir returns '.', '..' and directories in Unix format
Is that assuming to much? Would a subdirectory "foo" actually be returned as
"foo." or "foo.DIR"?
(I've also attached the entire file TEST, if anyone wants to try, and doesn't
have a patch tool on VMS. You can get a snapshot of blead from
http://perl5.git.perl.org/perl.git/snapshot/HEAD.tar.gz )
I'm not subscribed to the vmsperl list, so please Cc: me on responses.
Nicholas Clark
diff --git a/t/TEST b/t/TEST
index 2634485..511aac1 100755
--- a/t/TEST
+++ b/t/TEST
@@ -68,23 +68,30 @@ $ENV{EMXSHELL} = 'sh';# For OS/2
# Roll your own File::Find!
use TestInit;
-use File::Spec;
if ($show_elapsed_time) { require Time::HiRes }
-my $curdir = File::Spec->curdir;
-my $updir = File::Spec->updir;
+
+my %skip = (
+ '.' => 1,
+ '..' => 1,
+ 'CVS' => 1,
+ 'RCS' => 1,
+ 'SCCS' => 1,
+ '.svn' => 1,
+ );
sub _find_tests {
my($dir) = @_;
opendir DIR, $dir or die "Trouble opening $dir: $!";
foreach my $f (sort { $a cmp $b } readdir DIR) {
- next if $f eq $curdir or $f eq $updir or
- $f =~ /^(?:CVS|RCS|SCCS|\.svn)$/;
+ next if $skip{$f};
- my $fullpath = File::Spec->catfile($dir, $f);
+ my $fullpath = "$dir/$f";
- _find_tests($fullpath) if -d $fullpath;
- $fullpath = VMS::Filespec::unixify($fullpath) if $^O eq 'VMS';
- push @ARGV, $fullpath if $f =~ /\.t$/;
+ if (-d $fullpath) {
+ _find_tests($fullpath);
+ } elsif ($f =~ /\.t$/) {
+ push @ARGV, $fullpath;
+ }
}
}
@@ -113,7 +120,7 @@ unless (@ARGV) {
# Config.pm may be broken for make minitest. And this is only a refinement
# for skipping tests on non-default builds, so it is allowed to fail.
# What we want to to is make a list of extensions which we did not build.
-my $configsh = File::Spec->catfile($updir, "config.sh");
+my $configsh = '../config.sh';
my %skip;
if (-f $configsh) {
my (%extensions, %known_extensions);
@@ -140,7 +147,7 @@ unless (@ARGV) {
warn "No extensions line found in $configsh";
}
}
-my $mani = File::Spec->catfile($updir, "MANIFEST");
+my $mani = '../MANIFEST';
if (open(MANI, $mani)) {
while () { # similar code in t/harness
if
(m!^(ext/(\S+)/+(?:[^/\s]+\.t|test\.pl)|lib/\S+?(?:\.t|test\.pl))\s!) {
@@ -156,7 +163,7 @@ unless (@ARGV) {
$flat_extension =~ s!-!/!g;
next if $skip{$flat_extension}; # Foo/Bar may live in
Foo-Bar
}
- my $path = File::Spec->catfile($updir, $t);
+ my $path = "../$t";
push @ARGV, $path;
$::path_to_name{$path} = $t;
}
@@ -221,8 +228,7 @@ EOT
foreach my $t (@tests) {
unless (exists $::path_to_name{$t}) {
- my $tname = File::Spec->catfile('t',$t);
- $tname = VMS::Filespec::unixify($tname) if $^O eq 'VMS';
+ my $tname = "t/$t";
$::path_to_name{$t} = $tname;
}
}
