symlink support (was Re: Changing osname or ^O from VMS.C at startup?)
At 6:21 PM -0500 2/18/05, John E. Malmberg wrote: I am still working on getting the symlink support going but there are significant implications to using symbolic links on OpenVMS once the ECO kit is available. Perl is affected by this, so here is a preview: Symbolic links introduce and for some operations require the use of a new UNIX style filename convention that is being called POSIX format. The processing of these specifications is now done by RMS and not by the CRTL so there is a way to use them with almost all existing OpenVMS programs. Because of the differences in processing a symbolic link references, any program that does it's own translation of file specifications from UNIX format to OpenVMS native format will not return the right answer for symbolic links. I find this curious, since I'm not aware of any difference in the syntax of a filename that indicates whether or not it's a symbolic link on unix systems. What is there about the file specification of a symbolic link that cannot be expressed in VMS syntax? The CRTL will have a new feature setting to indicate when the user has selected that UNIX style pathnames should be parsed in POSIX style, and while I can easily test for this in the C code, but it looks like I would need a way to let the .PM scripts know that this feature has been activated by the user in order to for the special cased OpenVMS code to work with symbolic links. Possibly. The perl absolute path routine is one of the ones that is affected by this as it is special cased for OpenVMS to do a manual parse of the filename. It calls VMS::Filespec::rmsexpand, which invokes mp_do_rmsexpand() in [.vms]vms.c. We could check for the feature involved here and omit the call to do_tovmsspec() in mp_do_rmsexpand() on the assumption that, as you seem to be implying above, sys$parse can handle unix syntax filenames when the symlink feature is available. There may be side effects and this would have to be studied carefully. It looks like the way to do that is to have the affected scripts look up the setting of the DECC feature logicals while they run. If Perl is to support DECC$FILENAME_UNIX_ONLY and/or DECC$FILENAME_UNIX_REPORT, then yes, there will have to be some changes to various things and these will not be particularly easy changes. Historically, way back when the CRTL kinda sorta supported unix syntax filenames but often failed to handle them properly, the safest thing to do was just always convert to VMS syntax before calling the CRTL, and of course calls to system services and other RTLs have always had to do this as well. The Perl implementation is a very mixed bag of calls to the CRTL and to native APIs, so it might well be rather difficult to disentangle when we need unix syntax and when we need VMS syntax under conditions where we can no longer assume it's always safe to convert to VMS syntax. -- Craig A. Berry mailto:[EMAIL PROTECTED] ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
Re: Changing osname or ^O from VMS.C at startup?
Craig A. Berry wrote: At 12:32 AM -0500 2/16/05, John E. Malmberg wrote: $ search/exact *.c OSNAME ** D0:[CRAIG.PERL-5_8]perl.c;1 PL_osname = savepv(OSNAME); Some how I must have missed that one. The help of this mailing list has been of great assistance, these modifications that I am attempting, and I am sure I will probably need more. It looks like changing the osname is not the way to do what I need done. I am still working on getting the symlink support going but there are significant implications to using symbolic links on OpenVMS once the ECO kit is available. Perl is affected by this, so here is a preview: Symbolic links introduce and for some operations require the use of a new UNIX style filename convention that is being called POSIX format. The processing of these specifications is now done by RMS and not by the CRTL so there is a way to use them with almost all existing OpenVMS programs. Because of the differences in processing a symbolic link references, any program that does it's own translation of file specifications from UNIX format to OpenVMS native format will not return the right answer for symbolic links. The CRTL will have a new feature setting to indicate when the user has selected that UNIX style pathnames should be parsed in POSIX style, and while I can easily test for this in the C code, but it looks like I would need a way to let the .PM scripts know that this feature has been activated by the user in order to for the special cased OpenVMS code to work with symbolic links. The perl absolute path routine is one of the ones that is affected by this as it is special cased for OpenVMS to do a manual parse of the filename. It looks like the way to do that is to have the affected scripts look up the setting of the DECC feature logicals while they run. -John [EMAIL PROTECTED] Personal Opinion Only
Re: Changing osname or ^O from VMS.C at startup?
At 12:32 AM -0500 2/16/05, John E. Malmberg wrote: I did a search of the source code before I posted, while the osname macro is defined in config.h, I can not find anything in the perl .C files that reference any of the osname macros in any of the .H files, or any macros that would hide such references. I will have to look at the S_init_predump_symbols to see if that gives me any more clues. In the top-level directory: $ search/exact *.c OSNAME ** D0:[CRAIG.PERL-5_8]perl.c;1 Perl_sv_catpvf(aTHX_ PL_Sv,\ Built under %s\\n\,OSNAME); OSNAME, __DATE__, __TIME__)); PL_osname = savepv(OSNAME); -- Craig A. Berry mailto:[EMAIL PROTECTED] ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
Changing osname or ^O from VMS.C at startup?
It appears that the VMS specific file name behavior is tied to many perl scripts that are checking the ^O for 'VMS'. What I would like to investigate is having a option where a symbol or logical name could be used by the init and pre-init code in VMS.C to change the ^O to report 'GNV'. That same feature would also cause the DECC feature settings to go to full UNIX/POSIX filename compatibility mode. In investigating why the first standard test for symlinks on perl was failing, it turns out that the fast_abs_path() and abs_path() call VMS specific code including using rms calls to return the path. This existing code does not know how to follow a symbolic link to the real path, and since other code calls those same routines that should not return a real path, but the link value, a new routine would be needed for the abs_path routines to use. Alternatively, symbolic links virtually require UNIX/POSIX path names to function, which means that they are most likely only to be used in a full POSIX environment. It looks like when Perl does not recognize the value of ^O, it defaults to POSIX/UNIX behavior on file handling. I have not found out yet how to read or change the value of ^O from C, as it seems to only be set by config.pm, and I am not sure if config.pm is used after Perl is built. If anyone has any suggestions on an easy way to do this, please let me know. Thanks, -John [EMAIL PROTECTED] Personal Opinion Only
Re: Changing osname or ^O from VMS.C at startup?
John E. Malmberg [EMAIL PROTECTED] wrote on 02/15/2005 03:21:01 PM: If anyone has any suggestions on an easy way to do this, please let me know. I'd recommend rewriting config.h before you compile perl under GNV, later you could modify the Configure script itself to recognize GNV as a distinct osname. Here for example is how perl 5.8.1 is built to recognize VMS: $ search PERL_ROOT:[LIB.VMS_AXP.5_8_1.CORE]config.h osname /* OSNAME: #define OSNAME VMS/**/ Of course changing that and recompiling does not give you the flexibility of a symbol or logical that you requested. However there is precedent for such a distinct setting of $^O. Namely the Windows native port calls itself 'MSWin32' whereas the cygwin (GNU/bash et al) environment calls itself 'cygwin' in perl's $^O variable. Peter Prymmer
Re: Changing osname or ^O from VMS.C at startup?
On Tue, 15 Feb 2005 [EMAIL PROTECTED] wrote: John E. Malmberg [EMAIL PROTECTED] wrote on 02/15/2005 03:21:01 PM: If anyone has any suggestions on an easy way to do this, please let me know. I'd recommend rewriting config.h before you compile perl under GNV, later you could modify the Configure script itself to recognize GNV as a distinct osname. Here for example is how perl 5.8.1 is built to recognize VMS: Yes, I modified the configure.com script to allow osname=GNV. The build has completed and it is running the tests now. I am noticing on a previous build that a number of the tests are failing when you build with the VMS DEBUG option, and the tests use PERLSHR instead of DBGPERLSHR. When the tests are run using DBGPERLSHR they succeed. This is telling me that when the VMS DEBUG option is selected, PERLSHR is not being built correctly. $ search PERL_ROOT:[LIB.VMS_AXP.5_8_1.CORE]config.h osname /* OSNAME: #define OSNAME VMS/**/ Of course changing that and recompiling does not give you the flexibility of a symbol or logical that you requested. However there is precedent for such a distinct setting of $^O. Namely the Windows native port calls itself 'MSWin32' whereas the cygwin (GNU/bash et al) environment calls itself 'cygwin' in perl's $^O variable. Yes, I noticed that. The cygwin is a more isolated container environment than GNV is, and while I might have to settle for a separate binary, I would prefer that a system could have one installation of Perl for all of it's uses. That the build proceded completed with the -Dosname=GNV and the self tests are generating an active display on the monitor system command is encouraging. -John [EMAIL PROTECTED] Personal Opinion Only
Re: Changing osname or ^O from VMS.C at startup?
On Tuesday, February 15, 2005, at 02:21PM, John E. Malmberg [EMAIL PROTECTED] wrote: It appears that the VMS specific file name behavior is tied to many perl scripts that are checking the ^O for 'VMS'. What I would like to investigate is having a option where a symbol or logical name could be used by the init and pre-init code in VMS.C to change the ^O to report 'GNV'. That same feature would also cause the DECC feature settings to go to full UNIX/POSIX filename compatibility mode. The global PL_osname is set in S_init_predump_symbols in perl.c, which is called from S_parse_body when a Perl script is compiled. If you put in your own value in start-up code, I'm pretty sure it will get overwritten later. You could stick some code in S_init_predump_symbols to check for a logical name and reset accordingly. That might be a little dangerous since we don't really know what side effects there might be to dynamically changing the OS name. If you can live with configuration time changes, look for $ osname = F$EDIT(F$GETSYI(NODE_SWTYPE),COLLAPSE) in configure.com and modify it to say whatever you want. In investigating why the first standard test for symlinks on perl was failing, it turns out that the fast_abs_path() and abs_path() call VMS specific code including using rms calls to return the path. This existing code does not know how to follow a symbolic link to the real path, and since other code calls those same routines that should not return a real path, but the link value, a new routine would be needed for the abs_path routines to use. Alternatively, symbolic links virtually require UNIX/POSIX path names to function, which means that they are most likely only to be used in a full POSIX environment. It looks like when Perl does not recognize the value of ^O, it defaults to POSIX/UNIX behavior on file handling. I have not found out yet how to read or change the value of ^O from C, as it seems to only be set by config.pm, and I am not sure if config.pm is used after Perl is built. If anyone has any suggestions on an easy way to do this, please let me know. Thanks, -John [EMAIL PROTECTED] Personal Opinion Only
Re: Changing osname or ^O from VMS.C at startup?
On Tuesday, February 15, 2005, at 03:21PM, John E. Malmberg [EMAIL PROTECTED] wrote: I am noticing on a previous build that a number of the tests are failing when you build with the VMS DEBUG option, and the tests use PERLSHR instead of DBGPERLSHR. When the tests are run using DBGPERLSHR they succeed. This is telling me that when the VMS DEBUG option is selected, PERLSHR is not being built correctly. When you configure and build for the VMS debugger, there is not supposed to be a PERLSHR.EXE, only DBGPERLSHR.EXE. If you build in debug without configuring for it, you will see the symptoms you describe. Switching back and forth between a debug and non-debug build without reconfiguring is not supported. If you are configuring for the debugger and still seeing this issue, please holler. BTW, the simplest way to configure for the debugger is: $ @configure -Dusevmsdebug -des
Debugger - was: Changing osname or ^O from VMS.C at startup?
On Tue, 15 Feb 2005, Craig Berry wrote: On Tuesday, February 15, 2005, at 03:21PM, John E. Malmberg wrote: I am noticing on a previous build that a number of the tests are failing when you build with the VMS DEBUG option, and the tests use PERLSHR instead of DBGPERLSHR. When the tests are run using DBGPERLSHR they succeed. This is telling me that when the VMS DEBUG option is selected, PERLSHR is not being built correctly. When you configure and build for the VMS debugger, there is not supposed to be a PERLSHR.EXE, only DBGPERLSHR.EXE. If you build in debug without configuring for it, you will see the symptoms you describe. Switching back and forth between a debug and non-debug build without reconfiguring is not supported. If you are configuring for the debugger and still seeing this issue, please holler. Hollering. I did a MMK realclean between the builds, which gets rid if any dangling executables. Directory BUILD_ROOT:[00] DBGMINIPERL.EXE;1 7269 15-FEB-2005 16:20:51.35 (RWED,RWED,RE,) DBGPERL.EXE;1 52 15-FEB-2005 16:21:56.35 (RWED,RWED,RE,) DBGPERLSHR.EXE;17469 15-FEB-2005 16:21:54.61 (RWED,RWED,RE,) MINIPERL.EXE;3 4111 15-FEB-2005 16:54:19.09 (RWED,RWED,RE,) MINIPERL.EXE;2 4111 15-FEB-2005 16:54:17.57 (RWED,RWED,RE,) MINIPERL.EXE;1 3873 15-FEB-2005 16:20:53.78 (RWED,RWED,RE,) NDBGperl.exe;1 8 15-FEB-2005 16:21:56.72 (RWED,RWED,RE,) PERL.EXE;1 6 15-FEB-2005 16:54:46.44 (RWED,RWED,RE,) PERLSHR.EXE;1 3441 15-FEB-2005 16:54:45.53 (RWED,RWED,RE,) Ahhh Apparently the MMK/FROM_SOURCES TEST is creating the non debug version for it's use, and that is causing the tests to fail. So I need to modify that command in my master build file to also have the /macro=__DEBUG__=1 for that MMK command. BTW, the simplest way to configure for the debugger is: $ @configure -Dusevmsdebug -des Currently I am using: $ @build_root:['root']configure.com -Dusevmsdebug - -Dunlink_all_versions -Duselargefiles -Dusesymlinks - -Dusehardlinks -Dosname=GNV -de If the GNV build works, I am going to make it imply: unlink_all_versions, uselargefiles, usesymlinks, and usehardlinks, and possibly enabling all 64 bit options. I plan to also put in vms.h the #ifdef for the GNV environment to set the C feature logical to default to that all file specifications will be in UNIX format and case sensitive, unless the user overides this. -John [EMAIL PROTECTED] Personal Opinion Only
Re: Changing osname or ^O from VMS.C at startup?
On Tue, 15 Feb 2005, Craig Berry wrote: On Tuesday, February 15, 2005, at 02:21PM, John E. Malmberg [EMAIL PROTECTED] wrote: It appears that the VMS specific file name behavior is tied to many perl scripts that are checking the ^O for 'VMS'. What I would like to investigate is having a option where a symbol or logical name could be used by the init and pre-init code in VMS.C to change the ^O to report 'GNV'. That same feature would also cause the DECC feature settings to go to full UNIX/POSIX filename compatibility mode. The global PL_osname is set in S_init_predump_symbols in perl.c, which is called from S_parse_body when a Perl script is compiled. If you put in your own value in start-up code, I'm pretty sure it will get overwritten later. You could stick some code in S_init_predump_symbols to check for a logical name and reset accordingly. That might be a little dangerous since we don't really know what side effects there might be to dynamically changing the OS name. Apparently the OS name is dynamically set now by a perl script, config.p*, and I can not find a pre-compiled version of it. So what I think I might try is modifying that script to change the OS type and see what happens. If you can live with configuration time changes, look for $ osname = F$EDIT(F$GETSYI(NODE_SWTYPE),COLLAPSE) in configure.com and modify it to say whatever you want. I missed changing that to only be set if osname was not previously defined, so my last build was still with osname=VMS. I did find the test later on that bails if the osname .nes. VMS It turns out now, that perl will not build if the osname is not VMS, it is failing on opendir(./../../), and I have not found a way to get around that yet. So it looks like I need to change the osname after the build is complete. The value in config.h is not used by any of the C code. -John [EMAIL PROTECTED] Personal Opinion Only
Re: Debugger - was: Changing osname or ^O from VMS.C at startup?
At 3:58 PM -0600 2/15/05, John E. Malmberg wrote: Apparently the MMK/FROM_SOURCES TEST is creating the non debug version for it's use, and that is causing the tests to fail. So I need to modify that command in my master build file to also have the /macro=__DEBUG__=1 for that MMK command. Yes, sorry, I assumed that. Use the same macros when making the test target as when doing the build. -- Craig A. Berry mailto:[EMAIL PROTECTED] ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
Re: Changing osname or ^O from VMS.C at startup?
Craig A. Berry wrote: At 5:14 PM -0600 2/15/05, John E. Malmberg wrote: On Tue, 15 Feb 2005, Craig Berry wrote: The global PL_osname is set in S_init_predump_symbols in perl.c, which is called from S_parse_body when a Perl script is compiled. If you put in your own value in start-up code, I'm pretty sure it will get overwritten later. You could stick some code in S_init_predump_symbols to check for a logical name and reset accordingly. That might be a little dangerous since we don't really know what side effects there might be to dynamically changing the OS name. Apparently the OS name is dynamically set now by a perl script, config.p*, and I can not find a pre-compiled version of it. So what I think I might try is modifying that script to change the OS type and see what happens. That's not dynamic in the sense of being set at run time. Config is hard-wired to return the same value that's in config.h: $ perl -V:osname osname='VMS'; config.h has the C macro OSNAME, which is in turn used by S_init_predump_symbols as I mentioned above. Config the Perl module and config.h the C header both get their values from config.sh, which is generated at configuration time by configure.com. I did a search of the source code before I posted, while the osname macro is defined in config.h, I can not find anything in the perl .C files that reference any of the osname macros in any of the .H files, or any macros that would hide such references. I will have to look at the S_init_predump_symbols to see if that gives me any more clues. Thanks, -John Personal Opinion Only
