VAX/VMS 7.3 and build from copy of master.
[bcc Hartmut Becker per request of Paul Smith] The attach patch allows VMS to build from a clone of the master branch of the GNU make repository. The config.h-vms.template was using macros that were not predefined by the compiler. The macros that were in use are only defined after a VMS header file is included, so were preventing the GNU Make 4.0 from building on VMS older than 7.3-2. The config_h_from_vms_template.com converts the config.h-vms.template to a config.h-vms file It is reading the version string from NEWS. as that is the only place that I found it. If there is a better place to read the version, I can adjust the file. The gmk_default_h.com is used to generate the gmk-default.h file, that the makefile.vms is expecting. It appears that the guile module is currently not using that header file when built on VMS. The makefile.com and makefile.vms were adjusted to build from master as well as a release. I tried to get the GNU make 4.0 tarball to build on VAX/VMS 7.3. It failed because of the header macro bug in the config.h-vms file, and the then requirement of vsnprintf(). After bringing over the vsnprintf() from the gawk project. This got gnu make building with almost no output on VAX/VMS 7.3 The lack of output was caused by a buffer allocation bug in output.c. That buffer allocation bug does not exist in the output.c in master and master does not need the vsnprintf() routine. This patch was tested against master on VAX/VMS 7.3 and IA64/VMS 8.4. I am now looking at what it will take to get the perl scripts for testing to run on VMS. Regards, -John 0001-VAX-VMS-7.3-and-build-from-master.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VAX/VMS 7.3 and build from copy of master.
On 2/20/2014 9:52 AM, h.becker wrote: 1. building from a git-cloned repository config_h_from_vms_template.com is a rather long name, no problem. However, there seems to be something similar for w32: Windows32 SCM build preparation of That file is named prepare_w32.bat. So I suggest to rename config_h_from_vms_template.com to prepare_vms.com. OK, the lowercase vms/w32 is not consistent with other file names which have VMS/W32, if that is important ... Ironically the file case is significant if you are serving files via NFS to VAX/VMS. When doing that, only lower case filenames show up correctly on the VAX. The rest get NFS mangled. I am ok with prepare_vms.com but prefer a name that describes the function, since a prepare_vms.com could mean doing many things, and this is a specialized command procedure. I would rather use configure.ac to determine the package and the version. Now that I know where it that information is, I would agree. I was searching for the word version in that file. I did not try just reading it. Whether it is useful to check for NFS/ODS-2 encoded filenames, I don't know. But then, when a git clone is copied to an ODS-2 disk, somehow, how would one handle filenames with multiple dots, anyway. I suggest to simply require an ODS-5 disk for git cloned repositories and leave it to the user of old VMS systems to handle problems on other disks. As I stated above, I am using a common NFS served disk for all my systems. So for me to build on VAX, I have to deal with the NFS mangled names. How a clone shows up on ODS-2 depends on what tool was used to unpack it. The user can change the symbol config_template, anyway. Maybe a comment should explain this, or maybe a P1 can be added so that a user can supply an ODS-2 name. To me it seems removing these special filenames makes things simpler. And, to make generating the config.h-vms more obvious I would use some replace commands. Again, I use a new feature: pipe, here to avoid creating a temporary file. Some VMS utilities do not play well with random accessing files on an NFS volume, particularly before VMS 8.4. So I use a more restricted set for building. I did not create any temporary files in my scripts. Whether invoking this command procedure should go into makefile.com is another question. I think it isn't necessary. The prepare_w32.bat isn't run from any other .bat-file, as far as I can see. I think it should go there, then we have the same build procedure on master as in a release tarball. Eventually I hope to get to a single command file that will do the complete build and create a PCSI kit with no extra editing. 2. Compiling and linking guile Yes, after the release of 4.0 there was a change which added a call to guile_gmake_setup to main.c. I sent a patch to handle that for VMS, in last November. Obviously it didn't make it into the repository. But I don't see that you need gmk-default.h without having HAVE_GUILE defined. To me it seems enough to change makefile.com and makefile.vms as in the appended diff/patch and then there is no need for gmk_default_h.com. I am not sure yet what the guile module can do on VMS, so I would prefer to leave generating the header file in place until that is determined. 3. Changes in config.h-vms.template Isn't it more obvious to do what decc$types.h does? #ifndef __CRTL_VER # define __CRTL_VER __VMS_VER #endif It is also not a good idea. The double underscore macros should only be redefined as a last resort. Doing the above hack will also unexpectedly impact people doing cross version builds. Are you subscribed to bugs-make or equivalent, or do you want me to continue to bcc you? I will try to put together a revised patch this weekend. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VAX/VMS 7.3 and build from copy of master.
What I am proposing: 1. Rename config.h-vms.template to be config-h-vms.template (or similar) so that it is the same on all VMS volumes. Include this in the release tarballs. 2. Put a #include types.h near the start of config-h-vms.template so that __CRTL_VER is properly defined. 3. Have a procedure that converts the config-h-vms.template to config.h. Since the information needed for this is in the configure.ac file. 4. At this point we longer need the config.h_vms file at all so it no longer needs to be generated as part of making the release tarball. This simplifies things. 5. makefile.com will now always generate the config.h from config-h-vms.template. 4. Remove the gmk-default.h dependency from makefile.vms until such time that code becomes usable on VMS. This way the same procedures are used for the build on masters or on tarballs so that they get fully tested. On 2/21/2014 6:40 AM, h.becker wrote: Btw. are all NFS clients for VAX/VMS doing the identical file name mapping? Yes. This seems to indicate that handling all of the possible file names for config.h-vms.template is even more work. Not really. I only need to deal with NFS name mangling at this time. I don't understand your comment on NFS files and VMS utilities. A temporary file would go to SYS$SCRATCH, or? Anyway, I suggested to make it more obvious what is replaced. The script with reading the file and processing it line by line is less obvious to me, maybe because the to be replaced tokens didn't show. It is some of the utilities reading from the NFS volume that have the problem. They hit some sort of issue where the process hangs for 8 hours. My opinion here is that just for NFS-problems and only for building from a snapshot it isn't worth to add complexity to the scripts. Agreed, the pipe command isn't simple, either. I use this also for building from tarballs, and this is the only the first step in the GNV merge where I am getting the existing master to build on all the VMS versions that I have installed. Eventually I plan to have a single procedure that will build from source and produce a PCSI kit with one command, and I will have to deal with the NFS name issues at several points. How far in VMS versions do you want to go back that there is no pipe command available? It is not the issue of PIPE. It is the issue that some NFS read operations from VMS utilities randomly fail on VMS 8.3. alpha. The DCL reading through the file always works. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VAX/VMS 7.3 and build from copy of master.
On 2/24/2014 3:03 PM, h.becker wrote: On 02/24/2014 03:10 AM, John E. Malmberg wrote: On 2/22/2014 7:03 AM, h.becker wrote: On 02/22/2014 01:19 AM, John E. Malmberg wrote: What I am proposing: 1. Rename config.h-vms.template to be config-h-vms.template (or similar) so that it is the same on all VMS volumes. Include this in the release tarballs. Renaming the template file is not that easy as it is included in a list rule in mainMakefile. A significant rewrite would be needed. I didn't look at maintMakefile, maybe I should have done before suggesting a change in the name. There is $ make -nf maintMakefile config.h-vms rm -f config.h-vms sed -e 's@%VERSION%@@g' \ -e 's@%PACKAGE%@@g' \ config.h-vms.template config.h-vms chmod a-w config.h-vms $ Which with config-h-vms in TEMPLATES would become a rm -f config-h-vms sed -e 's@%VERSION%@@g' \ -e 's@%PACKAGE%@@g' \ config-h-vms.template config-h-vms chmod a-w config-h-vms Then the VMS command procedure and the makefile.vms have to change and copy from config-h-vms. instead of config.h-vms. Not really nice but managable. Then there is an entry for config.h-vms in .gitignore, which would need to change as well. Am I missing something else? VMS does not always like files with no extensions. Probably would not be an issue here. I do not know if Paul is following this thread, as a rename of the file would need to be tested with their other procedures. Therefore I have the prepare_vms.com handle both the ODS-5 (normal version) and the NFS mangled format for ODS-2 volumes. The prepare_vms.com (is still invoked in makefile.com, which I still don't understand) Because computers work for us, not us for computers. So if the script can figure out a step to be done, there is no reason for to have it as an extra manual step. We should have one command file that completely build GNU make, not several steps. handles config.h-vms.template (why is there a prefix with sys$disk:[]?) and the NFS/ODS-2 variant of that: CONFIG.H-VMS$5NTEMPLATE. The prefix of sys$disk:[] is used in many existing places. One of the things that it helps is bulding with the default on a search list. I have a search list of prj_root = lcl_root:,vms_root:,src_root: where all changes by the build scripts get written into lcl_root: lcl_root: is specific to the system that the build is on. vms_root: and src_root: are common to all the build systems. vms_root: is work in progress changes src_root: is pointed either at a checkout of master or an unpacked release tarball as needed. When I unpack a snapshot on an ODS-2 disk from a tar archive (with vmstar) I get CONFIG.H-VMS_TEMPLATE and from a zip archive I get CONFIG_H-VMS.TEMPLATE. Why is NFS preferred over the other two variants? Because I do not use vmstar, or was aware that anyone else was in this case, so I only added in the variants that I needed. When there is special code for one file name variant I don't see why there shouldn't be special code for the other ones - or none at all. Then I can add in the code for the existing vmstar variant. Again, there is no problem with the current released tarball, which contains a file which is named either config.h-vms or CONFIG.H-VMS on the VMS disks. Are there other file name variants, which I don't know? Pathworks V5/older and some SAMBA - CONFIG.H-VMS__2ETEMPLATE Pathworks V6+ and newer SAMBA - CONFIG_2EH-VMS.TEMPLATE Some older SAMBA variants will invert case when a $ is in the filename just like NFS does. Does not apply in this case. I would like to see vmstar enhanced to know about these filename encodings. So far, the best one could do is to f$search for config*vms.template and config*.*vms*template. I'm still not convinced that this should be there at all but that could be a compromise or a catch-almost-all/catch-more variant. Two f$search templates appear to be able to find all of the above encodings. I have update the prepare_vms.com procedure. 2. Put a #include types.h near the start of config-h-vms.template so that __CRTL_VER is properly defined. Done. No other formatting done to the file at this time. 3. Have a procedure that converts the config-h-vms.template to config.h. Since the information needed for this is in the configure.ac file. I agree to have this for builds from a snapshot. prepare_vms.com just makes that config.h_vms file. I had to modify the pipe command to fit the command line length restrictions of VAX/VMS 7.3. EDT is not one of the utilities that has a problem reading from NFS files. With reading from configure.ac it seems, the labels news_read_loop* should change as well. I missed that. Changing to ac_read_loop/ac_read_loop_end Now there is a f$$search in makefile.vms. I don't think it is a problem when the files aren't there, the - can handle this. What is the reason to check with extra DCL commands whether the files are present
Re: VAX/VMS 7.3 and build from copy of master.
Revised patch with HAVE_VMSDIR_H fix. Regards, -John 0001-VMS-Build-from-Master-and-VAX-VMS-7.3-revised-2.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VAX/VMS 7.3 and build from copy of master.
On 2/25/2014 5:45 PM, h.becker wrote: On 02/25/2014 01:36 AM, John E. Malmberg wrote: Because computers work for us, not us for computers. So if the script can figure out a step to be done, there is no reason for to have it as an extra manual step. We should have one command file that completely build GNU make, not several steps. From the released tarball: yes; from the snapshot: no. It is still my opinion that building from the snapshot is not the default. Default of non-default has nothing to do with it. Not having the makefile run the prepare_vms.com if needed just makes more work for me every time I have to rebuild the world. I have removed it, because we need to move on. I do not see why building from master should take more manual steps than building from a release tarball. It is just more work for the programmer. I will just create a local single command file that runs everything for now. handles config.h-vms.template (why is there a prefix with sys$disk:[]?) and the NFS/ODS-2 variant of that: CONFIG.H-VMS$5NTEMPLATE. The prefix of sys$disk:[] is used in many existing places. One of the things that it helps is bulding with the default on a search list. ... Maybe it is too late, here. I don't see the difference in accessing the file with sys$disk:[]config.h-vms or config.h-vms. It is an issue on search lists, in some cases if the sys$disk:[] is missing, the file may not get found due to the stickyness of DCL file handling. I may be putting in a few more places than absolutely needed just as a defensive measure. I had to modify the pipe command to fit the command line length restrictions of VAX/VMS 7.3. EDT is not one of the utilities that has a problem reading from NFS files. With the sys$disk:[]-prefix, the command token for the pipe command gets bigger. The token is the restriction, the command line itself can be longer. Using the result of the f$search for the templates makes this command even longer, as the full VMS file spec is returned. And then the pipe can break depending on the current environment: how deep in a directory tree the file is found, that is, where the default directory is. Therefore I suggested to f$parse the name, after the file is found. I have put that in, but formatted it to fit 80 column lines. Now there is a f$$search in makefile.vms. I don't think it is a problem when the files aren't there, the - can handle this. What is the reason to check with extra DCL commands whether the files are present, the ignored make error? (And then there is no need for the '- any more.) What kind of problem should this solve? It gets rid of the error messages written to SYS$OUTPUT. The '-' can be removed. I did not do it on this latest patch. These are usual error messages when the files are not present and the ignore prefix takes care of that. Anybody using make is probably aware of that. I would not count on that. Yes, as I have found many issues in other projects where serious real errors and warnings were ignored because there were so many other warnings and errors that were apparently harmless. I don't think this applies here. But for whatever it is good, I changed makefile.com to place the objects in the same directory as the source, the place where the makefile.vms expects them. And a clean target needs to deal with aborted build, so should not generate error messages when the target directory is already clean. In my opinion, with an aborted build, most people will expect to see such messages from make. I think it is sloppy to have error messages for expected conditions. Regards, -John 0001-Build-from-Master-and-VAX-VMS-7.3-revision-3.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Test harness on VMS working, 58 tests currently failing.
I have the test harness apparently working on VMS. I need to fix up the DCL command procedure that is needed set up a number of foreign commands needed by it before submitting a patch. If no one else is interested in immediately running the test harness on VMS, I can wait on submitting a patch until I determine what is causing all the test failures. So far it will require: Perl 5.18.1 or later. GNV 2.1.3 or later. Bash 4.2.45 or later. Coreutils 8.21 or later. This limits running the test harness on Alpha/VMS 8.3 and 8.4 and IA64 VMS 8.4. 58 Tests in 103 Categories Failed (See .diff* files in work dir for details) :-( As there appears to be 111 log files, this is just less than half the tests passing. I still need to investigate all of these failures. The features/archives test fails because the GNV AR utility requires real VMS object files at this time, and the test just creates some files. I will need to adjust the test setup section for VMS. Some of the other tests are failing because the VMS sets argv[0] differently than what is expected. I have a fix from the Bash 4.2.45 port that will handle that. 3 tests timed out after 5 seconds. Here is an example of a passing test. EAGLE perl run_make_tests.pl features/comments -- Running tests for GNU make on VMS GNU Make 4.0.90 -- Clearing /lcl_root/make/tests/work... Making work dirs... features/comments ... ok (1 passed) 1 Test in 1 Category Complete ... No Failures :-) Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS working, 58 tests currently failing.
On 2/28/2014 8:07 PM, John E. Malmberg wrote: I have the test harness apparently working on VMS. I need to fix up the DCL command procedure that is needed set up a number of foreign commands needed by it before submitting a patch. If no one else is interested in immediately running the test harness on VMS, I can wait on submitting a patch until I determine what is causing all the test failures. So far running the test harness on VMS will require: Perl 5.18.1 or later. GNV 2.1.3 or later. Bash 4.2.45 or later. Coreutils 8.21 or later. This limits running the test harness on Alpha/VMS 8.3 and 8.4 and IA64 VMS 8.4. 58 Tests in 103 Categories Failed (See .diff* files in work dir for details) :-( As there appears to be 111 log files, this is just less than half the tests passing. Investigation showed two additional VMS specific issues in the test harness that were affecting the test run. After fixing those issues: 69 Tests in 90 Categories Failed (See .diff* files in work dir for details) :-( As there were 126 log files created, this indicates that more tests were being run. I do not know why the category count changed. Still more than 1/2 the tests are failing. If I determine that a test failure is an actual bug in the VMS port instead of a feature of the implementation do you want tickets filed to track the issue, or just the the patch to fix it if it is possible to fix? Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS working, 58 tests currently failing.
On 3/2/2014 6:11 AM, h.becker wrote: On 03/01/2014 09:50 PM, John E. Malmberg wrote: On 2/28/2014 8:07 PM, John E. Malmberg wrote: I have the test harness apparently working on VMS. I need to fix up the DCL command procedure that is needed set up a number of foreign commands needed by it before submitting a patch. For which version? I assume for the current development stream from the git repository, correct? This is for master of course. If no one else is interested in immediately running the test harness on VMS, I can wait on submitting a patch until I determine what is causing all the test failures. So far running the test harness on VMS will require: Perl 5.18.1 or later. GNV 2.1.3 or later. Bash 4.2.45 or later. Coreutils 8.21 or later. From what I see, the tests are coded in perl. Can you give a short description for what the other stuff is needed? The make scripts expect a number of GNU utilities to be present and for Perl to use UNIX filename syntax. The following commands are used by makefiles in the tests. ar, cat, cc, cp, diff, echo, false, mkdir, rm, sleep, sh, and touch. GNV provides the ar, and cc commands. I have not updated them yet. The bash in GNV is ancient and buggy. The Bash 4.2.45 is much more reliable. Coreutils supplies the rest of the utilities. While they are also supplied by GNV, the Coreutils kit is more up to date with significant bug fixes. Example: rm -rf actually works. It appears that GNU make on VMS is attempting to emulate touch. This is showing up in several test failures where the touch command is used in the makefile. So it looks like that emulation needs to be improved. So I will need to have a DCL procedure set these up as foreign commands temporarily before running the run_make_tests.pl. I have not yet finished that script. My plan get the tests that can be run to pass before merging in the GNV fork which runs the Bash shell from GNU Make. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41760] VMS outputting extra blank lines.
On 3/3/2014 1:08 PM, h.becker wrote: On 03/03/2014 05:55 AM, John Malmberg wrote: URL: http://savannah.gnu.org/bugs/?41760 Summary: VMS outputting extra blank lines. Project: make Submitted by: wb8tyw Submitted on: Mon 03 Mar 2014 04:54:59 AM GMT Severity: 3 - Normal Item Group: Bug Status: None Privacy: Public Assigned to: None Open/Closed: Open Discussion Lock: Any Component Version: 4.0 Operating System: VMS Fixed Release: None Triage Status: None ___ Details: VMS is outputting extra blank lines in the output. For example a recipe line @continue results in a blank line in the output. I am patching my local copy of the test harness to not consider this a test failure until I can fix this. A simple makefile with all: @continue doesn't exhibit this, can you give an example? It shows up in the recorded output for the test harness for features/conditionals test 3, after the makefile has been modified to use VMS syntax on when the test is run on VMS via DCL. It does not show up when running the test interactively. When I redirect sys$output to a file, I end up with two files instead of the one that is expected. So there are a couple of odd things going on here. I am not seeing the blank lines on other tests. I have not converted all the tests to use VMS syntax for the recipe lines though. Regards -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms argv[0] and exit status fixes.
New patch attached as described below. On 3/5/2014 3:30 PM, h.becker wrote: I'm still looking at this, so I may change my mind or suggest to change the code :-) The main wrapper: There were/are problems with the program name, however I'm not sure whether the wrapper solves them. For DCL the wrapper, as before, requires a foreign command pointing to the executable (or the logical DCL$PATH pointing to the directory with the matching executable). snip but when you want to use a different symbol: $ del/symb make $ gmake := $USR_ODS5:[BECKER_H.make.make]make.exe $ gmake -f MAKE_COMMAND.mf make $ gmake -f MAKE_COMMAND.mf x make -f x.mf %DCL-W-IVVERB, unrecognized command verb - check validity and spelling \MAKE\ $ There is no way that I know of to know what foreign symbol was used to invoke an image. All we get is the image path in one of three formats based on what the decc$ feature settings. I already made some changes for this, but I'm not yet convinced - it's not really nice and there may be other side effects: I think that a solution is to convert the original arvg[0] to VMS format if needed and then use lib$set_symbol() to create a foreign command that is visible to the child programs with the new argv[0]. So lets try that with the make symbol missing, but a gmake symbol defined. EAGLE make -f makecommand.mf x %DCL-W-IVVERB, unrecognized command verb - check validity and spelling \MAKE\ EAGLE gmake -f makecommand.mf x make -f x.mf here we are The side effect is that it leaves a local symbol make behind. Not too bad. That source module is or can be a common source file for the latest ports of GNU bash, grep, and sed. I made setting the symbol a compile time option, since those other programs do not need it set. SYS$SCRATCH: I know that the VMS CRTL does some special handling for /tmp. As far as I know, this is translated to SYS$SCRATCH:. So there seems to be no need to replace sys$scratch with /tmp. But tmp (or TMP?) may be a logical name pointing to a different directory and then that directory is used as scratch directory. Whether that is a good thing or just confusing, is another question. SYS$SCRATCH seems more obvious to me. If someone has defined /TMP to other than SYS$SCRATCH: then they will probably be suprised if a GNU utility is not honoring it and still using SYS$SCRATCH:. exit - vms_exit: Looks good to me, except MAKE_FAILURE is mapped twice and MAKE_TROUBLE isn't mapped. A bug, easily fixed. If possible I would move the vms_exit function into a vms specific C source, either an existing one or a new one. I would move any other vms wrapper in such a module as well and reduce the to be included stuff to usual header file content. (OK, I know there is vmsjobs.c) If we move it out of a header file, it is no longer an in-lined static function. If die() is the only thing that calls exit, we can put it in the main.c module. The other place for it would be in the config.h-vms.template file. With the previous patch posted: 270 Tests in 75 Categories Failed (See .diff* files in work dir for details) :-( With the current patch posted: 270 Tests in 75 Categories Failed (See .diff* files in work dir for details) :-( No changes. I think I finally have the test harness running correctly on VMS. Regards, -John 0001-VMS-argv0-and-exit-status-fix-revision-1.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms argv[0] and exit status fixes.
On 3/6/2014 6:14 PM, h.becker wrote: I'm still looking at this. I moved the code to places where I think they should go, but I'm not yet done. But I have some comments: On 03/06/2014 06:18 AM, John E. Malmberg wrote: ... There is no way that I know of to know what foreign symbol was used to invoke an image. All we get is the image path in one of three formats based on what the decc$ feature settings. For DCL it will be only one format. Not true. The decc$ features can be set by logical names, and a user may have set them for the various formats. If a C program can be broken by an incorrect decc$ feature name, it should protect itself with a lib$initialize section. I have a standard one that can be used for almost all programs that detects if the program is running under DCL or GNV Bash and sets the features as would normally be expected. I think that a solution is to convert the original arvg[0] to VMS format if needed and then use lib$set_symbol() to create a foreign command that is visible to the child programs with the new argv[0]. ... The side effect is that it leaves a local symbol make behind. Not too bad. Hmm, silently making a symbol, maybe even silently overwriting an existing one? I disagree. How about this, I append the process ID string to the symbol to make it unique, and then use an at_exit() call to delete it. That way the makefile if the makefile goes to use a symbol like make by name instead of the macro, it will get what the user set make to be. That source module is or can be a common source file for the latest ports of GNU bash, grep, and sed. OK, but that's not applicable, here. It simplifies my porting efforts, when I can just drop in modules as needed with out editing the original source. I made setting the symbol a compile time option, since those other programs do not need it set. I agree, but I would want to have these symbols definable in config.h-vms[.template] This program wraps around the main() module, so either must be included into main either implicitly, or with a /first_include feature. I figured the macros should be set near the #include. The first_include feature is not available on VAX/VMS. If someone has defined /TMP to other than SYS$SCRATCH: then they will probably be suprised if a GNU utility is not honoring it and still using SYS$SCRATCH: For DCL, I don't think so. What ever is done, it needs to be documented in the readme.vms and probably the gnu make manual as a VMS feature. As clear To have it different under GNV will require a global variable to use as a flag to be set if getenv(SHELL) returns a value, as I do think we want to call gentenv(SHELL) every place GNV needs alternative behavior. exit - vms_exit: Looks good to me, except MAKE_FAILURE is mapped twice and MAKE_TROUBLE isn't mapped. A bug, easily fixed. It seems MAKE_TROUBLE is used with -q, so it is more an informational message. But I haven't yet checked whether MAKE_TROUBLE is used anywhere else. If make trouble is informational, then the severity can be adjusted. If possible I would move the vms_exit function into a vms specific C source, either an existing one or a new one. I would move any other vms wrapper in such a module as well and reduce the to be included stuff to usual header file content. (OK, I know there is vmsjobs.c) If we move it out of a header file, it is no longer an in-lined static function. Correct. But is in-lining necessary for a function which is called only once? Probably not. If die() is the only thing that calls exit, we can put it in the main.c module. The other place for it would be in the config.h-vms.template file. I prefer to have all vms specific code in the existing vms sources, vmsfunctions.c may be the one for the new code. I would prefer multiple modules, so that they can be re-used between other projects. A vmsfunctions.c could #include these modules. Also, it seems a lot of code just to adjust/manipulate argv[0], I would like to keep it simple. Adding an mcr prefix is simple. I agree, that may not work for gnv and I didn't do much testing with it. My concern is if a macro expands to two words instead of one, it could unexpectedly break something. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms argv[0] and exit status fixes.
On 3/8/2014 6:15 AM, h.becker wrote: On 03/07/2014 02:24 PM, John E. Malmberg wrote: On 3/6/2014 6:14 PM, h.becker wrote: ... For DCL it will be only one format. Not true. The decc$ features can be set by logical names, and a user may have set them for the various formats. It as a user/administrator error to enable a decc$ feature with a logical name other than /user prior to run the program which relies on that feature (and is incorrectly coded not to use image initialization code). Only if the user is aware of that restriction, which most are not. Many programs set feature logical names for their own use and then LIB$SPAWN children, and sometimes like GNU Make, the children can be any program. If a C program can be broken by an incorrect decc$ feature name, it should protect itself with a lib$initialize section. I disagree. The decc$ feature design is broken and should be fixed, maybe with a feature logical :-). Any program which relies on a decc$ feature should set it with image initialization code (or any other appropriate interface provided by the crtl). But that is another discussion and doesn't seem to belong here. You say you disagree, yet your statement agrees with mine. The lib$initialize section is the documented method that the CRTL states to use to accomplish this. And I agreee that the decc$ feature design is a bit broken, as in some cases it has the run-time setting of features that should be set at compile time. How about this, I append the process ID string to the symbol to make it unique, and then use an at_exit() call to delete it. I suggest to have it settable in config.h-vms[.template]. That would hard code it to a specific name. Do you have a suggestion for a such a name that will not collide with something a user has set and could expect to be visible in a makefile? This program wraps around the main() module, so either must be included into main either implicitly, or with a /first_include feature. As far as I see and coded it: no. The only reason to include code into other code is when the included code has to access static variables/functions of the includer. As far as I see, that's not the case. The original main function name needs to be changed so that the code can wrap it. The wrapper code can reside anywhere, but should remain in its own module so that it can be deployed to other projects with out editing. I am not just doing one project, I am doing a whole suite of projects and am reusing as much code as possible for the common issues. If someone has defined /TMP to other than SYS$SCRATCH: then they will probably be suprised if a GNU utility is not honoring it and still using SYS$SCRATCH: For DCL, I don't think so. What ever is done, it needs to be documented in the readme.vms and probably the gnu make manual as a VMS feature. As clear To have it different under GNV will require a global variable to use as a flag to be set if getenv(SHELL) returns a value, as I do think we want to call gentenv(SHELL) every place GNV needs alternative behavior. Again, I would want to make it settable in config.h-vms[.template]. Maybe it is necessary to create a config.h-gnv[.template], but I don't know, yet. And yes, it needs to be documented. Since there will only be one build procedure and one resulting binary, there would not be two templates. One of the main reasons for merging this fork back in is to get to one make binary. The one binary where needed will check if it is running under a UNIX shell or DCL and adjust its behavior as needed. This is the way that Curl and Perl now work. That way users do not have to track two implementations of the same version of product. On the other hand, P_temdir is defined in the recent version of stdio.h as SYS$SCRATCH:, so I'm not sure whether your change to use /tmp will work. And it seems like the RTL engineers or whoever wrote stdio.h think this is what developers/users expect. That bug has been previously reported to HP. It actually will cause a run-time error depending on the decc$feature settings. VMS format files specifications can not be exposed as constants in the header file. But the reality is that if someone sets the TMP: logical to something other than SYS$SCRATCH:, they may not find some of their temporary files in TMP: and some in SYS$SCRATCH:, and I think that needs to be noted in the VMS specific documentation of the application, as it will not be obvious. After all there very well could be cases where the CRTL is using /tmp instead of sys$scratch because it implemented the automatic conversion. I prefer to have all vms specific code in the existing vms sources, vmsfunctions.c may be the one for the new code. I would prefer multiple modules, so that they can be re-used between other projects. A vmsfunctions.c could #include these modules. The reason I want to use existing ones is simple: I don't want to add so many vms specific source
Test harness on VMS, running 512 tests, 68 tests failing.
It has taken a few passes at finding all the issues. Originally I found that test scripts were being aborted on the first test failure, so I did not have the full count of tests. Then I had to fix the command options to be quoted to preserve the case, and filter the output for the intentional and some possibly unintentional differences in the text reported. Currently the status is 68 failures in 31 categories out of 512 test log files found. I am still investigating all the failures. For most of the failures, the syntax of the test makefile recipes is using Shell syntax that is not being simulated on VMS, so I have to start modifying the tests to generate VMS compatible syntax. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Make recursion does not appear working in VMS
On 3/12/2014 7:10 AM, John E. Malmberg wrote: I converted the features parallelism.mk_1 test to VMS and if I run it with out the two generated 1.inc and 2.inc files, it always fails to find the rules specified by them. It does generate the files correctly and the output up to that point. As near as I can figure out, it means that make is not doing its recursion to re-run the makefile on VMS. Further tests show that the parallelism is not fully working on VMS, or not working as a Unix program would expect. Of the 9 parallelism tests, 4 work, 4 fail, and one is not current valid to run on VMS. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 512 tests, 68 tests failing.
On 3/10/2014 11:00 PM, John E. Malmberg wrote: Currently the status is 61 failures in 30 categories out of 511 test log files found. I do not know why the category count changed from just skipping one test in a file. It also appears the some tests result counts are affected by the -keep qualifier since it leaves files around that other tests expect not to be there. So there may be a few more or less failures. With out -keep, 64 Tests in 30 Categories Failed, as an example, but then I do not get a count of all the log files. 5 archive tests fail, because that code is not completely implemented on VMS. Ticket 41761. 1 override test fails because multi-line defines not correctly implemented on VMS. Ticket 41813 Of the 9 parallelism tests, 4 work, 4 fail, and one is not current valid to run on VMS. 1. Make recursion appears not to be working, at least with parallelism enabled. 2. Parallelism is not working the same on VMS as UNIX. Needs more investigation. The feature patternrules test 4 after being converted to use VMS syntax is exiting with a status of 5584176 or 0x03538110, which is an invalid exit code. Need to find out where that is coming from. It may be the result of the new vms_exit() function getting a VMS exit code instead of the expected UNIX exit code. I am temporarily having the test harness interpret that as a MAKE_FAILURE since that is what the test expects until that issue is resolved. Also need to document the VMS syntax for makefiles. I can not find it in the readme.vms file or anywhere else in the checkout of master. I am having to figure it out based on the logs from the tests and some trial and error. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Make recursion does not appear working in VMS
On 3/13/2014 4:19 AM, h.becker wrote: On 03/13/2014 05:29 AM, John E. Malmberg wrote: Further tests show that the parallelism is not fully working on VMS, or not working as a Unix program would expect. As far as I know, --jobs is not implemented for VMS, unfortunately I can't point to an obvious line in the sources. It seems to be hidden in #if[n]def VMS in job.c and vmsjobs.c. Silently explicitly or implicitly reseting jobs to 1 isn't good. To look at this, can you post or send (one of) the modified makefiles for VMS? Sorry for long lines, only a seconds before I have to run. recurse line is wrapped in the e-mail client. 1.inc and 2.inc lines are wrapped in the e-mail client. This is one of the files that fails for recursion. I will have to send one that shows the parallelism failures later. EAGLE del 1.inc;,2.inc; EAGLE EAGLE EAGLE make -f /lcl_root/make/tests/work/features/parallelism.mk_2 -j4 THREE.inc ONE.inc TWO.inc make: execve: make: no such file or directory /lcl_root/make/tests/work/features/parallelism.mk_2:2: recipe for target 'recurse' failed make: *** [recurse] Error 0xbf2bffbc Or if you prefer: EAGLE del 1.inc;,2.inc; EAGLE make -f [.work.features]parallelism.mk_2 -j4 THREE.inc ONE.inc TWO.inc make: execve: make: no such file or directory [.work.features]parallelism.mk_2:2: recipe for target 'recurse' failed make: *** [recurse] Error 0xbf2bffbc Directory LCL_ROOT:[make.tests.work.features] parallelism.log_2;1 parallelism.mk_2;1 Total of 2 files. EAGLE type LCL_ROOT:[make.tests.work.features]parallelism.mk_2 recurse : ; @$(MAKE) --no-print-directory -f /lcl_root/make/tests/work/features/ parallelism.mk_2 INC=yes all all : 1 2 ; @echo success INC = no ifeq ($(INC),yes) -include 1.inc 2.inc endif 1.inc : ; @pipe echo ONE.inc ; sleep 2 ; echo TWO.inc ; define/user sys$$output $@ ; echo 1 : ; @pipe echo ONE ; sleep 2 ; echo TWO ; close sys$$output 2.inc : ; @pipe sleep 1 ; echo THREE.inc ; define/user sys$$output $@ ; echo 2: ; @pipe sleep 1 ; echo THREE ; close sys$$output Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Make recursion does not appear working in VMS
On 3/13/2014 8:14 AM, h.becker wrote: It looks like execve uses argv[0] to run make. With a VMS/DCL-style version of the Makefile and my mcr version of make it seems to work: I changed from the wrapper code to code that sets the makefile name, and temporarily set a local symbol as a foreign command. At exit the symbol is either deleted or replaced with the original value, which leaves argv[0] alone. I also found that the vms_exit() code had a bug in it, because it was written with the expectation of getting a UNIX exit status, and GNU make is passing it VMS exit statuses. As only codes 0, 1, 2 are defined for GNU make status, and the lowest failure status from VMS is 8, for GNU make, the conversion to VMS status should only be done for codes 1-7, the rest should be passed through. With that those two changes, I now have: EAGLE del lcl_root:[...]*.*;*/exc=(*.dir,*.pm) EAGLE perl run_make_tests.pl features/parallelism -- Running tests for GNU make on VMS GNU Make 4.0.90 -- Clearing /lcl_root/make/tests/work... Making work dirs... features/parallelism Error running make (expected 0; got 512): make -f /lcl_root/make/tests/work/features/parallelism.mk_1 -j4 Error running make (expected 512; got 0): make -f /lcl_root/make/tests/work/features/parallelism.mk_3 -rR -j5 FAILED (6/8 passed) 2 Tests in 1 Category Failed (See .diff* files in work dir for details) :-( The first failure is that the recursion is not happening: The 1.inc and 2.inc recipe lines are probably wrapped in the e-mail client. LCL_ROOT:[make.tests.work.features]parallelism.log_1;1 THREE.inc ONE.inc TWO.inc make: *** No rule to make target '1', needed by 'all'. Stop. LCL_ROOT:[make.tests.work.features]parallelism.mk_1;1 all : 1 2 ; @echo success -include 1.inc 2.inc 1.inc : ; @pipe echo ONE.inc ; sleep 3 ; echo TWO.inc ; define/user sys$$output $@ ; echo 1 : ; @pipe echo ONE ; sleep 2 ; echo TWO ; close sys$$output 2.inc : ; @pipe sleep 2 ; echo THREE.inc ; define/user sys$$output $@ ; echo 2: ; @pipe sleep 1 ; echo THREE ; close sys$$output LCL_ROOT:[make.tests.work.features]parallelism.run_1;1 make -f /lcl_root/make/tests/work/features/parallelism.mk_1 -j4 The second failure is that the make prematurely exits on the first fail. It should have 3 failures. The test script currently converts the 0x1035a00a to 1 so that it matches. That should probably be done by Make. There should be 3 failure lines, not just one. EAGLE type LCL_ROOT:[make.tests.work.features]parallelism.*_3 LCL_ROOT:[make.tests.work.features]parallelism.log_3;1 Fail /lcl_root/make/tests/work/features/parallelism.mk_3:6: recipe for target 'fail.1 ' failed make: *** [fail.1] Error 0x1035a00a LCL_ROOT:[make.tests.work.features]parallelism.mk_3;1 .PHONY: all fail.1 fail.2 fail.3 ok all: fail.1 ok fail.2 fail.3 fail.1 fail.2 fail.3: @sleep $(patsubst fail.%,%,$@) @echo Fail @exit 271949834; ok: @sleep 4 @echo Ok done LCL_ROOT:[make.tests.work.features]parallelism.run_3;1 make -f /lcl_root/make/tests/work/features/parallelism.mk_3 -rR -j5 Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 511 tests, 34 tests failing.
On 3/13/2014 12:08 AM, John E. Malmberg wrote: On 3/10/2014 11:00 PM, John E. Malmberg wrote: I changed from the wrapper code to code that sets the makefile name, and temporarily set a local symbol as a foreign command. At exit the symbol is either deleted or replaced with the original value, which leaves argv[0] alone. I also found that the vms_exit() code had a bug in it, because it was written with the expectation of getting a UNIX exit status, and GNU make is passing it VMS exit statuses. As only codes 0, 1, 2 are defined for GNU make status, and the lowest failure status from VMS is 8, for GNU make, the conversion to VMS status should only be done for codes 1-7, the rest should be passed through. I had to fix a VMS specific timeout value on the parallelism test. With that those two changes, currently the status is 34 failures in 22 categoriesout of 511 test log files found. I do not know why the category count changed from just skipping one test in a file. It also appears the some tests result counts are affected by the -keep qualifier since it leaves files around that other tests expect not to be there. So there may be a few more or less failures. With out -keep, 33 Tests in 21 Categories Failed, as an example, but then I do not get a count of all the log files. The 5 archive tests now pass, which means that Ticket 41761 can probably be closed All the override tests also now pass even though multi-line defines not correctly implemented on VMS. Ticket 41813. Of the 9 parallelism tests, 6 work, 2 fail, and one is not current valid to run on VMS. 1. Make recursion appears not to be working, at least with parallelism enabled. 2. Parallelism is not working the same on VMS as UNIX. Needs more investigation. The feature patternrules still has one failure that needs to be investigated along with the other failures I have not yet gotten to. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41813] Mult-line override define failing on VMS.
On 3/16/2014 4:13 PM, h.becker wrote: Appended is a temporary fix for the multiline problem. As I mentioned in the make recursion thread: I changed from the wrapper code to code that sets the makefile name, and temporarily set a local symbol as a foreign command. At exit the symbol is either deleted or replaced with the original value, which leaves argv[0] alone. I also found that the vms_exit() code had a bug in it, because it was written with the expectation of getting a UNIX exit status, and GNU make is passing it VMS exit statuses. As only codes 0, 1, 2 are defined for GNU make status, and the lowest failure status from VMS is 8, for GNU make, the conversion to VMS status should only be done for codes 1-7, the rest should be passed through. EAGLE del lcl_root:[...]*.*;*/exc=(*.dir,*.pm) EAGLE perl run_make_tests.pl features/override -- Running tests for GNU make on VMS GNU Make 4.0.90 -- Clearing /lcl_root/make/tests/work... Making work dirs... features/override ... ok (4 passed) The archive tests are also now all passing. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Make recursion does not appear working in VMS
On 3/16/2014 6:14 PM, John E. Malmberg wrote: On 3/13/2014 8:14 AM, h.becker wrote: It looks like execve uses argv[0] to run make. With a VMS/DCL-style version of the Makefile and my mcr version of make it seems to work: I changed from the wrapper code to code that sets the makefile name, and temporarily set a local symbol as a foreign command. At exit the symbol is either deleted or replaced with the original value, which leaves argv[0] alone. I also found that the vms_exit() code had a bug in it, because it was written with the expectation of getting a UNIX exit status, and GNU make is passing it VMS exit statuses. As only codes 0, 1, 2 are defined for GNU make status, and the lowest failure status from VMS is 8, for GNU make, the conversion to VMS status should only be done for codes 1-7, the rest should be passed through. With that those two changes, I now have: EAGLE del lcl_root:[...]*.*;*/exc=(*.dir,*.pm) EAGLE perl run_make_tests.pl features/parallelism -- Running tests for GNU make on VMS GNU Make 4.0.90 -- Clearing /lcl_root/make/tests/work... Making work dirs... features/parallelism Error running make (expected 0; got 512): make -f /lcl_root/make/tests/work/features/parallelism.mk_1 -j4 Error running make (expected 512; got 0): make -f /lcl_root/make/tests/work/features/parallelism.mk_3 -rR -j5 FAILED (6/8 passed) 2 Tests in 1 Category Failed (See .diff* files in work dir for details) :-( The first failure is that the recursion is not happening: The 1.inc and 2.inc recipe lines are probably wrapped in the e-mail client. LCL_ROOT:[make.tests.work.features]parallelism.log_1;1 THREE.inc ONE.inc TWO.inc make: *** No rule to make target '1', needed by 'all'. Stop. LCL_ROOT:[make.tests.work.features]parallelism.mk_1;1 all : 1 2 ; @echo success -include 1.inc 2.inc 1.inc : ; @pipe echo ONE.inc ; sleep 3 ; echo TWO.inc ; define/user sys$$output $@ ; echo 1 : ; @pipe echo ONE ; sleep 2 ; echo TWO ; close sys$$output 2.inc : ; @pipe sleep 2 ; echo THREE.inc ; define/user sys$$output $@ ; echo 2: ; @pipe sleep 1 ; echo THREE ; close sys$$output LCL_ROOT:[make.tests.work.features]parallelism.run_1;1 make -f /lcl_root/make/tests/work/features/parallelism.mk_1 -j4 The second failure is that the make prematurely exits on the first fail. It should have 3 failures. The test script currently converts the 0x1035a00a to 1 so that it matches. That should probably be done by Make. There should be 3 failure lines, not just one. Found another bug in the exit_failure routine, and now second test is passing. So only the parallelism_mk_1 test is failing out of the set. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 511 tests, 60 tests failing.
On 3/16/2014 6:34 PM, John E. Malmberg wrote: On 3/13/2014 12:08 AM, John E. Malmberg wrote: On 3/10/2014 11:00 PM, John E. Malmberg wrote: The previous vms_exit fix was incorrectly changing some failures to success status. With that correction, I am back to 60 tests failing in 29 categories failing if I use the -keep option. With out -keep, 58 tests failiing in 38 categories fail. Only 3/10 archive tests are passing again, so ticket 41761 still applies. The override test is now failing with out the new fix from h.becker applied yet. Ticket 41813. Of the 9 parallelism tests, 7 work, 1 fails, and one is not currently valid to run on VMS. The feature patternrules is now working. It is what exposed the error in vms_exit. Regards -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41813] Mult-line override define failing on VMS.
On 3/16/2014 4:13 PM, h.becker wrote: Appended is a temporary fix for the multiline problem. The fix appears to work. I think your comment about it breaking the echo being defined as sys$output is wrong though. I tested it with no echo symbol and with echo as a symbol and it generates the expected output now in both cases. The next line is just processed as the next line in the recipe. Before this fix and with echo :== $gnv$gnu:[bin]gnv$echo: 60 Tests in 29 Categories Failed (See .diff* files in work dir for details) :-( After this fix, and with or with out a foreign command for echo: 54 Tests in 27 Categories Failed (See .diff* files in work dir for details) :-( Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41813] Mult-line override define failing on VMS.
On 3/18/2014 11:30 AM, h.becker wrote: On 03/17/2014 11:34 PM, John E. Malmberg wrote: On 3/16/2014 4:13 PM, h.becker wrote: Appended is a temporary fix for the multiline problem. The fix appears to work. I think your comment about it breaking the echo being defined as sys$output is wrong though. I tested it with no echo symbol and with echo as a symbol and it generates the expected output now in both cases. It seems you aren't using plain VMS, which has no echo command nor echo symbol. I'm talking about the make variable ECHO as defined for VMS in: The test works as long as echo is a foreign command because no macro substition is done. Because of a bug in my copy of the test harness, I was not seeing the errors about undefined symbols. The broken ECHO is one reason I flagged the fix temporary. It seems, there should be a builtin_echo for VMS. Shouldn't be too hard, but I haven't looked at it. That way some old VMS makefile which make use of ECHO will continue to work. The question is, whether one would want to support DCL symbols within the to be echoed text. Another question is how much of the Unix environment should be simulated when running Gnu make from DCL? Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 511 tests, 60 tests failing.
On 3/16/2014 11:27 PM, Paul Smith wrote: On Sun, 2014-03-16 at 21:43 -0500, John E. Malmberg wrote: With that correction, I am back to 60 tests failing in 29 categories failing if I use the -keep option. You should never use -keep when invoking the full test suite. I understand that. The -keep option is the easiest way to get a count of how many tests are actually getting run. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 489 tests, 141 tests failing.
A bug in having the test harness compare_output that was supposed to compensate for VMS differences in output caused a number of tests to report success that actually failed. Running with out -keep, so do not have an exact count of logs. Current status: 136 Tests in 54 Categories Failed. Known issues so far: features/archives: fail - 3/10 pass. Ticket 41758 - VMS archive support incomplete. features/override: with patch - pass 4/4. features/parallism: fail - 3/9 pass, 1 skipped. VMS not running jobs in parallel so output in wrong order. features/recursion: fail - 1/2 pass. Looks like it is expecting environment variables to pass information to a child, and child is not getting them. features/reinvoke: fail - 4/5 pass. One run is not interpreting a macro properly. features/se_explicit: fail 9/10 pass $(addsuffix ) and $(addprefix ) expect space delimited arguments. On VMS commas are used. Not sure how that can be reconciled. features/vpath3: Skipping. Test requires .LIBPATTERNS, which does not appear to be implemented on VMS, yet the variables/LIBPATTERNS tests are passing. features/vpathgpath: fail - 0/1 pass gpaths not followed on VMS. Reason why not obvious. Still working my way triaging the other issues. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 489 tests, 141 tests failing.
On 3/20/2014 7:28 PM, John E. Malmberg wrote: Current status = 463 tests, 391 passed, 56 failing. Additional issues so far: features/se_explicit: 10/10 pass Needed to adjust for VMS syntax. features/se_implicit: 10/11 pass, 1 failure. Unknown issue. features/value: 0/1 pass, 1 failure. Test does not appear to work on VMS. misc/bs-nl: 8/20 pass, 8 failures. The backslash-newline on recipe lines is not working on VMS. options/dash-I 2/3 pass, 1 failure. recurse not working properly. options/dash-e 0/1 pass, 1 failure. No environment override. options/dash-n 3/6 pass, 3 failures. + handling broken. MAKEFLAG appears broken. options/dash-q 5/8 pass, 3 failures. Non-existent symbol fails, 2 bs-nl failures. options/eval2/3 pass, 1 failure. recurse not working. options/symlinks skipped. Symlinks not enabled VMS 8.3+ supports symbolic links. options/DEFAULT 0/1 pass, 1 failure. make invoking make not working right. options/INTERMEDIATE 5/8 pass, 3 failures. rm missing from output when targets cleaned up. Deleted targets on separate lines. options/ONESHELL skipped. Tests all assume a posix shell. options/SECONDARY 8/12 pass, 4 failures. rm missing as above. variables/GNUMAKEFLAGS 2/3 pass, 1 failures. Child failing, missing entering... and leaving dire... messages. variables/MAKE 0/1 pass, 1 failure. Missing entering... and leaving... variables/MAKEFILES 1/2 pass, 1 failure. Not sure if bug | feature yet. variables/MAKEFLAGS 2/3 pass, 1 failure. Looks like child is failing. variables/MAKELEVEL 0/1 pass, 1 failure. Environment variables not exported to children. variables/MAKE_RESTARTS 2/3 pass, 1 failure. Missing entering.. etc. variables/automatic 7/6 pass, 1 failure. Echo text with or D fails. variables/special 1/2 pass, 1 failure. bs-nl issue. variables/undefine 3/4 pass, 1 failure. Undefine of multi-line define fails. The next step is to document at least in the readme.vms file the VMS specific behaviors that were discovered from running these tests. Looks like some problems are affecting multiple tests. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness on VMS, running 463 tests, 56 tests failing.
On 3/24/2014 9:29 PM, John E. Malmberg wrote: On 3/20/2014 7:28 PM, John E. Malmberg wrote: Forgot to update title, missed documenting options/dash-k failure. Current status = 463 tests, 391 passed, 56 failing. Additional issues so far: features/se_explicit: 10/10 pass Needed to adjust for VMS syntax. features/se_implicit: 10/11 pass, 1 failure. Unknown issue. features/value: 0/1 pass, 1 failure. Test does not appear to work on VMS. misc/bs-nl: 8/20 pass, 8 failures. The backslash-newline on recipe lines is not working on VMS. options/dash-I 2/3 pass, 1 failure. recurse not working properly. options/dash-e 0/1 pass, 1 failure. No environment override. options/dash-k2/3 pass, 1 failure. Keeps going when should not. options/dash-n 3/6 pass, 3 failures. + handling broken. MAKEFLAG appears broken. options/dash-q 5/8 pass, 3 failures. Non-existent symbol fails, 2 bs-nl failures. options/eval2/3 pass, 1 failure. recurse not working. options/symlinks skipped. Symlinks not enabled VMS 8.3+ supports symbolic links. options/DEFAULT 0/1 pass, 1 failure. make invoking make not working right. options/INTERMEDIATE 5/8 pass, 3 failures. rm missing from output when targets cleaned up. Deleted targets on separate lines. options/ONESHELL skipped. Tests all assume a posix shell. options/SECONDARY 8/12 pass, 4 failures. rm missing as above. variables/GNUMAKEFLAGS 2/3 pass, 1 failures. Child failing, missing entering... and leaving dire... messages. variables/MAKE 0/1 pass, 1 failure. Missing entering... and leaving... variables/MAKEFILES 1/2 pass, 1 failure. Not sure if bug | feature yet. variables/MAKEFLAGS 2/3 pass, 1 failure. Looks like child is failing. variables/MAKELEVEL 0/1 pass, 1 failure. Environment variables not exported to children. variables/MAKE_RESTARTS 2/3 pass, 1 failure. Missing entering.. etc. variables/automatic 7/6 pass, 1 failure. Echo text with or D fails. variables/special 1/2 pass, 1 failure. bs-nl issue. variables/undefine 3/4 pass, 1 failure. Undefine of multi-line define fails. The next step is to document at least in the readme.vms file the VMS specific behaviors that were discovered from running these tests. Looks like some problems are affecting multiple tests. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms argv[0] and exit status fixes.
On 3/24/2014 3:07 PM, h.becker wrote: There are several problems with the checked-in old exit handling, which aren't resolved by the recently suggested new code. Old problems are: - the %NONAME-?-NOMSG messages, which are generated by VMS when a program returns a Unix style exit code; an exit code 2 shows as %NONAME-E-NOMSG, Message number 0002. - VMS style warnings do not stop make. New problems are: - redirecting exit (and _exit, which seems wrong) to vms_exit, where the passed exit code is adjusted, doesn't really know whether this is a make exit code or one passed from an action routine, which can be of any style - in case of an error there are error numbers printed by make, with the adjusted exit codes, these numbers are no longer obvious. That is because the conversion is not complete, only the parent handling has been fixed, not the child status. Documented Make exit codes are 0, 1, 2 and VMS style exit codes are 0 for success, and VMS style failure codes start at 8 for Access violation. The CRTL decc$exit() and decc$_exit() are already doing a translation of exit code 0 to be SS$_NORMAL. So 0 is passed through to decc$exit, and codes 1 through 7 need to be encoded as POSIX exit codes. And after looking at how MAKE_TROUBLE works, as you had it, it does need to have an ERROR severity level. A warning status should stop the make, but since the target actually got created, re-running make should continue. Since MAKE_TROUBLE is returned when a target is not created, re-running the make should result in a failure again. I am now able to run the complete test suite, and while not everything passes, I can now see the impact of using the POSIX encoding is, and where more adjustments are needed. For programs that never see VMS status codes, the _POSIX_EXIT macro definition fixes almost everything. But we can not use it on GNU make because we need the VMS exit codes and want the VMS severity codes for backwards compatibility. I have not submitted my latest exit handling patch because I am waiting for the GNU copyright assignment. It has been a week now, and I have not seen a response. It seems that having VMS specific codes for MAKE_SUCCESS, MAKE_TROUBLE and MAKE_FAILURE is the way to go. Easy, as long as these codes are always used. But there are hard coded numbers 0, 1 and 2 as well as EXIT_FAILURE, passed to die() and/or exit(). That may introduce other problems if Make or a child is using the hard coded number instead of the symbol. It is safer to have the exit() do the encoding, and that is more transparent to majority of the maintainers to make. The less VMS specific code they have to work around, the better. And the less chance that they may break something VMS specific by accident. So I suggest to always use the MAKE_mumble macros and to define them for VMS as posix compatible exit codes. This avoids redirecting exit to vms_exit; this assumes that the exit status of a child process is VMS style, which makes checking for failure simpler; this allows to decode and print the known make error numbers. If we were compiling /DEFINE=_POSIX_EXIT, then all this happens magically, except then we can not set the VMS severity bits. But it doesn't solve everything. The usual action routines will be VMS utilities which set the VMS style exit codes. Any program which adheres to the VMS style exit codes will work as an action routine as well. That includes all programs which make use of the posix compatible exit codes as well. Action routines, which just return a Unix style number, would need different checks. Although it seems simple to check for zero (when there is no VMS facility code set), it turns out that the child processes, as they are implemented today, never ever return zero. This way it is not possible to find out whether it was successful or not. That is already being handled in the lib$spawn. lib$spawn is returning the VMS status in a VMS specific variable, so the VMS status which includes POSIX encoded status. There is some extra handling needed in the VMS specific code around LIB$SPAWN to decode the POSIX status back to the original UNIX code. That will make the numbers reported by Make look like they should. Once that is done, then the original UNIX constants for MAKE_mumble will match the output. The only case that is problematic to handle is when a child utility ported from Unix using UNIX exit codes is not compiled with /DEFINE=_POSIX_EXIT/MAIN=POSIX_EXIT. In that case, 0 and 1 are both treated as 1, and the rest of the numbers get passed through. I have been through this conversion on the VMS Perl project which has similar backwards compatibility issues. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
How much Unix should be simulated with GNU Make on VMS?
Starting fresh thread instead of continuing the bug 41813 discussion. On 03/20/2014 11:32 PM, John E. Malmberg wrote: Another question is how much of the Unix environment should be simulated when running Gnu make from DCL? In my local copy of the sources, I changed the ECHO variable defined in default.c to builtin_echo, which I implemented in vmsjobs.c as a simple wrapper for write sys$output. That way, whatever worked for VMS/DCL users with $(ECHO) will work, even with multilines. OK there are a few exceptions. If someone knew that there was a missing double quote and coded $(ECHO) hello the old output was hello and the new output will be hello. Also, if someone wanted to use DCL symbols/lexcials and knew that $(ECHO) +f$$time() would get the wanted result, the new code will give a %DCL-W-EXPSYN and the code has to change to $(ECHO) +f$$time()+ to get the wanted result. I don't know of anything of a Unix environment which should be simulated, here. But obviously there is no parity. In Unix you can have a shell variable in the to be echoed text and - if defined - it will show. As I showed in the above example, it's not that intuitive/easy for VMS/DCL symbols. GNU Make on VMS is currently a combination of Unix specific macros, simulated Unix commands and quite a few mostly undocumented VMS specific changes. The GNU Make as is, can not execute the existing Unix makefiles on VMS because it has no knowledge of GNV or equivalent utilities and the default rules are changed. And on VMS, while GNU make supports basic DCL commands, it does not fully support DCL commands, and the VMS specific changes actually interfere with other make features such as functions that expect space delimited arguments in a string. I have not found a way to make DCL symbol substitution work for using lexical functions in recipes. I can build strings with lexical functions like you showed above, but I can not do: pipe xx = hello ; write sys$$output ''xx'. It just writes out ''xs'. So there is no question about the existing port have a mostly VMS flavor of doing things. It has a mostly Unix flavor of doing things. So parts of Unix are being simulated, but as the test failures show, there are still many parts that are not, but probably could easily be simulated. Some of the self tests issue cat commands (not with a macro), and these showed up in the test logs as failing because cat did not understand the VMS convention of using commas to separate the input files. I have not figured out how it was running the Unix cat command, because I made cat a foreign command to run TYPE, and did not have a DCL$PATH set to find the coreutils cat program. I had to modify the test to generate the makefile with a type command to verify what was being tested. Unless there is something going on that I have not discovered, it looks like the cat utility is being internally simulated, at least on VMS. I am currently trying to summarize what I learned from getting the test harness to run as a section in the readme.vms better document the current behavior. Now from the readme.vms, I can see that there are users of this VMS mode. The issue is that because it is mostly undocumented and has quirks, there is a risk that fixing what looks like an obvious bug could break what an existing makefile.vms is depending on. Now if you want to have a mostly VMS look and feel for the Makefile: There is another open source product MMK (Madgoat Make): https://github.com/endlesssoftware/mmk MMK issues a super set of the commercial MMS product original rules and macro set and better supports ODS-5 file names than the commercial MMS product. Fully documented with its own documentation or you can use HP's documentation for MMS. The recipe lines are pure VMS, and DCL is fully supported in what GNU make refers to as ONESHELL mode. Which means that you can use DCL lexical functions, and create multiple line recipes to operate on a target. IMHO: If someone had to port a Unix makefile to VMS, using VMS specific syntax and tools, it is faster to write or convert to a MMK compatible makefile than it is to write a VMS variant of the makefile that runs on the existing GNU MAKE. Most of the Open Source products ported to VMS that I am aware of that use makefiles instead of DCL procedures, use the MMK variant, not GNU make. Now that the coreutils 8.21 and Bash 4.2.47 are available for VMS Alpha and IA64, we can now run the configure scripts and generate makefiles just like on UNIX, as long as we use the older forked port of GNU make. Unless you are building Bash, Coretuils, gnu sed, gawk, binutils, or make so you can not depend on them already existing, on VMS the UNIX format makefile can be both generated and run, which means that there must less reasons for manually maintaining a makefile.vms for any other project. And of the above set, only Gnu Make and binutils use a custom makefile.vms
Test harness patch for VMS.
This is the first cut at the patches for running the test harness on VMS. The test_make.com sets up the environment and runs the test. As it makes persistent changes to the logged in session, I recommend dedicating a logged in session to running tests. The -help option to test_make.com explains the usage for VMS. The updates to the individual test scripts will follow as separate patches. This has some temporary hacks in it to get around some known issues with VMS that affects multiple tests that will need to be removed as the issues are resolved. The test harness on VMS requires: Perl 5.18 or later. PCSI kits available from http://sourceforge.net/projects/vmsperlkit/files/ GNV 2.1.3 or later. GNV 3.0.1 has not tested with this script. Bash 4.2.47 or later. Coreutils 8.21 or later. http://sourceforge.net/projects/gnv/files/ Read before installing: http://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/ Copyright assignments have been signed and submitted to fsf.org. Regards, -John 0001-Test-harness-updates-for-VMS.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Proposed VMS readme fixes.
The existing readme.vms is getting too hard to follow. I propose splitting off the historical data into the changelog.vms and only have information that is relevant to running the current version of GNU make in the readme.vms. Added documentation for the issues discovered by running the test suite. Regards, -John 0001-Proposed-VMS-readme-changes.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Skipped tests on VMS.
These entire tests are currently skipped on VMS, so very simple patch. Regards, -John 0001-Skipped-tests-on-VMS.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
VMS progname and exit handling - Take 4
This has the makefile changes needed to build from master and on VAX/VMS. In addition, vms_progname.c that is compatible as a VMS specific replacement for the progname.c that is used for several GNU projects. This variant of vms_progname.c has an additional feature that it temporarily creates a foreign command that make uses for a macro to represent the command that invoked make. This foreign command has no embedded spaces or dollar sign characters which could cause unexpected results if further string processing is done. The vms_exit() is similar to the /DEFINE=_POSIX_EXIT, except that it allows pass-through of VMS exit statuses of 8 or higher, and allows setting the SEVERITY and message inhibit settings for the encoded Posix status. The message inhibit bits prevents the DCL shell from re-displaying status messages that have already been displayed. Additional work needs to be done to convert POSIX encoded child exit statuses back to the original Unix status internally. This is usually handled by the /DEFINE=_POSIX_EXIT. We are not simply using the /DEFINE=_POSIX_EXIT as it does not support mixing the VMS exit codes and Unix exit codes, along with setting special VMS status bits. Regards, -John 0001-VMS-progname-and-exit-handling-take-4.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VMS progname and exit handling - Take 4
On 3/31/2014 12:48 PM, h.becker wrote: On 03/31/2014 06:46 AM, John E. Malmberg wrote: This has the makefile changes needed to build from master and on VAX/VMS. In addition, vms_progname.c that is compatible as a VMS specific replacement for the progname.c that is used for several GNU projects. This variant of vms_progname.c has an additional feature that it temporarily creates a foreign command that make uses for a macro to represent the command that invoked make. This foreign command has no embedded spaces or dollar sign characters which could cause unexpected results if further string processing is done. The vms_exit() is similar to the /DEFINE=_POSIX_EXIT, except that it allows pass-through of VMS exit statuses of 8 or higher, and allows setting the SEVERITY and message inhibit settings for the encoded Posix status. The message inhibit bits prevents the DCL shell from re-displaying status messages that have already been displayed. Additional work needs to be done to convert POSIX encoded child exit statuses back to the original Unix status internally. This is usually handled by the /DEFINE=_POSIX_EXIT. We are not simply using the /DEFINE=_POSIX_EXIT as it does not support mixing the VMS exit codes and Unix exit codes, along with setting special VMS status bits. It seems I need more than this patch to compile a test version. It seems this needs to be applied on top of a previous patch, or? I was missing the changed main.c and makeint.h. New patch includes them. This patch was based on a branch made directly from master. Again, this looks like a lot of code to set the program name and to handle the exit code. And I'm still not convinced that defining a DCL symbol and deleting it at image exit is the right thing to do. As far as I can see, it can be simpler than that. If you have the make variable be a MCR command, it has a space which can have side effects when if the resulting string is processed by other strings or macros. With ODS-5 format filenames, the sys$filescan system service is the most reliable way detect and parse a VMS filename. This adds to the complexity. The other complexity is working around some bugs in how the C pre-main function renders the Unix format filenames. Regards, -John 0001-VMS-progname-and-exit-handling-take-5.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness patch for VMS.
On 3/31/2014 12:22 PM, h.becker wrote: On 03/31/2014 01:53 AM, John E. Malmberg wrote: This is the first cut at the patches for running the test harness on VMS. The test_make.com sets up the environment and runs the test. As it makes persistent changes to the logged in session, I recommend dedicating a logged in session to running tests. The -help option to test_make.com explains the usage for VMS. The updates to the individual test scripts will follow as separate patches. This has some temporary hacks in it to get around some known issues with VMS that affects multiple tests that will need to be removed as the issues are resolved. The test harness on VMS requires: Perl 5.18 or later. PCSI kits available from http://sourceforge.net/projects/vmsperlkit/files/ GNV 2.1.3 or later. GNV 3.0.1 has not tested with this script. Bash 4.2.47 or later. Coreutils 8.21 or later. http://sourceforge.net/projects/gnv/files/ Read before installing: http://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/ I appreciate your work on the tests with/for/on VMS. This wasn't done before and the tests obviously exhibit problems, features and limitations. (To me they also indicate that there isn't much or heavy usage of GNU make on VMS.) I do not see much active use of it for the VMS mode. There is quite a bit of work being done with the GNV fork, but the users are running into problems with bugs, and rather than trying to fix that older forked port, it seems like the thing to do go back to the master, especially since there seems to be some significant VMS maintenance being done. I tried the test_make.com but it failed. As far as I can see it checks whether the required components are present. And they seem to be available. Maybe I missed something, but it fails with: $ perl run_make_tests.pl -verbose %DCL-W-IVKEYW, unrecognized keyword - check validity and spelling \-C\ %DCL-W-IVKEYW, unrecognized keyword - check validity and spelling \-C\ %DCL-W-IVKEYW, unrecognized keyword - check validity and spelling \-C\ For some reason perl is only able to run foreign commands that are global DCL symbols, not local DCL symbols. In test_make.com this change is needed. $! $ testname = $ make :== $bin:make.exe $! $ i = 0 $param_loop: $ The bin logical name is not visible outside of the test_make.com procedure, so to run the test harness outside of the test harness, the full path is needed. The original intent was only to set local symbols for foreign commands so that the test harness did not leave them behind. Unfortunately Perl does not seem to be passing these symbols to its child processes. If it is desired to have the DCL procedure just set up for running the Perl suite, some logical names that it defines just before it runs perl should also be defined. The error message was being generated because the make global symbol foreign command was missing. I missed removing that definition before re-running the test_make.com -- Running tests for GNU make on VMS GNU Make 4.0.90 -- Also, with all the dependencies on GNV I wonder whether it makes sense to run this in/from DCL at all. With GNV and Bash etc. required, wouldn't it be better (and require less VMS specifc perl code?) to run it from bash? And, because it depends on all of this, I think the test driver should be named with a prefix gnv. Perl changes mode when run under Bash because it detects the bash shell. It becomes more Unix like. That still is a work in progress. The next step is to get it to run child programs under the Bash shell instead of DCL. Once that step is done, it may be possible to run automake on VMS. The test harness patches, which I have not yet posted most of them, only expect the VMS behavior when not run under bash. When the GNV for of GNU make is merged back in, the same test suite will be run with Perl under Bash to verify that behavior. So the tests will end up being run twice. The whole purpose of running the Perl under DCL for this round of tests is to fully document the existing VMS behavior and to make sure that there are no regressions when the GNV code is merged in. But it looks like there are a number of features that should be common to both modes that need to be fixed first. We will probably need to add some VMS specific tests to the suite. I did the same regression tests with the Perl port to add in ODS-5 support, while still allowing the old ODS-2 behavior. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VMS progname and exit handling - Take 4
On 4/1/2014 4:22 PM, h.becker wrote: On 04/01/2014 06:41 AM, John E. Malmberg wrote: If you have the make variable be a MCR command, it has a space which can have side effects when if the resulting string is processed by other strings or macros. You mentioned this already. Can you point me to an example where this fails? Is this in a test, which I couldn't run so far? For the $(MAKE), macro, I do not have an example. In the tests though, several examples show up where the VMS specific change to have some macros return , delimiters instead of delimiters broke the tests and required some workarounds. That is why adding an unexpected space to the value returned by a standard macro can be predicted to have problems. With 20/20 hindsight, a new set of macros should have been created that returned the data with comma delimiters that could have been used with VMS specific rules and recipes. That can not be done now with out breaking backward VMS compatibility. But having a set of functions to easily swap commas for spaces and the reverse would probably still be useful to add, as it would be more convenient than the hack I had to use. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness patch for VMS.
On 4/1/2014 1:14 PM, h.becker wrote: I still can't get it to work: Missed that it also needs the sh as a foreign command to run Bash. Replacement patch attached. Regards, -John 0001-Test-harness-updates-for-VMS-take-2.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VMS progname and exit handling - Take 4
On 4/3/2014 4:15 PM, h.becker wrote: On 04/02/2014 07:15 AM, John E. Malmberg wrote: snip With 20/20 hindsight, a new set of macros should have been created that returned the data with comma delimiters that could have been used with VMS specific rules and recipes. That can not be done now with out breaking backward VMS compatibility. But having a set of functions to easily swap commas for spaces and the reverse would probably still be useful to add, as it would be more convenient than the hack I had to use. As far as I understand you are testing GNV, you are not testing VMS. Currently I am testing VMS behavior. Once the remaining VMS test issues are resolved, then I will start adding the GNV code, and start running tests in both modes. No suprise that there are problems with the special variables, which have comma delimiters ($+, $^, $?). I think I already said this, from my point of view, a GNV GNU make should be separated from a VMS GNU make. I only want one GNU make binary on my system that I can use anywhere I need to use GNU Make and have it just work. Here, in commands.c, you want to have the FILE_LIST_SEPARATOR defined as ' ' and not as ',' as it is in the #if VMS: you need to turn off the VMS macro or add an own GNV macro. Or make it a runtime behavior based on the presence of getenv(SHELL) which GNU make is already checking for the other operating systems and adjusting its behavior. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms: tests features - archives - parallelism
I previously posted the patch for the tests that should currently be skipped on VMS. This is the first part of the changes VMS needs to the tests in the features directory. I am sending the test changes in chunks to keep each commit size down. archives: Most tests are failing. This is what the test should probably look like once the feature is fully implemented on VMS. VMS currently needs to use real object files for the archives. This test currently uses the GNV AR wrapper to the VMS librarian for the recipes. conditionals: On VMS continue is used instead of :. Possible enhancement is to have GNU Make on VMS automatically do this translation. Currently the I have the test harness patched to ignore the error message from passing : as a DCL command. Otherwise even more tests will eventually need to be patched for VMS. double_colon: On VMS, the pipe command is needed for multiple commands on a line. Possible enhancement for VMS is to automatically convert such lines if this can be detected. That enhancement could remove a lot of VMS specific changes to the test suite. Because the ; is the version delimiter for files on VMS, it may not be possible to do this detection. escape: On VMS one of the tests in this script is not valid when not using a Posix shell like GNV Bash. parallelism: Some tests are failing. The feature does not appear to be implemented on VMS. This is close to what the test should look like if the feature can be implemented. I did not skip this test as I am hopeful that the feature can be implemented. Regards, -John 0001-vms-tests-features-archives-to-parallelism.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms: test features - patspecific-vars - vpathplus.
I previously posted the patch for the tests that should currently be skipped on VMS. This is the second part of the changes VMS needs to the tests in the features directory which completes this directory. patspecific_vars: Create a VMS format makefile Skip one test because VMS does not currently export environment variables. Possible enhancement to create DCL symbols for those environment variables. patternrules: Use pipe command multiple commands on a line. Use POSIX encoded exit code, as exit 1 is success on VMS. reinvoke: One test is failing to interpret a macro properly. Convert makefile and result to VMS syntax. se_explicit: VMS makefile needs hack to convert commas back to spaces. se_implicit: One test is failing, claiming no action needed. VMS makefile needs hack to convert commas back to spaces. se_statpat: VMS makefile needs hack to convert commas back to spaces. targetvars: VMS makefile needs VMS syntax. Two tests skipped because they need a Posix shell vpath: VMS makefile needs hack to convert commas back to spaces. vpath only works with VMS syntax files. Enhancement: The rest of Make on VMS works with either VMS or UNIX filename syntax, so VPATH should also. vpathgpath: Test is currently failing, apparently gpath not implemented. Converted test to use VMS syntax for VPATH. vpathplus: vpath only works with VMS syntax files. Use VMS type, pipe command and POSIX encoded exit codes. 0001-vms-tests-features-patternrules-to-vpathplus.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms: tests functions directory patches.
This is changes for the functions directory test scripts. dir: The returned path is VMS format. eval: Skip one test on VMS. file: Change to use VMS pipe command. filter-out: VMS escaping is apparently different. foreach: VMS logical names showing up as environment override instead of environment. origin: VMS logical names showing up as environment override instead of environment. substitution: VMS needs to use a pipe command. word: Change to use VMS specific syntax. Regards, -John 0001-vms-tests-functions.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms: tests misc
This is the patch for the misc tests for VMS. general1: VMS can not handle the line continuation, needs a single line. TODO: there should be a way to make this work on VMS. general3: VMS can not handle the line continuation, skipping 7 tests. general4: CC program on VMS can not do a link. Regards, -John 0001-vms-test-misc.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms: test options patches.
These are the patches that VMS needs for the option tests. dash-k: 1 test is failing as it is not aborting when it should. Need posix encoded exit status, exit 1 is a success on VMS. dash-W: VMS vpaths need to be in VMS format. Fix up for null commands and not having a Posix shell. eval: 1 test is failing because recurse is not working right. VMS needs to use a pipe command. Regards, -John 0001-vms-test-options.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms: test targets - INTERMEDIATE
This is the patch to the targets test for VMS. INTERMEDIATE: Tests are failing because Gnu Make is not echoing the rm command for removing files. Change to use VMS syntax. Regards, -John 0001-vms-test-targets-INTERMEDIATE.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vms:test variables
This is the patches for the variables tests on VMS. This is the last of the tests that I have patched so far. variables: Tests are failing because VMS is not handing pre-defined macros in the names. Fixed up some expected output for VMS. GNUMAKEFLAGS: Test is failing because it is missing the entering/leaving directory messages. Probably related to recursions not working. Use pipe command on VMS. private: Two tests are skipped on VMS. Use pipe command on VMS. Regards, -John 0001-vms-test-variables.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
On 4/4/2014 12:07 AM, Philip Guenther wrote: On Thu, Apr 3, 2014 at 9:56 PM, John E. Malmberg wb8...@qsl.net wrote: This is the patches for the variables tests on VMS. I'm kinda surprised that you haven't worked out some direct coordination with the GNU make developers for the handling of these diffs. I don't see how sending them to the full bug-make list is useful; are you expecting random list members to provide feedback on them? It is what I was requested to do in http://savannah.gnu.org/bugs/?41611 Before I can make significant changes, I need a way both regression test and test new functionality. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
On 4/4/2014 7:05 PM, h.becker wrote: On 04/04/2014 06:56 AM, John E. Malmberg wrote: variables: Late evening transcription error, I meant automatic. After modifying that for VMS (especiall using write sys$output) I do not see any problem with any of the ''s. (I see problems with $(^D), $(+D) and $(?D) - and other incorrect values may be caused by the same underlying problem - but I want to double check the results on a newer system.) I missed that VMS was not handling the quoting the same, and also switched it to only use VMS file syntax and write sys$output. Resubmitted the patch with that change for the first test in automatic. Now it still fails, but the failure is that $(^D), $(^F), $(+D), $(+F) have the wrong values. The $(?D) is showing the correct output in the test. These macros should handle filenames in both VMS and Unix syntax, so once fixed will need tests for both cases. What's your echo command? Are you using that without VMS style quoting of the text? Without such quotes the VMS specific input redirection will kick in and may cause problems. As above, I missed that DCL would not be handling the quoting the same. I tried it with echo :== write sys$output and echo :== $gnv$gnu:[bin]gnv$echo.exe and was getting the same results. Regards, -John 0001-vms-test-variables2.patch.gz Description: application/gzip ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
Ok, so now we have the a baseline test suite ported, what is it going to take to get this stuff committed so that we can start working on the bugs that have been exposed? ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: error reporting
On 4/7/2014 11:26 PM, Philip Guenther wrote: On Mon, Apr 7, 2014 at 4:45 PM, Rob Juergens rjuerg...@esri.com wrote: Given the make file (makefile): -- foo : bar -- gmake 3.75 gives this: gmake: *** No rule to make target 'bar', needed by 'foo'. Stop. gmake 4.0 gives this: gmake: ***. stop. This has caused a lot of time trying to figure out what is wrong with the makefile. A big step backward in error reporting. Please fix this. I am unable to reproduce this: : morgaine; ls -l total 2 -rw-rw-r-- 1 guenther wheel 10 Apr 7 21:22 Makefile : morgaine; cat Makefile foo : bar : morgaine; gmake --version GNU Make 4.0 Built for x86_64-unknown-openbsd5.5 Copyright (C) 1988-2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. : morgaine; gmake gmake: *** No rule to make target 'bar', needed by 'foo'. Stop. : morgaine; What information about your situation did you leave out? I was able to reproduce a similar issue on VAX/VMS 7.3 with Gnu Make 4.0 from the release tarball. Under some code paths the output buffer size is initialized to zero length and output gets truncated. The buffer expansion algorithm was to double the size of the buffer if it detects that there is not room for the output of vsnprintf, but since the size starts at zero in some cases, zero * 2 is still zero, so the buffer was not expanded. Which results in the output reported above. I did not see this issue on Alpha or IA64/VMS 8.4, and did not find out why it works on that platform and failed on VAX/VMS, since I would have expected similar behavior in both cases. The section of code that causes the problem has been completely replaced in master. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
On 4/7/2014 4:37 PM, h.becker wrote: On 04/06/2014 07:34 PM, John E. Malmberg wrote: Ok, so now we have the a baseline test suite ported, what is it going to take to get this stuff committed so that we can start working on the bugs that have been exposed? I don't know who will/should answer such a question. I still have no luck nor fun with these tests: $ @TEST_MAKE -- Running tests for GNU make on VMS GNU Make 4.0.90 -- Clearing work... Finding tests... features.dir/archives ... ^^ This is a day one bug in the older Perl on VMS versions that was fixed in about the Perl 5.10 timeframe, but the fix was not made the default behavior until Perl 5.18, as it is tied to supporting EFS character filenames. Perl 5.18 and later suppresses the incorrect .dir being appended to paths by default. The incorrect behavior can be optionally turned on for existing PERL/VMS scripts that expect the buggy behavior. I did not test the script with older versions of Perl, or with the older behavior enabled. FEATURES/ARCHIVES ... Error running make (expected 0; got 512): make -f /lcl_root/make/tests/work/FEATURES/ARCHIVES.mk skip FAILED (2/10 passed) 8 features/archive tests are failing for me because the archive support is incomplete. features.dir/comments ... FEATURES/COMMENTS ok (1 passed) features.dir/conditionals ... FEATURES/CONDITIONALS ok (4 passed) features.dir/default_names .. FEATURES/DEFAULT_NAMES ... ok (2 passed) OK, I can run one when I specify one, but it fails due to bash's :: It is not failing, the warning from DCL is written out, but does not otherwise affect the execution. Rather than modify all the tests that were using : as null command, I changed the compare_output routine in the test harness to ignore it. $ make -f [.tests.work.variables]undefine.mk recursive simple recursive undefined recursive undefined $ @TEST_MAKE -keep variables/undefine VARIABLES/UNDEFINE .. run_command_with_output(/lcl_root/make/tests/work/VARIABLES/UNDEFINE.log,): make -f /lcl_root/make/tests/work/VARIABLES/UNDEFINE.mk run_command_with_output returned 0. vms status = 1 Comparing Output DIFFERENT OUTPUT passing tests removed FAILED (3/4 passed) So it seems the tests still contain Unix/shell commands and not only VMS commands. I only made changes needed to test the feature. GNU Make does not care if a command is a foreign command or a native VMS command. $ make -f [.tests.work.variables]undefine.mk recursive simple recursive undefined recursive undefined *** /lcl_root/make/tests/work/VARIABLES/UNDEFINE.base Wed Feb 4 23:01:01 2015 --- /lcl_root/make/tests/work/VARIABLES/UNDEFINE.logWed Feb 4 23:01:01 2015 *** *** 1,2 recursive simple recursive ! undefined undefined undefined --- 1,5 recursive simple recursive ! ! undefined undefined recursive ! ! %DCL-W-NOCOMD, no command on line - reenter with alphabetic first character Or am I missing something? The extra lines and the %DCL-W-NOCOMD lines are ignored by the test harness output compare. If that compare fails, then the test harness issues a diff command on the raw output, not the cooked output used for grading the test, so every thing is present. The actual failure appear to be that the second line should be undefined undefined undefined and on VMS is showing up as undefined undefined recursive. I have not fully investigated this issue, but it appears that this happens when make recurses or invokes itself, and on VMS, the second invocation of make is missing some information from the parent. My guess is that information is normally passed by exported shell symbols. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
On 4/8/2014 12:48 PM, h.becker wrote: On 04/08/2014 07:28 AM, John E. Malmberg wrote: On 4/7/2014 4:37 PM, h.becker wrote: $ @TEST_MAKE features.dir/archives ... ^^ This is a day one bug in the older Perl on VMS versions that was fixed in about the Perl 5.10 timeframe, but the fix was not made the default behavior until Perl 5.18, as it is tied to supporting EFS character filenames. test_make did not complain about the perl version so I concluded perl is/was OK. It seems it isn't: $ perl -V Summary of my perl5 (revision 5 version 10 subversion 0) configuration: ... The test is mainly testing for the older Perl 5.8.x from the HP kit to avoid using it. Craig Berry did a manual custom install of 5.10 on Encompasserve that looks a lot like the new PCSI kit install for 5.18, so the test_make.com procedure did not detect that it was an older Perl. I will find out if Craig Berry has time to update Perl on Encompasserve, or if he is ok if I just do it. Perl 5.18 and later suppresses the incorrect .dir being appended to paths by default. The incorrect behavior can be optionally turned on for existing PERL/VMS scripts that expect the buggy behavior. I did not test the script with older versions of Perl, or with the older behavior enabled. and can the incorrect behavior turned off in older versions? (OK, temporarily I can change run_make_tests.pl). $DEFINE DECC$EFS_CHARSET ENABLE Should do the trick. That turns off most of the backwards behavior. As long as there are not any ODS-5 specific files in the directory, the older behavior mostly works. Some other DECC$ logicals may also be needed for fixing various bugs in the UNIX format file name parsing, as the CRTL is by default maintaining backwards compatibility for programs that are expecting buggy behavior. Most of the fixes should be in Perl 5.10, just not enabled by default. OK, I can run one when I specify one, but it fails due to bash's :: It is not failing, the warning from DCL is written out, but does not otherwise affect the execution. Rather than modify all the tests that were using : as null command, I changed the compare_output routine in the test harness to ignore it. Make, at least the version I made, propagates the DCL warning and stops with an error: $ mcr USR_ODS5:[BECKER_H.MAKE.MAKE]MAKE -f work/variables/undefine.mk_2 undefined %DCL-W-NOCOMD, no command on line - reenter with alphabetic first character work/variables/undefine.mk_2:11: recipe for target 'all' failed make: *** [all] Error 0x380b0 %DCL-W-NOCOMD, no command on line - reenter with alphabetic first character $ ed ^UP^work/variables/undefine.mk_2 1 *s/@:/@continue/W 11 all: ;@continue 1 substitution *exit USR_ODS5:[BECKER_H.make.x.tests]UNDEFINE.MK_2;1 11 lines $ rena UNDEFINE.MK_2;1 ^UP^work/variables/undefine.mk_2 $ mcr USR_ODS5:[BECKER_H.MAKE.MAKE]MAKE -f work/variables/undefine.mk_2 undefined $ GNU make doesn't care about the commands, but either my make (likely, or maybe) behaves, or the used commands (I don't see any in this test case) behave, or the test environment behaves differently: $ mcr USR_ODS5:[BECKER_H.MAKE.MAKE]MAKE -f work/variables/undefine.mk recursive simple recursive undefined undefined undefined %DCL-W-NOCOMD, no command on line - reenter with alphabetic first character work/variables/undefine.mk:19: recipe for target 'all' failed make: *** [all] Error 0x380b0 %DCL-W-NOCOMD, no command on line - reenter with alphabetic first character $ and with the replacement for :: $ mcr USR_ODS5:[BECKER_H.MAKE.MAKE]MAKE -f work/variables/undefine.mk recursive simple recursive undefined undefined undefined $ but in the log/diff created under control of the perl script there is $ ty ^UP^work/variables/undefine.log recursive simple recursive undefined undefined recursive $ ty ^UP^work/variables/undefine.diff *** work/variables/undefine.baseTue Apr 8 13:22:21 2014 --- work/variables/undefine.log Tue Apr 8 13:22:21 2014 *** *** 1,2 recursive simple recursive ! undefined undefined undefined --- 1,4 recursive simple recursive ! ! undefined undefined recursive ! $ which matches your output. This is at least confusing, one would expect to see the same output in a stand-alone run and in a run under the control of the perl script. That is something that needs to be investigated. Something about running in the test script is changing the behavior of GNU make for a few tests. I saw only about 2 cases where a test passed when I manually ran the makefile and it failed on the test harness. I am sure that the test harness and tests will need further changes as we progress, and that we will need to add tests to explicitly test for intentional VMS differences in behavior. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
On 4/11/2014 5:18 PM, h.becker wrote: I hacked a fix for the automatic variables. To pass the test, the vms version of the test script needs a fix as well: --- ../origtests/automatic Fri Apr 11 18:11:32 2014 +++ tests/scripts/variables/automatic Fri Apr 11 18:10:19 2014 @@ -73,9 +73,10 @@ # Fix vms directory syntax $answer =~ s#\]/#\]#g; # vms returns current directory as [] - $answer =~ s#\s\.,# \[\],#; + $answer =~ s#\s\.,# \[\],#g; # vms returns current directory as [] with a comma separator $answer =~ s#\s\.\s# \[\],#g; + $answer =~ s#\]\s\[#\],\[#g; # vms has comma separator instead of space. $answer =~ s/baz\.z\s\[/baz\.z,\[/g; $answer =~ s/baz\.z\sb/baz\.z,b/g; I see that. The test is still failing for me: Expected output fragment: $^ = baz.z,[-.tests]bar.y, $(^D) = [],[-.tests], $(^F) = baz.z,bar.y $+ = baz.z,[-.tests]bar.y,baz.z, $(+D) = [],[-.tests],[], $(+F) = baz.z,bar.y,baz.z Actual output: ! $^ = baz.z,[-.tests]bar.y, $(^D) = baz.z,[-.tests], $(^F) = bar.y ! $+ = baz.z,[-.tests]bar.y,baz.z, $(+D) = baz.z,[-.tests], $(+F) = bar.y,baz.z The first line is missing bar.z. In the second line expected ,[] is missing, and so is the second baz.z. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
default.c - Need help in crafting fixed archive rules for VMS
I was able to get one type of archive rules working for VMS, but have
not been able to figure how to translate code from the makefiles into
the internal default pattern rules for a more general case.
The rule below works for object libraries suffixes .a and .olb only.
For VMS I have to use slightly different rules for different library
types, something like:
.a(%) : %
tabif f$$search(@$) .eqs. then $(LIBRARY)/CREA/OBJ
tab($LIBRARY)$(LIBFLAGS) $@ $
.hlb(%) : %
tabif f$$search(@$) .eqs. then $(LIBRARY)/CREA/TEXT
tab($LIBRARY)$(LIBFLAGS) $@ $
.mlb(%) : %
tabif f$$search(@$) .eqs. then $(LIBRARY)/CREA/MACR
tab($LIBRARY)$(LIBFLAGS) $@ $
If I try to put the library suffix in a compiled rule, the rule is ignored.
Below is what I could get to work:
(Slightly modified to avoid text wrapping)
@@ -53,9 +55,15 @@
static struct pspec default_pattern_rules[] =
{
+#ifdef VMS
{ (%), %,
-$(AR) $(ARFLAGS) $@ $ },
+if f$$search(\$@\) .eqs. \\ then $(LIBRARY)/CRE/OBJ $@\n \
+$(LIBRARY) $(ARFLAGS) $@ $ },
+#else
+{ (%), %,
+$(AR) $(ARFLAGS) $@ $ },
+#endif
/* The X.out rules are only in BSD's default set because
BSD Make has no null-suffix rules, so 'foo.out' and
'foo' are the same thing. */
@@ -78,6 +86,7 @@
So I do not understand how to get something like the above rules added
to default.c.
Regards,
-John
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: vms:test variables
On 4/14/2014 3:04 PM, h.becker wrote: As already mentioned, the last test for variables/GNUMAKEFLAGS fails for VMS. I think I know why, but I'm not yet sure whether I have the full picture. There are at least two reasons: the port to VMS of the makefile for this test case is not working as expected and exporting make variables is not implemented on VMS. In the test case, in the original mk_string there is x.mk: ; @echo $@; echo MAKEFLAGS = $$MAKEFLAGS; echo GNUMAKEFLAGS = $$GNUMAKEFLAGS; echo $@ which was ported to VMS as x.mk: ; @pipe echo $@ ; echo MAKEFLAGS = $$MAKEFLAGS ; echo GNUMAKEFLAGS = $$GNUMAKEFLAGS ; echo $@ On Unix, all echo commands except the last one write to stdout: only a single '\n' goes into $@. On VMS, the VMS specific redirection in GNU make forces the output of all echo commands into $@. The fix there is on VMS to replace echo $@ with: 'open/write xxx $@ ; write xxx ; close xxx' On Unix, the shell variables $MAKEFLAGS and $GNUMAKEFLAGS are printed, on VMS $mumble does not reference a DCL symbol (nor are these variables known to the used GNV echo command, as here is no shell used at all). That needs a different fix of course. The GNU make variables MAKEFLAGS and GNUMAKEFLAGS are usually exported (where GNUMAKEFLAGS is set to an empty string by GNU make). Exporting of GNU make variables is not implemented for VMS. But GNUMAKEFLAGS is somehow known and visible when the action routine runs, still holding its initial value. When running the test under the control of the perl test script, a shell variable for a run of GNU make is exported via %ENV. On VMS, added/changed %ENV items show up in the process logical name table. This table is visible when GNU make runs and when the actions run. There may be good reasons why perl for VMS choose to use logical names for implementing %ENV. Usually an environment variable is retrieved with getenv(), which, as implemented in the VMS CRTL, accepts any local, or global DCL symbol and any logical name as an environment variable. And that sequence also describes the order: local symbols are preferred over global ones and over logical names. Perl choosing logical names as the default for implementing %ENV has turned out to be a major bug. It is causing a lot of problems with porting programs and with the Perl self tests and also the perl makemaker, along with issues with the make test harness. Perl has a mode where it can use DCL symbols, but it it not the default or well tested. At this time, the default mode of Perl on VMS can not be easily changed because of the number of programs that expect this behavior. logical names act more like symbolic links than they do environment variables. Even having the CRTL treat logical names as environment variables has turned out to cause problems. If the recursive make is exec(), then there is a variant that allows an environment array to be passed. This is the way that Bash passes exported variables to children. That means, if someone uses a local DCL symbol to set GNUMAKEFLAGS, GNU make will see it, use it and will ignore the logical name created by the perl test script. To have something working and to pass this test, one needs to - export exportable make variables, that is create, or save and overwrite local DCL symbols. - delete the local DCL symbols which were created and restore those which were overwritten - at program termination. Yes a general export routine that saves the old value/state of a local DCL symbol on something like a linked list, creates a new local variable, and an atexit() routine that restores everything off of the list. - change the port of the makefile regarding redirection and what's echoed. That part is done above. But we also need to get symbol substitution working in the spawned DCL command. Either by translating the makefile to use one of the VMS substitutions or by having the wrapper to lib$spawn do a shell emulation for such variables. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 4/27/2014 10:04 AM, h.becker wrote: On 04/26/2014 12:42 AM, John Malmberg wrote: I have attached a preliminary patch to the default.c module that fixes the library rules as follows: 1. Create the library if it does not exist when inserting a module into the library. Wouldn't it make sense to suppress the check and creation of the libraries? That is, replace if f$$search(\$@\) .eqs. ... with @if f$$search(\$@\) .eqs. ... A quick look shows only a few cases were a default rule output is suppressed. In this case, the output is messy enough that suppressing it is probably warranted. 2. Support all the types of library that VMS does. Tests still need to be written for that. Adding other VMS library types is OK with me. There is one more: shareable image libarary. Easily added. 3. Fixes the rcs rules so they have a chance of working. Of course since no-one noticed that they did not work due to a syntax error, it might be time to just remove them. I agree to remove them. 4. Add a bunch of missing rules. VMS should have almost all the same rules as Unix plus some more for VMS specific file extensions and commands. I don't think that VMS should include all the Posix/Unix extensions. The Posix subsystem for VMS (had it's own Posix compliant make and) is gone and GNV is not VMS. I know, I said this before, I think for Gnu make, GNV should be separated from VMS. GNU make is needed so that programs ported from Posix sources and posix like makefiles to VMS can be more easily ported. VMS will use the same file extensions as Posix instead of the defaults, so there is no reason not to support those rules. And this reduces the amount of VMS specific edits needed for a makefile. It also makes it easier to run the existing self tests to validate the port. Adding rules to support VMS file types makes sense as an extension, but they should not be instead of the existing expected posix rules. The recipe for the rules may vary based on GNV or DCL, but all the same rules should be present. If you are going to force a port to always use VMS specific file extensions for builtin-rules, the person doing the port is probably better off just writing a new makefile to use the MadGoat Make program. Another question: how should these new pattern rules for non-object libraries be used? For objects one can use all: o.olb(o) This syntax will work. and all: o.olb(o.obj) This syntax will not work. It matches the older archive rule. For the VMS specific library file types, the file extensions can not be in the parenthesis. Based on the way that Make is interpreting the rules, the extension of the file that is to be added to the library is what determines what type of library will be created. all: h.hlp(h) doesn't work if there are h.h and h.hlp in the same directory. Help libraries are .hlb, not .hlp. all: h.hlb(h) Are all: o(o) all: o(o.obj) all: h(h) all: h(h.hlp) expected to work? No. If the name in the parenthesis has a type, the existing default archive rule takes over, and it will be processed as an object library. It would take a change in how makes handles archive syntax parsing to allow it. The name in the parenthesis is the module name, and on VMS, the module name can be different than the filename and does not have a type. The second rule worked with the old VMS make. With the new code, always a new library is created. It may break existing makefiles, so it needs to be documented. And a comment: on OpenVMS/I64 CXXLINK is not required and not recommended. The rule for CXXLINK on IA64 can be mapped to LINK then. And a side note: in the features_archives.gdiff there is a trailing dot in the filename which prohibits patching the file without manual specifying the correct filename. Habit from using VMS syntax with no extensions. I will try to remember that for next time. Next step is to find out what is wrong with wildcard and vpath with libraries. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 4/28/2014 1:51 PM, h.becker wrote: On 04/27/2014 11:50 PM, John E. Malmberg wrote: Help libraries are .hlb, not .hlp. all: h.hlb(h) My bad, sorry for the misleading typo, but I can't get that to work You need a bit more, as the rule is based on the extension of the source file, not the library file. EAGLE type test_help.mk file1.hlp : tabpipe open/write xxx file1.hlp ; write xxx test help ; close xxx file1 : file1.hlp help : help.hlb(file1) But it is still not working. It is creating a text library instead of a help library. I thought that I had it working for all the cases EAGLE make --debug=b -f test_help.mk help GNU Make 4.0.90 Built for VMS Copyright (C) 1988-2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Reading makefiles... Updating goal targets File 'help' does not exist. File 'help.hlb(file1)' does not exist. Must remake target 'help.hlb(file1)'. library help.hlb file1.txt Successfully remade target file 'help.hlb(file1)'. Must remake target 'help'. Successfully remade target file 'help'. EAGLE lib/list help.hlb Directory of TEXT library LCL_ROOT:[make.tests]help.hlb;1 on 28-APR-2014 18:51:3 Looks like I need to try something else, like a number of unechoed commands that test the target library extension and creating the library based on it. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 5/2/2014 5:22 AM, h.becker wrote: On 05/02/2014 05:44 AM, John Malmberg wrote: Follow-up Comment #3, bug #41758 (project make): Updated default.c patch with fixed shared image library rule. That's not a complete fix. It still has the wrong target. The suffix of such a library is .olb, not .exe. The suffix is identical to the suffix of an object module library. With this implementation it seems to be enough to also check for a suffix .exe in $. The check for an image library is done before the check for an object module library, so in this implementation there is no need to change the latter check for an additional .obj. Sounds reasonable. I have not used image libraries in user mode code so was not aware that they were also .olb. That raises questions about the suggestion to merge/mix VMS and Posix/Unix suffixes: can and should a .o be added to a .olb and a .obj to a .a? Can and should a .so be added to an image library? OK, .so is not (yet?) in $(SUFFIXES). .o and .obj should be handled the same. .so should be added to an image library and added to $(SUFFIXES) With this implementation a couple of processes are created just to do the suffix checks and, depending on these checks, possibly do the creation of the corresponding library. It seems that the checks can (and in my opinion should) be handled within make. I have some ideas how to implement it, but didn't try to, so far. I would like to see only one additional action routine (process) for all kind of libraries with a check whether the corresponding file exists and create it, if it doesn't exist. Current plan is to prototype using DCL and get tests for all VMS specific features that are not covered by other tests. Then after the tests are working correctly, to use built-ins where appropriate to speed things up. A one-shell implementation will also help. Currently I have found 3 bugs in the VMS specific ar_scan code from code inspection. One of them may explain the vpath issue, but I have not found the cause of the wildcards not working yet. It should also be possible to get the touch feature working for VMS libraries. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 5/3/2014 4:30 AM, h.becker wrote: On 05/02/2014 03:17 PM, John E. Malmberg wrote: Sounds reasonable. I have not used image libraries in user mode code so was not aware that they were also .olb. Hmm, the VMS linker uses imagelib.olb and you probably linked against shareable images from that library. So you may talk about something else. To me it seems convenient to use image libraries. I meant explicit use, not the implicit use where the linker handles things automatically. Anyway, appended is a patch to the suggested patch for default.c. It implements checking for the VMS libraries within make, using make functions. Some notes: To me it looks OK to use this complex make function calls. Maybe it can be simplified. The other option is to write a new VMS specific function, like $(vms_create_library LIBRARY,MEMBER), which expands to the if f$search That seems like a valid optimization. Because process creation on VMS is more expensive than on Unix, anytime a process creation can be avoided can be a significant optimization. I'm not sure I understand why the previous code has the 'f$edit(...,LOWERCASE)'. My change doesn't really LOWERCASE the suffix, but it works on all lower- and uppercase suffixes. The lowercase was needed to do the compare. I have switched to your functions and they are passing the test case after revising the image library to be an .OLB. My local [.vms]library test script is now testing image libraries in addition to the other types as I have incorporated your examples into it. I have the wildcard library processing coded and now all 10 [.features]archive tests are usually passing on VMS, including the one that uses a VPATH. A few of them are intermittently failing, so I am not posting the patch yet until I isolate it and come up with a fix. The test script is passing about 3 times in a row, and then fails either because an object module is replaced when it should not be, or is not replaced when it should be. It appears that there is a problem somewhere in comparing of the library insertion timestamp and the VMS object module timestamp. Most of the time it works, but occasionally it fails. I have instrumented the test makefile to log the timestamps that VMS sees, since that was trivial, and those timestamps are all as expected. The next step is to have a test version of arscan.c start printing out the timestamps before and after the VMS local time to UNIX UTC conversion and build it as a standalone executable. Unfortunately none of the built in make debug options will display the timestamps of the files as it considers the rules. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 5/31/2014 7:21 AM, h.becker wrote: On 05/15/2014 05:51 AM, John Malmberg wrote: Follow-up Comment #4, bug #41758 (project make): I admit I never looked at all the archive/library code before. So I may need some education. The patch works for my simple test but that worked with the unpatched version as well. So what's really different? Or which test cases are addressed? This fixes 8 failing tests in the features/archives test. Wildcard access now works correctly. In addition the vpath3 test that was failing before now works correctly. It needs to be modified to expect the result in VMS syntax. However, the patch doesn't work if I dare to specify a module name different from the file name. That worked with the unpatched version. I need to look at that to see how it could have worked, since GNU Make is what is assuming that the module name is the same as the filename, and I do not see a syntax to tell it otherwise. On VAX/VMS 7.3, the version I run in simh on my laptop (when I have no access to the net) I get: %SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=, PC=, PSL=03C0 That has been traced down to a bug in lbr$_set_module on VAX/VMS 7.3. It was ignoring the size set in the input buffer and always writing 49 bytes. In addition both the original and my modified code was incorrectly passing the address of the buffer descriptor length as the variable to get the actual length of the header. I did not experiment to see if the stack corruption went away when I fixed that. The mhddef structure, which has the mhd$l_dattim member is sufficient. If there is more data, the lbr$_set_module is supposed to truncate the data to the size given and return a status of LBR-W-HDRTRUNC which is 2525184. I can not find that value in any header file. That is what was corrupting the stack on VAX. Part of the fix is to allocate LBR$C_MAXHDRSIZ for the mhd structure and then free it after it is used. Diagnosing this also exposed a bug in mhddef.h. The definition for mhd$l_datim is wrong. It has it as a 32 bit longword instead of a 64 bit structure or array. If the mhddef.h was correct, then the decc$fix_time() would not have returned anything resembling a sane time. Also if you run the test program with no input, a null pointer get referenced, so that also needs a check. Not enough time to produce a diff tonight, but thought you would like to know why it was failing. Anyway, some comments ... Having descriptive names for some variables and using const is good. But when introducing different mechanisms/algorithms such changes make it more difficult to see what's really new. Maybe having two patches would help, one for the code improvements and one for the real changes. Part of the problem was that some of the old names and comments were completely wrong, which would make comparing the new code to the old code difficult at best. There are some new conditionals for VMS in the source code. I understand that they are required to have VMS specific code. But it would make the source more readable if such code could be moved into VMS specific sources or sections within the source file. I didn't try to do that nor do I have a real plan, it is just an idea to make this code easier to understand. Unfortunately it was about the only way to do things the way the code is structured. Only the caller and the callback routine know what the structure of the argument is, and that structure needs some VMS specific data in it at those points. I can look too see if macros can be added to hide the #ifdef VMS code to see if that could make it easier to read. It seems that the existing and new code for VMS is complex, at least callbacks are not straight forward and deserve some good comments. Rather than commenting on the lines, like On VMS, there is probably a saved suffix... I suggest to have one big comment to explain the whole picture, examples included. Then, someone looking at the VMS specific code doesn't need to look at all the comments in all the conditional code to understand the overall picture. Also, someone who isn't interested in VMS code will have shorter source code sections to skip (OK, I should have known that everybody is using Eclipse and has folding for preprocessor branches enabled :-) I will look at that tomorrow. It is late here. It may be that a VMS programmers notes section of the readme is needed. I suggest to remove the (old) comments on decc$fix_time, it looks like decc$fix_time is always needed (and not only on VAX and Alpha). And the old comments are in git anyway. I will look at simplifying it. For the tests, the perl scripts, I suggest to change $vms, which is explained as '# $vms is for VMS with out a GNV Bash shell'. Then this is very likely DCL mode, or? So it should be named either DCL[_mode], or for GNV and bash there should be a flag named GNV[_mode|_bash]. I prefer the latter just
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 5/31/2014 7:45 AM, h.becker wrote: On 05/15/2014 05:51 AM, John Malmberg wrote: Follow-up Comment #4, bug #41758 (project make): One more comment on (obecjt) module names in a VMS (obejct) library. At least for I64 there can be lowercase module names in an object library. I didn't check on Alpha. On VAX I couldn't convince the C compiler to generate a lowercase module name. That is, I don't know and didn't try to find out, whether this is a feature of Librarian for I64 or a feature of the language processors to write a lowercase module name into the object module. While I have not checked on VAX, on both Alpha and Itanium the module name can have both upper and lower case in it. This will mainly show up when the source module is on an ODS-5 volume. Right now, ODS-5 support is broken because the vmsify() routines are not only assuming ODS-2, they are also not converting Unix format file specifications correctly when the first path component is a logical name. I am assuming that the unixify routines are in just as bad state and have a bug open on them. After the other issues are fixed, I plan to write a test for proper handling of ODS-2 and ODS-5 pathnames, and use it to validate fixed vmsify and unixify routines. GNU Make can detect when the DECC feature to use ODS-5 names is active, and it can also detect if the process has been set to case sensitive mode. Until then, I am not going to worry about ODS-5 issues, except to mark them in the code. Also, currently neither MMS or MMK properly handle the case of library module names. A fix is in the queue for MMK and a bug report has been filed with MMS. No eta on either, but this causes all modules to be reinserted into the library each time. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #42447] Fix exporting symbols for recursive make on VMS.
On 6/1/2014 3:22 PM, h.becker wrote: On 05/28/2014 07:00 AM, John Malmberg wrote: This patch fixes Make on VMS to export symbols so that recursive and nested make commands will work. This also exports a foreign command for the make image that was run. Some of the tests still fail because VMS currently does not do shell expansion of exported symbols. I will be looking at adding that capability in the future as part of improving posix shell simulation. Sorry, I don't understand what is menat with VMS currently does not do shell expansion of exported symbols. Any example? I can not put ''foobar' in a recipe where foobar is an exported symbol. While DCL is not technically a shell, in the context of make it is easiest to consider it a shell. On the other hand, wouldn't it be enough to ALWAYS generate a VMS command procedure for an action and insert DCL symbol definitions at the beginning of that file? I am more inclined to try to avoid generating the VMS command procedure where ever possible at this time. For one thing, it does not handle the case for built-ins. And it may be desired to create more built-ins such as handling library creation more efficiently. And the pipe command may allow implementing I/O redirection with out a command procedure if the lines are short enough. The current code appears to be always using the older limitation on the command size when determining when to switch to command files, instead of basing it on the version of VMS used for the compile. For exported symbol substitution, I would prefer it to be done when the command to be sent to the child process is created. At the present time with this patch, the archive patch, and your multi-line variable hack: 454 total tests. 405 Passing. 16 skipped. 43 failing. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #42447] Fix exporting symbols for recursive make on VMS.
On 6/6/2014 10:19 AM, h.becker wrote: On 06/06/2014 05:44 AM, John E. Malmberg wrote: On 6/1/2014 3:22 PM, h.becker wrote: On 05/28/2014 07:00 AM, John Malmberg wrote: And the pipe command may allow implementing I/O redirection with out a command procedure if the lines are short enough. As I documented somewhere, I/O redirection is not really something make has to do. But it was already there, when I looked at the sources. Whether VMS pipe commands are the right solution for I/O redirection is another question: I don't see that the pipe command supports appending output. And I guess that pipe commands tend to be longer than simple commands. The make self tests are expecting recipes to redirect output, so it is useful to have for VMS. And someone needed in a VMS specific makefile to add it. I think there is a way to use pipe commands to do appending of output. I may have even used it in the patches for the make tests. By moving the code into the vmsjobs.c or equivalent, it can remove VMS specific changes from the self tests. I have to do some tests. Right now, my focus is on getting all the tests to pass. The current code appears to be always using the older limitation on the command size when determining when to switch to command files, instead of basing it on the version of VMS used for the compile. for the compile? Maybe I have a language problem, but I think it depends on the VMS version/environment where Gnu make runs. A program built on a version of VMS that does not support the longer command lines would not be able to see the compile time symbols that the support was present on, so while it could build the support in, the code would look different. A program built on a version of VMS that supports the longer command lines would not be able to run on the older versions of VMS, unless special (and mostly undocumented) build procedures are used. And then you need the older version of VMS to validate that the build works. Currently I am limited to VAX/VMS 8.3, Alpha 8.3/8.4, and Itanium 8.4. So that is why I consider it a compile time option. For exported symbol substitution, I would prefer it to be done when the command to be sent to the child process is created. Does that mean re-implementing in make what DCL can do? As with I/O redirection, I don't think this is something make should do. Also, I'm not sure if that's what a user expects. For example, f$time(0) may be slightly off and f$getjpi(0,pid) is simply wrong. Lexical functions work in recipe commands now. Symbol substitution does not. I am just looking at simulating $xxx expansion at this point, done before the command is run. GNU make is already supporting other bits of shell simulation on VMS, so the precedent of doing this type of thing has already been set. I also plan to look at having a pool of subprocesses and sending commands to them over mailboxes instead of spawning a new process for each line of a recipe. This should speed make, particularly on the older systems. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 6/10/2014 3:24 PM, h.becker wrote: On 06/06/2014 05:15 AM, John Malmberg wrote: lbr$set_module, the documentation says: snip So the size of struct mhddef may not be enough and returning more than that is not an error. Allocating LBR$C_MAXHDRSIZ bytes for the buffer is probably the best one can do, because the contents of the fields mhd$b_usrdat and cre$l_uhd is not known, here. This issue is only present on older versions of VMS, not the current version, so someone else also considered it a bug and fixed it. The library routine is given a descriptor that sets the bounds that it can write to, and there is a return status code to indicate when there is more data to be read, so that the call can be repeated with a larger buffer. and Length of the module header. The buflen argument is the address of a longword receiving the length of the returned module header. Will fix in the next patch submission. That bug was present in the original code also. This is why a prefer to hand construct the prototypes for the library routines and system services. I can set them up to force a compile time error in these cases. We are saved from data corruption because the compiler does not tightly pack variables so 32 bits was available for the write. But this should get fixed. So in arscan.c unsigned short buffer_length; /* Actual buffer length */ should be changed to a type of unsigned int. Regarding comments, I would like to have some comments like /* On VMS, (object) modules in libraries do not have suffixes. That is, to find a match for a pattern, the pattern must not have any suffix. So the suffix from the pattern is saved and the pattern is stripped (ar_glob). If there is a match and the match, which is a module name, is added to the chain, the saved suffix is added back to construct a source filename (ar_glob_match). */ maybe in ar.c just prior to struct ar_glob_state. I admit, I don't know and didn't try to understand the code, so the above constructed comment may not be correct. (And it seems that this comment style is almost always used for block comments). It is mostly correct and the next patch will have a similar comment. I do not think it only applies to object modules though. In arscan.c, I suggest to use LBR$_HDRTRUNC instead of LIB_W_HDRTRUNC. A globalvalue unsigned int LBR$_HDRTRUNC; replaces the #define LIB_W_HDRTRUNC 2525184 and the check changes to if ((status != LBR$_HDRTRUNC) ... Globalvalue is the right storage class for this declaration as per definition LBR$_HDRTRUNC can change any time. With resolving it at link time and doing a fixup at activation time the code will get the correct value even if it is changed. I agree, I don't expect it to be changed, but having own #defines for VMS defined values looks wrong to me. And as far as I can see there are no LBR$_code return codes defined in any of the LBR header files and it seems unlikely that this will be changed (sooner or later or at all) by VMS engineering. VMS convention has always been to use these as compile time constants which means that once they are set, they are never changed for backwards compatibility. globalvalue is only visible if VAXC compatibility is active, and turning on stricter ANSI compliance in the compiler turns it off. Turning on the stricter ANSI compliance finds bugs that VAX C compatibility ignores, so that is something that is desired. While there is a ANSI compatible VMS extension to allow the equivalent of a globalvalue, it is a bit ugly to use. And the compiler best optimizes as a constant, not as a global value that is not known until run time. So with all that considered, just setting a constant missing from the header files seems the best solution. As VAX C is only needed for building on VMS 5.5-1 and earlier, unless there is active builders for that old of VMS, I have been removing VAX C specific code as I update modules or converting it to ANSI compliance. In tests/scripts/vms/library I suggest to remove the universal option from the test script. For image libraries, librarian seems to be happy with an image which is flagged as a shareable image no matter whether it has a global symbol table or not. So there is no need to have one on VAX but none on Alpha/I64; in other words on all VMS platforms I suggest to either use a global symbol table or to use none. I will look at doing that in a future patch. It appears that the UNIVERSAL qualifier is ignored by non-VAX, so is not causing a problem. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 6/6/2014 9:39 AM, h.becker wrote: On 06/06/2014 05:28 AM, John E. Malmberg wrote: On 5/31/2014 7:45 AM, h.becker wrote: On 05/15/2014 05:51 AM, John Malmberg wrote: While I have not checked on VAX, on both Alpha and Itanium the module name can have both upper and lower case in it. This will mainly show up when the source module is on an ODS-5 volume. Maybe mainly. But as we all know, the module name space is separated from the file name space. Independent of ODS5 and/or process settings like case sensitive mode and/or DECC features for ODS5, for C sources, lower case module names will show as soon as someone has a '#pragma module lowercase' and compiles with /case=as_is with a recent C compiler. Agreed, on VMS the module name space is separate from the filesystem. As pointed out before, GNU make does not have a way to make that distinction. So GNU make can only use case sensitive module name comparisons on VMS with ODS-5 support. It is something that can be looked at for a future enhancement after some tests are added for the expected behavior. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #42447] Fix exporting symbols for recursive make on VMS.
On 6/20/2014 7:59 AM, h.becker wrote: On 06/14/2014 07:14 PM, John E. Malmberg wrote: About command line length: I think I understand. So it would be a compile time option for Alpha/I64. Or do you really have VAX/VMS 8.3? A typo. VAX/VMS stopped at 7.3 I know of 8.2 but never saw 8.3. I left VMS Engineering just after VMS 8.3 got shipped. There were significant fixes to the CRTL for Unix compatibility. Again, me thinks, letting make always generate command procedures is a much simpler approach with less code changes. We can potentially add that as a mode later, but right now I want to implement it this way, after we get all the tests passing, then we can look at making other changes. It looks like there will be a lot of incremental changes to get all the test working and get symbolic links and dynamic loaded modules working. So I now want to get this set of changes committed so that we do not end up with a large set of changes that will be hard for anyone to review or follow. Then DCL has to read from the mailbox and then all the symbol substitutions should work. To me this looks more like a ONESHELL implementation. It is a ONESHELL implementation. Yes, even for a single action, one can replace the file based command procedures with mailbox based command procedures. To me using mailboxes seems to be a bigger change. With fast disks and cached files it may not be worth the effort. But what do I know who uses old systems, anyway. My DS-10 has slow disks (IDE), and I am using NFS served disks for source files. Mailboxes will be a bigger change, but they have the potential for the greatest speedup. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #41758] VMS Make incorrectly reports archives support present.
On 6/20/2014 8:53 AM, h.becker wrote:
On 06/14/2014 07:44 PM, John E. Malmberg wrote:
globalvalue is visible in /STANDARD=VAXC and /STANDARD=RELAXED which is
the default. Is that what you refer to as 'ANSI compatible VMS
extension'? The default gives at least fair diagnostics for VAXC
problems. At the moment gnu make is compiled either without a standard
switch or the /STANDARD=RELAXED, but with
/WARNING=(DISABLE=QUESTCOMPARE). In stricter modes you can use something
like
#include stdio.h
extern unsigned int LBR$_HDRTRUNC;
int main (void) {
printf (LBR$_HDRTRUNC: %%x%08x\n, LBR$_HDRTRUNC);
}
I will change the code to use an extern not as efficient as a constant,
but the code is not on section that would really notice the difference
in speed.
and you may want to add at least DOLLARID to the disabled warning messages.
I plan on looking at the compiler options in the future. They will be
come more relevant when implementing dynamic loaded modules.
I consider /standard=vaxc == ignore potential bugs.
I will look at doing that in a future patch. It appears that the
UNIVERSAL qualifier is ignored by non-VAX, so is not causing a problem.
Like %LINK-I-OPTIGNALPHA, UNIVERSAL option ignored; not allowed for AXP
images but
%ILINK-F-OPTSYNERR, syntax error in options file ...
-ILINK-E-OPTLIN, options line in error
'UNIVERSAL'=MAIN
I didn't try the tests on I64, but the -F should trigger something.
I was not seeing this, but will remove it from the next patch. I will
not be posting an updated patch until next week though.
Regards,
-John
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness patch for VMS.
On 4/1/2014 7:38 PM, John E. Malmberg wrote: On 4/1/2014 1:14 PM, h.becker wrote: I still can't get it to work: Missed that it also needs the sh as a foreign command to run Bash. Replacement patch attached. What do we need to do to get the patches to for the tests to be run on VMS checked in? Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #42447] Fix exporting symbols for recursive make on VMS.
On 6/21/2014 5:36 PM, John E. Malmberg wrote: So I now want to get this set of changes committed so that we do not end up with a large set of changes that will be hard for anyone to review or follow. Just found some tabs in the new code and a small bug in use __VAX instead of the version of VMS in VMS_EXPORT_SYMBOL.C. The corrected code is: #if __CRTL_VER = 70301000 #define transpath_parm transpath #else static char transpath[MAX_DCL_SYMBOL_VALUE]; #endif I will not be able to submit a revised patch until next week. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Test harness patch for VMS.
On 6/24/2014 9:04 AM, Paul Smith wrote: On Sat, 2014-06-21 at 17:56 -0500, John E. Malmberg wrote: On 4/1/2014 7:38 PM, John E. Malmberg wrote: On 4/1/2014 1:14 PM, h.becker wrote: I still can't get it to work: Missed that it also needs the sh as a foreign command to run Bash. Replacement patch attached. What do we need to do to get the patches to for the tests to be run on VMS checked in? Hi John. I was away last weekend but I'll have time to work on this in the evenings this week. I've been following along as best I can (not being a VMS user) with your conversations. If you point out to me the current agreed patch for this I'll look into adding into the repository. 0001-Test-harness-updates-for-VMS-take-2.patch.gz from 4/1/2014. Hartmut Becker wanted to change the name of the test_make.com script and put it in the tests directory. I would like to eventually move all the VMS specific code and scripts into a vms subdirectory. Do you have any access to Git in your development environment? I have Git on cygwin and am NFS serving the directory to the VMS systems used for development. I am away from those systems until this weekend though. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [PATCH] Fix VMS handling of action lines
On 9/18/2014 5:08 PM, h.becker wrote: Fix VMS handling of action lines * job.c: fix make action continuation lines. * vmsjobs.c: fix writing DCL command files when trimming (white spaces and $ signs) especially after a split (command continuation). --- job.c |2 ++ vmsjobs.c | 96 - 2 files changed, 53 insertions(+), 45 deletions(-) Before applying this patch, but with my pending patches: Using command file only setting: 100 Tests in 33 Categories Failed Using the command file only for very long commands or redirection, which is the traditional way: 44 Tests in 20 Categories Failed. After applying this patch and with my pending patches: Using the command file only setting: 96 Tests in 31 Categories Failed Using the command file only for very long commands or redirection, which is the traditional way: 41 Tests in 20 Categories Failed Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
VMS run_make_tests fixes
This changes run_make_test.pl to set the $port_type to 'VMS-DCL' or
default to 'UNIX' based on how the test harness is run on VMS as was
previously discussed with Paul.
This also fixes the run_make_tests.com to work with a search list.
A search list on VMS is similar to an overlay file system.
Regards,
-John
From cc3c5d012bdcd120e1c0f0b2e63e46d6575f237b Mon Sep 17 00:00:00 2001
From: John Malmberg wb8...@qsl.net
Date: Mon, 6 Oct 2014 21:46:34 -0500
Subject: [PATCH] VMS run_make_tests fixes.
Fixes for running tests on VMS.
The changes for the test scripts will follow in groups to be determined.
* run_make_tests.pl: set $port_type to be 'VMS-DCL' when the test are
run from the VMS DCL Interpreter. When the tests are run from GNV on VMS,
the $port_type will be 'UNIX'.
* run_make_tests.com: VMS search list support. This is needed for using
a search list such as prj_root = lcl_root:,vms_root:,src_root: for
building and testing.
---
tests/run_make_tests.com |6 +++---
tests/run_make_tests.pl |7 +++
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/tests/run_make_tests.com b/tests/run_make_tests.com
index 2f17f7f..2b620cf 100755
--- a/tests/run_make_tests.com
+++ b/tests/run_make_tests.com
@@ -238,18 +238,18 @@ $!
$ if no_gnv .or. no_perl then exit 44
$!
$!
-$ make := $bin:make.exe
$ default = f$environment(DEFAULT)
+$ default_dev = f$element(0, :, default) + :
$ this = f$environment(PROCEDURE)
$ on error then goto all_error
-$ set default 'f$parse(this,,,DEVICE)''f$parse(this,,,DIRECTORY)'
+$ set default 'default_dev''f$parse(this,,,DIRECTORY)'
$!
$! Need to make sure that the config-flags.pm exists.
$ if f$search(config-flags.pm) .eqs.
$ then
$ @config_flags_pm.com
$ endif
-$ define/user bin 'default',gnv$gnu:[bin]
+$ define/user bin 'default_dev'[-],gnv$gnu:[bin]
$ define/user decc$filename_unix_noversion enable
$ define/user decc$filename_unix_report enable
$ define/user decc$readdir_dropdotnotype enable
diff --git a/tests/run_make_tests.pl b/tests/run_make_tests.pl
index 9468fab..1145c73 100644
--- a/tests/run_make_tests.pl
+++ b/tests/run_make_tests.pl
@@ -359,6 +359,12 @@ sub set_more_defaults
elsif ($osname =~ m%OS/2%) {
$port_type = 'OS/2';
}
+
+ # VMS has a GNV Unix mode or a DCL mode.
+ # The SHELL environment variable should not be defined in VMS-DCL mode.
+ elsif ($osname eq 'VMS' !defined $ENV{SHELL}) {
+ $port_type = 'VMS-DCL';
+ }
# Everything else, right now, is UNIX. Note that we should integrate
# the VOS support into this as well and get rid of $vos; we'll do
# that next time.
@@ -377,6 +383,7 @@ sub set_more_defaults
# Find the full pathname of Make. For DOS systems this is more
# complicated, so we ask make itself.
if ($osname eq 'VMS') {
+ $port_type = 'VMS-DCL' unless defined $ENV{SHELL};
# On VMS pre-setup make to be found with simply 'make'.
$make_path = 'make';
} else {
--
1.7.9
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
VMS: Exit and exported symbol fixes.
+vms_export_symbol.obj: vms_export_symbol.c
+
config.h: config.h-vms
$(CP) $ $@
diff --git a/makeint.h b/makeint.h
index fdcae75..72d29e5 100644
--- a/makeint.h
+++ b/makeint.h
@@ -99,6 +99,15 @@ extern int errno;
# define isblank(c) ((c) == ' ' || (c) == '\t')
#endif
+#ifdef __VMS
+/* In strict ANSI mode, VMS compilers should not be defining the
+ VMS macro. Define it here instead of a bulk edit for the correct code.
+ */
+#ifndef VMS
+#define VMS
+#endif
+#endif
+
#ifdef HAVE_UNISTD_H
# include unistd.h
/* Ultrix's unistd.h always defines _POSIX_VERSION, but you only get
@@ -201,6 +210,9 @@ unsigned int get_path_max (void);
# include perror.h
/* Needed to use alloca on VMS. */
# include builtins.h
+
+extern int vms_use_mcr_command;
+extern int vms_always_use_cmd_file;
#endif
#ifndef __attribute__
@@ -624,6 +636,25 @@ extern const char *program;
#ifdef VMS
const char *vms_command(const char *argv0);
const char *vms_progname(const char *argv0);
+
+void vms_exit(int);
+# define _exit(foo) vms_exit(foo)
+# define exit(foo) vms_exit(foo)
+
+extern char * program_name;
+
+void set_program_name(const char * arv0);
+
+int need_vms_symbol(void);
+
+int create_foreign_command(const char * command, const char * image);
+
+int vms_export_dcl_symbol(const char * name, const char * value);
+
+int vms_putenv_symbol(const char * string);
+
+void vms_restore_symbol(const char * string);
+
#endif
extern char *starting_directory;
@@ -643,18 +674,9 @@ extern int handling_fatal_signal;
#endif
-#ifdef VMS
-/* These are the VMS __posix_exit compliant exit codes, constructed out of
- STS$M_INHIB_MSG, C facility code, a POSIX condition code mask, MAKE_NNN3 and
- the coresponding VMS severity, here STS$K_SUCCESS and STS$K_ERROR. */
-# define MAKE_SUCCESS 0x1035a001
-# define MAKE_TROUBLE 0x1035a00a
-# define MAKE_FAILURE 0x1035a012
-#else
-# define MAKE_SUCCESS 0
-# define MAKE_TROUBLE 1
-# define MAKE_FAILURE 2
-#endif
+#define MAKE_SUCCESS 0
+#define MAKE_TROUBLE 1
+#define MAKE_FAILURE 2
/* Set up heap debugging library dmalloc. */
diff --git a/vms_exit.c b/vms_exit.c
new file mode 100644
index 000..6effcb9
--- /dev/null
+++ b/vms_exit.c
@@ -0,0 +1,73 @@
+/* vms_exit.c
+ *
+ * Wrapper for the VMS exit() command to tranlate UNIX codes to be
+ * encoded for POSIX, but also have VMS severity levels.
+ * The posix_exit() variant only sets a severity level for status code 1.
+ *
+ * Author: John E. Malmberg
+ */
+
+#include makeint.h
+
+#include stsdef.h
+void decc$exit(int status);
+#ifndef C_FACILITY_NO
+#define C_FACILITY_NO 0x35
+#endif
+
+/* Lowest legal non-success VMS exit code is 8 */
+/* GNU make only defines codes 0, 1, 2 */
+/* So assume any exit code 8 is a VMS exit code */
+
+#ifndef MAX_EXPECTED_EXIT_CODE
+#define MAX_EXPECTED_EXIT_CODE 7
+#endif
+
+/* Build a Posix Exit with VMS severity */
+void vms_exit(int status) {
+int vms_status;
+/* Fake the __posix_exit with severity added */
+/* Undocumented correct way to do this. */
+vms_status = 0;
+
+/* The default DECC definition is not compatible with doing a POSIX_EXIT */
+/* So fix it. */
+if (status == EXIT_FAILURE) {
+status = MAKE_FAILURE;
+}
+
+/* Trivial case exit success */
+if (status == 0) {
+decc$exit(STS$K_SUCCESS);
+}
+
+/* Is this a VMS status then just take it */
+if (status MAX_EXPECTED_EXIT_CODE) {
+/* Make sure that the message inhibit is set since message has */
+/* already been displayed. */
+vms_status = status | STS$M_INHIB_MSG;
+decc$exit(vms_status);
+}
+
+/* Unix status codes are limited to 1 byte, so anything larger */
+/* is a probably a VMS exit code and needs to be passed through */
+/* A lower value can be set for a macro. */
+/* Status 0 is always passed through as it is converted to SS$_NORMAL */
+/* Always set the message inhibit bit */
+vms_status = C_FACILITY_NO | 0xA000 | STS$M_INHIB_MSG;
+vms_status |= (status 3);
+
+/* STS$K_ERROR is for status that stops makefile that a simple */
+/* Rerun of the makefile will not fix. */
+
+if (status == MAKE_FAILURE) {
+vms_status |= STS$K_ERROR;
+} else if (status == MAKE_TROUBLE) {
+/* Make trouble is for when make was told to do nothing and */
+/* found that a target was not up to date. Since a second */
+/* of make will produce the same condition, this is of */
+/* Error severity */
+vms_status |= STS$K_ERROR;
+}
+decc$exit(vms_status);
+}
diff --git a/vms_export_symbol.c b/vms_export_symbol.c
new file mode 100644
index 000..2b7551b
--- /dev/null
+++ b/vms_export_symbol.c
@@ -0,0 +1,481 @@
+/* File: vms_export_symbol.c
+ *
+ * Some programs need special environment variables deported as DCL
+ * DCL symbols.
+ */
+
+/* Copyright 2014 Free Software Foundation, Inc.
+
+GNU Make is free software; you can redistribute
VMS: Fix explicit rules for VMS
This patch fixes make on VMS to be able to run the se_implicit tests.
1. Add the missing default rules that were needed for the se_implicit
tests to pass.
2. Remove the disabling of UNIX path parsing on VMS.
3. Fix some of the VMS path parsing. More work still needs to be done
there, but this is a start.
The above got the se_implicit tests to pass, but then most of the other
tests failed.
4. Found a static buffer being used to hold translated filenames being
used simultaneously for two names being compared for exactness.
That got most of the tests that were broken by fixing the path parsing
working as expected except for the VPATH tests.
5. Removed the disabling of UNIX path parsing on VMS.
Now the self tests with all the previous uncommitted fixes posted
yesterday are reporting 42 tests in 21 categories failing.
Before this patch, but with the those uncommitted fixes, it was 44 tests
failing.
There are currently a total of 464 tests that are run for VMS.
Need to get the test script changes checked in next after this, so need
some guidance on how to split them up.
Need to compose a usage note for readme.vms. Short version:
For 4.1 and earlier:
1. Most Unix file path parsing is disabled.
2. Default rules were missing.
3. VMS file parsing is very incomplete and quite broken in many places.
4. Files that were not existing were found by make.
With this fix:
1. Most Unix file parsing is working.
2. Default rules are all present, but some probably need VMS specific
tweaks and verifications.
3. VMS file parsing is working better, but running with -d is still
showing some obvious bugs.
Regards,
-John
From 8a703b6c6c45e7e8b3ca75156ab95bc35ccd8699 Mon Sep 17 00:00:00 2001
From: John Malmberg wb8...@qsl.net
Date: Tue, 7 Oct 2014 19:23:47 -0500
Subject: [PATCH] VMS Fix implicit rules and Unix Paths
This fixes the implicit rules and Unix style pathname handling.
It also fixes some of the VMS style pathname handling, more work there
will be needed later.
TODO: There is at least another case insensitive platform besides VMS.
We need to find out why there is some extra VMS code for this. This
indicates either the extra VMS code is not needed, or the case
insensitive support may not be complete on the other case insensitive platforms.
* default.c: default_suffix_rules[] was missing definitions.
default.c: default_variables[] was missing definitions.
TODO: As it is important that VMS DCL mode definitions must always be a
superset of Unix definitions, a better way of maintaining the VMS DCL
mode definitions should be devised.
* dir.c: (downcase_implace) New routine. Existing downcase() routine
is not reentrant.
dir.c: Add future support for VMS 8.2+ _USE_STD_STAT macro which will
disable a lot of VMS specific code from compiling.
dir.c: (dir_file_exists_p) vmsify filename only if directory name has
VMS directory delimiters.
dir.c: (file_exists_p) Handle both VMS and Unix directories.
dir.c: (file_impossible) Handle both VMS and Unix directories. Keep
track of if a VMS format path is needed for the return value.
* file.c: (lookup_file) Need to check if vmsify is needed, handle Unix paths.
* implicit.c: (pattern_search) Enable Unix paths.
* read.c: (parse_file_seq) Enable Unix paths.
* rule.c: (count_implicit_rule) Enable Unix paths, Fix VMS paths.
* vpath.c: (selective_vpath_search) Enable Unix paths.
Please enter the commit message for your changes. Lines starting
---
default.c | 83 +++
dir.c | 216
file.c | 28 ++--
implicit.c | 25 +---
read.c | 22 --
vpath.c| 34 -
6 files changed, 323 insertions(+), 85 deletions(-)
diff --git a/default.c b/default.c
index 3b6f7ae..93c993e 100644
--- a/default.c
+++ b/default.c
@@ -131,10 +131,47 @@ static struct pspec default_terminal_rules[] =
static const char *default_suffix_rules[] =
{
#ifdef VMS
+.o,
+$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.obj,
+$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.s,
+$(LINK.s) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.S,
+$(LINK.S) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.c,
+$(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.cc,
+$(LINK.cc) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.C,
+$(LINK.C) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.cpp,
+$(LINK.cpp) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.f,
+$(LINK.f) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.m,
+$(LINK.m) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.p,
+$(LINK.p) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.F,
+$(LINK.F) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.r,
+$(LINK.r) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.mod,
+$(COMPILE.mod) -o $@ -e $@ $^,
+
+.def.sym,
+$(COMPILE.def) -o $@ $,
+
+.sh,
+copy $ $@,
+
.obj.exe,
$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
.mar.exe,
$(COMPILE.mar) $^ \n
Re: VMS: Exit and exported symbol fixes.
On 10/7/2014 11:20 PM, Paul Smith wrote:
John: thanks for these fixes!
On Mon, 2014-10-06 at 23:50 -0500, John E. Malmberg wrote:
+#ifndef C_FACILITY_NO
+#define C_FACILITY_NO 0x35
+#endif
+#ifndef VMS_POSIX_EXIT_MASK
+#define VMS_POSIX_EXIT_MASK (C_FACILITY_NO | 0xA000)
+#endif
We try to indent preprocessor macros inside conditionals; this isn't
100% followed but it's preferred.
+ if ((c-cstatus VMS_POSIX_EXIT_MASK) == VMS_POSIX_EXIT_MASK)
+{
+ status = (c-cstatus 3 255) 8;
+}
The GNU coding standard used by GNU make says that braces should not be
used for single-statement blocks (I know this is controversial in some
quarters, but that's how all the rest of the GNU make code is written).
There are many places like this.
$dayjob coding standard is always include braces, but I can follow your
rules.
+ {
+ char * value;
+ value = getenv(GNV$MAKE_USE_MCR);
The coding standards require * to be cuddled with the variable name.
Also, I think value should be const char *value; here.
And, the standard requires a space between the function name and open
paren, so value = getenv ( There are a number of instances of
missing space.
I missed that one, I noticed that the extra space is there for the rest
of Make.
+/* Build a Posix Exit with VMS severity */
+void vms_exit(int status) {
Return types should come on the previous line, and function opening
braces should be on the next line in the first column of the next line,
plus whitespace between the name and open paren, like this:
void
vms_exit (int status)
{
I see the braces indented 2 spaces for most of the code.
Same for structs.
+++ b/vms_export_symbol.c
This file seems to use a very different coding style; is it shared with
another program such that you don't want to use the GNU standard style?
It will probably eventually be in a VMS analog to GNULIB. The coding
style is one that $dayjob recommends.
+++ b/vms_progname.c
That one is shared with other open source projects as it emulates the
GNU progname.c.
Ditto here.
Plus, we need to add all these new files to the Makefile.am in the
EXTRA_DIST variable otherwise they'll be left out of the distribution
tarballs etc.
I always forget about the Makefile.am since we can not currently run
autoconf on VMS.
I can try to resubmit tomorrow evening.
Regards,
-John
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
Re: VMS: Exit and exported symbol fixes.
get_path_max (void);
# include perror.h
/* Needed to use alloca on VMS. */
# include builtins.h
+
+extern int vms_use_mcr_command;
+extern int vms_always_use_cmd_file;
#endif
#ifndef __attribute__
@@ -624,6 +636,31 @@ extern const char *program;
#ifdef VMS
const char *vms_command(const char *argv0);
const char *vms_progname(const char *argv0);
+
+void vms_exit(int);
+# define _exit(foo) vms_exit(foo)
+# define exit(foo) vms_exit(foo)
+
+extern char * program_name;
+
+void
+set_program_name (const char * arv0);
+
+int
+need_vms_symbol (void);
+
+int
+create_foreign_command (const char * command, const char * image);
+
+int
+vms_export_dcl_symbol (const char * name, const char * value);
+
+int
+vms_putenv_symbol (const char * string);
+
+void
+vms_restore_symbol (const char * string);
+
#endif
extern char *starting_directory;
@@ -643,18 +680,9 @@ extern int handling_fatal_signal;
#endif
-#ifdef VMS
-/* These are the VMS __posix_exit compliant exit codes, constructed out of
- STS$M_INHIB_MSG, C facility code, a POSIX condition code mask, MAKE_NNN3 and
- the coresponding VMS severity, here STS$K_SUCCESS and STS$K_ERROR. */
-# define MAKE_SUCCESS 0x1035a001
-# define MAKE_TROUBLE 0x1035a00a
-# define MAKE_FAILURE 0x1035a012
-#else
-# define MAKE_SUCCESS 0
-# define MAKE_TROUBLE 1
-# define MAKE_FAILURE 2
-#endif
+#define MAKE_SUCCESS 0
+#define MAKE_TROUBLE 1
+#define MAKE_FAILURE 2
/* Set up heap debugging library dmalloc. */
diff --git a/vms_exit.c b/vms_exit.c
new file mode 100644
index 000..591377a
--- /dev/null
+++ b/vms_exit.c
@@ -0,0 +1,76 @@
+/* vms_exit.c
+ *
+ * Wrapper for the VMS exit() command to tranlate UNIX codes to be
+ * encoded for POSIX, but also have VMS severity levels.
+ * The posix_exit() variant only sets a severity level for status code 1.
+ *
+ * Author: John E. Malmberg
+ */
+
+#include makeint.h
+
+#include stsdef.h
+void
+decc$exit (int status);
+#ifndef C_FACILITY_NO
+# define C_FACILITY_NO 0x35
+#endif
+
+/* Lowest legal non-success VMS exit code is 8 */
+/* GNU make only defines codes 0, 1, 2 */
+/* So assume any exit code 8 is a VMS exit code */
+
+#ifndef MAX_EXPECTED_EXIT_CODE
+# define MAX_EXPECTED_EXIT_CODE 7
+#endif
+
+/* Build a Posix Exit with VMS severity */
+void
+vms_exit (int status) {
+
+ int vms_status;
+ /* Fake the __posix_exit with severity added */
+ /* Undocumented correct way to do this. */
+ vms_status = 0;
+
+ /* The default DECC definition is not compatible with doing a POSIX_EXIT */
+ /* So fix it. */
+ if (status == EXIT_FAILURE)
+status = MAKE_FAILURE;
+
+ /* Trivial case exit success */
+ if (status == 0)
+decc$exit (STS$K_SUCCESS);
+
+ /* Is this a VMS status then just take it */
+ if (status MAX_EXPECTED_EXIT_CODE)
+{
+ /* Make sure that the message inhibit is set since message has */
+ /* already been displayed. */
+ vms_status = status | STS$M_INHIB_MSG;
+ decc$exit (vms_status);
+}
+
+ /* Unix status codes are limited to 1 byte, so anything larger */
+ /* is a probably a VMS exit code and needs to be passed through */
+ /* A lower value can be set for a macro. */
+ /* Status 0 is always passed through as it is converted to SS$_NORMAL */
+ /* Always set the message inhibit bit */
+ vms_status = C_FACILITY_NO | 0xA000 | STS$M_INHIB_MSG;
+ vms_status |= (status 3);
+
+ /* STS$K_ERROR is for status that stops makefile that a simple */
+ /* Rerun of the makefile will not fix. */
+
+ if (status == MAKE_FAILURE)
+vms_status |= STS$K_ERROR;
+ else if (status == MAKE_TROUBLE)
+{
+ /* Make trouble is for when make was told to do nothing and */
+ /* found that a target was not up to date. Since a second */
+ /* of make will produce the same condition, this is of */
+ /* Error severity */
+ vms_status |= STS$K_ERROR;
+}
+ decc$exit (vms_status);
+}
diff --git a/vms_export_symbol.c b/vms_export_symbol.c
new file mode 100644
index 000..eb44192
--- /dev/null
+++ b/vms_export_symbol.c
@@ -0,0 +1,523 @@
+/* File: vms_export_symbol.c
+ *
+ * Some programs need special environment variables deported as DCL
+ * DCL symbols.
+ */
+
+/* Copyright 2014 Free Software Foundation, Inc.
+
+GNU Make is free software; you can redistribute it and/or modify it under the
+terms of the GNU General Public License as published by the Free Software
+Foundation; either version 3 of the License, or (at your option) any later
+version.
+
+GNU Make is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
+A PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+this program. If not, see http://www.gnu.org/licenses/. */
+
+
+/* Per copyright assignment agreement with the Free Software Foundation
+ this software may be available under under other license
Re: VMS: Fix explicit rules for VMS
On 10/7/2014 11:43 PM, Paul Smith wrote:
Some of the same formatting notes as before; please check your code
against the style of the rest of the code.
Reformatted patch to GNU coding standard.
Regards,
-John
From 1260e3ae9c0725e3f8f7e7a790c9f5fcf67fbde6 Mon Sep 17 00:00:00 2001
From: John Malmberg wb8...@qsl.net
Date: Tue, 7 Oct 2014 19:23:47 -0500
Subject: [PATCH] VMS Fix implicit rules and Unix Paths Oct 8
This fixes the implicit rules and Unix style pathname handling.
It also fixes some of the VMS style pathname handling, more work there
will be needed later.
TODO: There is at least another case insensitive platform besides VMS.
We need to find out why there is some extra VMS code for this. This
indicates either the extra VMS code is not needed, or the case
insensitive support may not be complete on the other case insensitive platforms.
* default.c: default_suffix_rules[] was missing definitions.
default.c: default_variables[] was missing definitions.
TODO: As it is important that VMS DCL mode definitions must always be a
superset of Unix definitions, a better way of maintaining the VMS DCL
mode definitions should be devised.
* dir.c: (downcase_implace) New routine. Existing downcase() routine
is not reentrant.
dir.c: Add future support for VMS 8.2+ _USE_STD_STAT macro which will
disable a lot of VMS specific code from compiling.
dir.c: (dir_file_exists_p) vmsify filename only if directory name has
VMS directory delimiters.
dir.c: (file_exists_p) Handle both VMS and Unix directories.
dir.c: (file_impossible) Handle both VMS and Unix directories. Keep
track of if a VMS format path is needed for the return value.
* file.c: (lookup_file) Need to check if vmsify is needed, handle Unix paths.
* implicit.c: (pattern_search) Enable Unix paths.
* read.c: (parse_file_seq) Enable Unix paths.
* rule.c: (count_implicit_rule) Enable Unix paths, Fix VMS paths.
* vpath.c: (selective_vpath_search) Enable Unix paths.
Please enter the commit message for your changes. Lines starting
---
default.c | 83 +
dir.c | 202 +++-
file.c | 28 ++---
implicit.c | 17 +++---
read.c | 22 ---
vpath.c| 40 +---
6 files changed, 302 insertions(+), 90 deletions(-)
diff --git a/default.c b/default.c
index 3b6f7ae..93c993e 100644
--- a/default.c
+++ b/default.c
@@ -131,10 +131,47 @@ static struct pspec default_terminal_rules[] =
static const char *default_suffix_rules[] =
{
#ifdef VMS
+.o,
+$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.obj,
+$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.s,
+$(LINK.s) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.S,
+$(LINK.S) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.c,
+$(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.cc,
+$(LINK.cc) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.C,
+$(LINK.C) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.cpp,
+$(LINK.cpp) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.f,
+$(LINK.f) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.m,
+$(LINK.m) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.p,
+$(LINK.p) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.F,
+$(LINK.F) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.r,
+$(LINK.r) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.mod,
+$(COMPILE.mod) -o $@ -e $@ $^,
+
+.def.sym,
+$(COMPILE.def) -o $@ $,
+
+.sh,
+copy $ $@,
+
.obj.exe,
$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
.mar.exe,
$(COMPILE.mar) $^ \n $(LINK.obj) $(subst .mar,.obj,$^) $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
+.s.o,
+$(COMPILE.s) -o $@ $,
.s.exe,
$(COMPILE.s) $^ \n $(LINK.obj) $(subst .s,.obj,$^) $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
.c.exe,
@@ -205,6 +242,27 @@ static const char *default_suffix_rules[] =
.tex.dvi,
$(TEX) $,
+.cpp.o,
+$(COMPILE.cpp) $(OUTPUT_OPTION) $,
+.f.o,
+$(COMPILE.f) $(OUTPUT_OPTION) $,
+.m.o,
+$(COMPILE.m) $(OUTPUT_OPTION) $,
+.p.o,
+$(COMPILE.p) $(OUTPUT_OPTION) $,
+.r.o,
+$(COMPILE.r) $(OUTPUT_OPTION) $,
+.mod.o,
+$(COMPILE.mod) -o $@ $,
+
+.c.ln,
+$(LINT.c) -C$* $,
+.y.ln,
+$(YACC.y) $ \n rename y_tab.c $@,
+
+.l.ln,
+@$(RM) $*.c\n $(LEX.l) $ $*.c\n$(LINT.c) -i $*.c -o $@\n $(RM) $*.c,
+
#else /* ! VMS */
.o,
@@ -413,19 +471,44 @@ static const char *default_variables[] =
LDLIBS, ,
#endif
+LINK.o, $(LD) $(LDFLAGS),
LINK.obj, $(LD) $(LDFLAGS),
#ifndef GCC_IS_NATIVE
CXXLINK.obj, $(CXXLD) $(LDFLAGS),
COMPILE.cxx, $(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH),
#endif
COMPILE.c, $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH),
+LINK.c, $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH),
+COMPILE.m, $(OBJC) $(OBJCFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c,
+LINK.m, $(OBJC) $(OBJCFLAGS) $(CPPFLAGS) $(LDFLAGS) $(TARGET_ARCH),
COMPILE.cc, $(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH),
Re: VMS: Exit and exported symbol fixes.
On 10/9/2014 1:00 AM, Paul Smith wrote: Thanks John, this looks much better. One question: On Wed, 2014-10-08 at 21:18 -0500, John E. Malmberg wrote: +++ b/vms_exit.c This file doesn't have any copyright statement in it; I assume that's an oversight and it's intended to have the standard FSF copyright? Yes. ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: VMS: Exit and exported symbol fixes.
On 10/9/2014 7:26 AM, John E. Malmberg wrote: On 10/9/2014 1:00 AM, Paul Smith wrote: Thanks John, this looks much better. One question: On Wed, 2014-10-08 at 21:18 -0500, John E. Malmberg wrote: +++ b/vms_exit.c This file doesn't have any copyright statement in it; I assume that's an oversight and it's intended to have the standard FSF copyright? Yes. Corrected patch against today's master. Regards, -John From c006bf2c5c187104ae48efed9817941395c2431b Mon Sep 17 00:00:00 2001 From: John Malmberg wb8...@qsl.net Date: Sat, 13 Sep 2014 20:20:22 -0500 Subject: [PATCH] Bug 42447: VMS simulate exporting symbols Oct 9. This also includes fixing the most of the exit handling code for VMS. The vms_progname.c is designed as a drop-in replacement for the gnulib progname.c used in various gnu packages. Self tests: Previously about 94 Tests in 36 categories fail. Now about 45 tests in 22 categories fail. Because some tests do not properly clean up, the number of tests that fail can vary by one or two test cases between consecutive runs. The required VMS updates for most of the self tests has not yet been checked in. * Makefile.am: Add new VMS files. * job.c: add prototype for vms_strsignal(). * job.c: (child_error): Remove VMS specific code as no longer needed. * job.c: (reap_children): The VMS specific code was setting the status to 0 instead of setting it to the proper exit status. * job.h: Add vms_launch_status to struct child. * main.c: (main): Use environment variables for options to use MCR * instead of a foreign command, and to always use command files for subprocesses. For VMS use (set_program_name) routine which is common to ports of other GNU packages to VMS to set the program name used internally. Use (vms_putenv_symbol) to set up symbols to be visible in child programs, including recursive make launched by execve() Start of Bash shell detection code for VMS. * makefile.com: Need nested_include=none for building on VMS search lists. Add vms_progname, vms_exit, and vms_export_symbol. * makefile.vms: Need nested_include=none for building on VMS search lists. Add vms_progname, vms_exit, vms_export_symbol. * makeint.h: Make sure non-standard VMS macro is defined. Add prototypes for new VMS routines. Remove VMS specific MAKE failure codes. * vmsjobs.c: Add VMS Posix exit code constants. Add (_is_unixy_shell) for detection of Bash shell. Add (vms_strsignal) for strsignal() simulation on VMS. Fix (vmsHandleChildTerm) fix to properly report failed LIB$SPAWN() exit status codes. Remove code that duplicated the now fixed code in job.c. (child_execute_job): Export environment symbols before spawning a child and restore afterward unless option to use command files for subprocesses is set. Improve handling of Unix null commands :. * vms_exit.c: New file. Provides (vms_exit) which detects if an exit code is Unix or VMS, and encodes the Unix codes into the appropriate VMS exit code. * vms_export_symbol.c: New file. Routines to create DCL symbols that work like shell aliases or exported shell symbols and clean them up when GNU make exits. * vms_export_symbol_test.com:: Unit test for vms_export_symbol_test.com * vms_progname.c: New file: VMS specific replace for progname.c that is used in some GNU projects. --- Makefile.am|3 +- job.c | 37 ++-- job.h |1 + main.c | 65 +- makefile.com | 11 +- makefile.vms | 19 +- makeint.h | 52 - vms_exit.c | 95 vms_export_symbol.c| 523 vms_export_symbol_test.com | 37 +++ vms_progname.c | 463 +++ vmsjobs.c | 210 ++ 12 files changed, 1381 insertions(+), 135 deletions(-) create mode 100644 vms_exit.c create mode 100644 vms_export_symbol.c create mode 100644 vms_export_symbol_test.com create mode 100644 vms_progname.c diff --git a/Makefile.am b/Makefile.am index 204ab32..d2451b8 100644 --- a/Makefile.am +++ b/Makefile.am @@ -79,7 +79,8 @@ EXTRA_DIST = README build.sh.in $(man_MANS) \ README.W32 NMakefile config.h.W32 build_w32.bat subproc.bat \ make_msvc_net2003.sln make_msvc_net2003.vcproj \ README.VMS makefile.vms makefile.com config.h-vms \ - vmsdir.h vmsfunctions.c vmsify.c \ + vmsdir.h vmsfunctions.c vmsify.c vms_exit.c vms_progname.c \ + vms_export_symbol.c vms_export_symbol_test.com \ gmk-default.scm gmk-default.h # This is built during configure, but behind configure's back diff --git a/job.c b/job.c index 2989249..3baa5c7 100644 --- a/job.c +++ b/job.c @@ -59,9 +59,21 @@ int batch_mode_shell = 0; #elif defined (VMS) # include descrip.h +# include stsdef.h const char *default_shell = ; int batch_mode_shell = 0; +#define strsignal vms_strsignal +char
Re: VMS: Archive and test fixes.
On 10/6/2014 10:22 PM, John E. Malmberg wrote:
This is an update to a previous submitted archive patch.
Just noticed that this still was reverting a change for the test to use
the AR program discovered by configure.
Since VMS does not run configure, also put in a fallback to 'ar' if
configure did not discover the AR program.
This fixes arscan.c to use ANSI compatible code instead of VAX C extension.
The tests have been updated based on changes just submitted for the
run_make_test.pl that had been discussed with Paul.
There are also some fixes based on bug-fixes discovered to get the all
the se_implicit tests to pass on VMS.
Regards,
-John
From 862f16891aebd6889358513f64982e2023cdfaad Mon Sep 17 00:00:00 2001
From: John Malmberg wb8...@qsl.net
Date: Thu, 11 Sep 2014 22:39:34 -0500
Subject: [PATCH] vms_archive_fixes_with_tests_oct 9
Bug 41758: Fix archive support for VMS.
Upated to match change to run_make_tests and some future fixes to make
on VMS.
* arscan.c: Use ANSI compatible pragmas instead of VAX C extensions.
* tests/scripts/features/archives: Fix tests to use VMS rules and
answers when running on VMS and using DCL as a shell.
* tests/scripts/features/vpath3: Fix epected answer on test when
run on VMS.
* tests/scripts/vms/library: (New) Test the VMS library rules that
are not tested by existing tests.
---
arscan.c| 12 ++-
tests/scripts/features/archives | 155 +--
tests/scripts/features/vpath3 |8 ++-
tests/scripts/vms/library | 73 ++
4 files changed, 221 insertions(+), 27 deletions(-)
create mode 100644 tests/scripts/vms/library
diff --git a/arscan.c b/arscan.c
index 24286fd..a3c2b8b 100644
--- a/arscan.c
+++ b/arscan.c
@@ -38,12 +38,18 @@ this program. If not, see http://www.gnu.org/licenses/. */
#include ssdef.h
#include stsdef.h
#include rmsdef.h
-globalvalue unsigned int LBR$_HDRTRUNC;
-#if __DECC
+/* This symbol should be present in lbrdef.h. */
+#ifndef LBR$_HDRTRUNC
+#pragma extern_model save
+#pragma extern_model globalvalue
+extern unsigned int LBR$_HDRTRUNC;
+#pragma extern_model restore
+#endif
+
#include unixlib.h
#include lbr$routines.h
-#endif
+
const char *
vmsify (const char *name, int type);
diff --git a/tests/scripts/features/archives b/tests/scripts/features/archives
index b0acfec..effaa95 100644
--- a/tests/scripts/features/archives
+++ b/tests/scripts/features/archives
@@ -9,68 +9,161 @@ This only works on systems that support it.;
exists $FEATURES{archives} or return -1;
# Create some .o files to work with
-utouch(-60, qw(a1.o a2.o a3.o));
+if ($osname eq 'VMS') {
+ use Cwd;
+ my $pwd = getcwd;
+ # VMS AR needs real object files at this time.
+ foreach $afile ('a1', 'a2', 'a3') {
+# Use non-standard extension to prevent implicit rules from recreating
+# objects when the test tampers with the timestamp.
+1 while unlink $afile.c1;
+1 while unlink $afile.o;
+open (MYFILE, $afile.c1);
+print MYFILE int $afile(void) {return 1;}\n;
+close MYFILE;
+system(cc $afile.c1 /object=$afile.o);
+ }
+} else {
+ utouch(-60, qw(a1.o a2.o a3.o));
+}
my $ar = $CONFIG_FLAGS{AR};
+# Fallback if configure did not find AR, such as VMS
+# which does not run configure.
+$ar = 'ar' if $ar eq '';
+
+my $redir = 21;
+$redir = if $osname eq 'VMS';
+
# Some versions of ar print different things on creation. Find out.
-my $created = `$ar rv libxx.a a1.o 21`;
+my $created = `ar rv libxx.a a1.o $redir`;
# Some versions of ar print different things on add. Find out.
-my $add = `$ar rv libxx.a a2.o 21`;
+my $add = `$ar rv libxx.a a2.o $redir`;
$add =~ s/a2\.o/#OBJECT#/g;
# Some versions of ar print different things on replacement. Find out.
-my $repl = `$ar rv libxx.a a2.o 21`;
+my $repl = `ar rv libxx.a a2.o $redir`;
$repl =~ s/a2\.o/#OBJECT#/g;
unlink('libxx.a');
# Very simple
+my $answer = ar rv libxx.a a1.o\n$created;
+if ($port_type eq 'VMS-DCL') {
+ $answer = 'library /replace libxx.a a1.o';
+}
run_make_test('all: libxx.a(a1.o)',
- '', $ar rv libxx.a a1.o\n$created);
+ '', $answer);
# Multiple .o's. Add a new one to the existing library
($_ = $add) =~ s/#OBJECT#/a2.o/g;
+
+$answer = ar rv libxx.a a2.o\n$_;
+if ($port_type eq 'VMS-DCL') {
+ $answer = 'library /replace libxx.a a2.o';
+}
run_make_test('all: libxx.a(a1.o a2.o)',
- '', $ar rv libxx.a a2.o\n$_);
+ '', $answer);
# Touch one of the .o's so it's rebuilt
-utouch(-40, 'a1.o');
+if ($port_type eq 'VMS-DCL') {
+ # utouch is not changing what VMS library compare is testing for.
+ # So do a real change by regenerating the file.
+ 1 while unlink('a1.o');
+ # Later time stamp than last insertion.
+ sleep(2);
+ system('cc a1.c1 /object=a1.o');
+ # Next insertion will have a later timestamp.
+ sleep(2);
+} else {
+ utouch(-40, 'a1.o');
+}
+
($_ = $repl) =~ s/#OBJECT#/a1.o/g
README.VMS and NEWS update patch.
This patch is assuming that the previously submitted patches get applied for the current README.VMS behavior to match. Regards, -John From 0730fd4e8bd8c61f16bd233b56c57b928897bae6 Mon Sep 17 00:00:00 2001 From: John Malmberg wb8...@qsl.net Date: Wed, 15 Oct 2014 07:19:42 -0500 Subject: [PATCH] Update news and readme.vms This merges the history information from README.VMS to NEWS and updates README.VMS with more usage information. * NEWS: Merge in VMS history. * README.VMS: Remove VMS history, document current behavior and known issues. --- NEWS | 272 ++- README.VMS | 625 ++-- 2 files changed, 669 insertions(+), 228 deletions(-) diff --git a/NEWS b/NEWS index 9e1ade4..4566fc7 100644 --- a/NEWS +++ b/NEWS @@ -15,6 +15,13 @@ A complete list of bugs fixed in this version is available here: http://sv.gnu.org/bugs/index.php?group=makereport_id=111fix_release_id=106set=custom +* VMS: Perl test harness now works. + +* VMS: Full support for converting Unix exit status codes to VMS exit + status codes. BACKWARD INCOMPATIBILITY Notice: On a child failure the + VMS exit code is now the encoded Unix exit status that Make usually + generates, not the VMS exit status of the child. + Version 4.1 (05 Oct 2014) @@ -36,6 +43,31 @@ http://sv.gnu.org/bugs/index.php?group=makereport_id=111fix_release_id=105set make. Makefiles that rely on this syntax should be fixed. See https://savannah.gnu.org/bugs/?33034 +* VMS: Support for library files added, including support for using the + GNV ar utility. + +* VMS: Partial support for properly encoding Unix exit status codes into + VMS exit status codes. + + WARNING: Backward-incompatibility! These are different exit status codes + than Make exited with in the past. + +* VMS: Macros to hold the current make command are set up to translate + the argv[0] string to a VMS format path name and prefix it with + MCR so that the macro has a space in it. + + WARNING: Backward-incompatibility! This may break complex makefiles that + do processing on those macros. This is unlikely because so much in that + area was not and is still not currently working on VMS, it is unlikely to + find such a complex makefile, so this is more likely to impact construction + of a future makefile. + +* VMS: A command file is always used to run the commands for a recipe. + + WARNING: Backward-incompatibility! Running the make self tests has exposed + that there are significant differences in behavior when running with + the command file mode. It is unknown if this will be noticed by most + existing VMS makefiles. Version 4.0 (09 Oct 2013) @@ -244,6 +276,57 @@ http://sv.gnu.org/bugs/index.php?group=makereport_id=111fix_release_id=104set after the variable name, to allow for simple, conditional, or appending multi-line variable assignment. +* VMS specific changes (from README.VMS): + + * Michael Gehre (at VISTEC-SEMI dot COM) supplied a fix for a problem with +timestamps of object modules in OLBs. The timestamps were not correctly +adjusted to GMT based time, if the local VMS time was using a daylight +saving algorithm and if daylight saving was switched off. + + * John Eisenbraun (at HP dot COM) supplied fixes and and an enhancement to +append output redirection in action lines. + + * Rework of ctrl+c and ctrl+y handling. + + * Fix a problem with cached strings, which showed on case-insensitive file +systems. + + * Build fixes for const-ified code in VMS specific sources. + + * A note on appending the redirected output. With this change, a simple +mechanism is implemented to make work in action lines. In VMS +there is no simple feature like to have DCL command or program +output redirected and appended to a file. GNU make for VMS already +implements the redirection of output. If such a redirection is detected, +an on the action line, GNU make creates a DCL command procedure to +execute the action and to redirect its output. Based on that, now +is also recognized and a similar but different command procedure is +created to implement the append. The main idea here is to create a +temporary file which collects the output and which is appended to the +wanted output file. Then the temporary file is deleted. This is all done +in the command procedure to keep changes in make small and simple. This +obviously has some limitations but it seems good enough compared with +the current implementation. (And in my opinion, redirection is not +really what GNU make has to do.) With this approach, it may happen that +the temporary file is not yet appended and is left in SYS$SCRATCH. +The temporary file names look like CMDx.. Any time the created +command procedure can not complete, this happens. Pressing Ctrl+Y to +abort make is one case. In case of Ctrl+Y the associated
Re: VMS: Fix explicit rules for VMS
On 10/8/2014 10:14 PM, John E. Malmberg wrote:
On 10/7/2014 11:43 PM, Paul Smith wrote:
Some of the same formatting notes as before; please check your code
against the style of the rest of the code.
Reformatted patch to GNU coding standard.
Rebuilt to add a fix to remake.c to call gpath_search correctly for a
VMS path.
This gets the features/vpathgpath test to pass on VMS with VMS format
gpaths.
This makes all the features tests that are currently applicable to VMS
to pass.
Regards,
-John
From 4b5a2da6ef5c5afe525cfb1eced42d2d284b8ad1 Mon Sep 17 00:00:00 2001
From: John Malmberg wb8...@qsl.net
Date: Tue, 7 Oct 2014 19:23:47 -0500
Subject: [PATCH] VMS Fix implicit rules and Unix Paths Oct 16
This fixes the implicit rules and Unix style pathname handling.
It also fixes some of the VMS style pathname handling, more work there
will be needed later.
TODO: There is at least another case insensitive platform besides VMS.
We need to find out why there is some extra VMS code for this. This
indicates either the extra VMS code is not needed, or the case
insensitive support may not be complete on the other case insensitive platforms.
* default.c: default_suffix_rules[] was missing definitions.
default.c: default_variables[] was missing definitions.
TODO: As it is important that VMS DCL mode definitions must always be a
superset of Unix definitions, a better way of maintaining the VMS DCL
mode definitions should be devised.
* dir.c: (downcase_implace) New routine. Existing downcase() routine
is not reentrant.
dir.c: Add future support for VMS 8.2+ _USE_STD_STAT macro which will
disable a lot of VMS specific code from compiling.
dir.c: (dir_file_exists_p) vmsify filename only if directory name has
VMS directory delimiters.
dir.c: (file_exists_p) Handle both VMS and Unix directories.
dir.c: (file_impossible) Handle both VMS and Unix directories. Keep
track of if a VMS format path is needed for the return value.
* file.c: (lookup_file) Need to check if vmsify is needed, handle Unix paths.
* implicit.c: (pattern_search) Enable Unix paths.
* read.c: (parse_file_seq) Enable Unix paths.
* remake.c: (f_mtime) Fix gpath_search call for VMS paths.
* rule.c: (count_implicit_rule) Enable Unix paths, Fix VMS paths.
* vpath.c: (selective_vpath_search) Enable Unix paths.
---
default.c | 83 +
dir.c | 202 +++-
file.c | 28 ++---
implicit.c | 17 +++---
read.c | 22 ---
remake.c | 13 +++-
vpath.c| 40 +---
7 files changed, 313 insertions(+), 92 deletions(-)
diff --git a/default.c b/default.c
index 3b6f7ae..93c993e 100644
--- a/default.c
+++ b/default.c
@@ -131,10 +131,47 @@ static struct pspec default_terminal_rules[] =
static const char *default_suffix_rules[] =
{
#ifdef VMS
+.o,
+$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.obj,
+$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.s,
+$(LINK.s) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.S,
+$(LINK.S) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.c,
+$(LINK.c) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.cc,
+$(LINK.cc) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.C,
+$(LINK.C) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.cpp,
+$(LINK.cpp) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.f,
+$(LINK.f) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.m,
+$(LINK.m) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.p,
+$(LINK.p) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.F,
+$(LINK.F) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.r,
+$(LINK.r) $^ $(LOADLIBES) $(LDLIBS) -o $@,
+.mod,
+$(COMPILE.mod) -o $@ -e $@ $^,
+
+.def.sym,
+$(COMPILE.def) -o $@ $,
+
+.sh,
+copy $ $@,
+
.obj.exe,
$(LINK.obj) $^ $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
.mar.exe,
$(COMPILE.mar) $^ \n $(LINK.obj) $(subst .mar,.obj,$^) $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
+.s.o,
+$(COMPILE.s) -o $@ $,
.s.exe,
$(COMPILE.s) $^ \n $(LINK.obj) $(subst .s,.obj,$^) $(LOADLIBES) $(LDLIBS) $(CRT0) /exe=$@,
.c.exe,
@@ -205,6 +242,27 @@ static const char *default_suffix_rules[] =
.tex.dvi,
$(TEX) $,
+.cpp.o,
+$(COMPILE.cpp) $(OUTPUT_OPTION) $,
+.f.o,
+$(COMPILE.f) $(OUTPUT_OPTION) $,
+.m.o,
+$(COMPILE.m) $(OUTPUT_OPTION) $,
+.p.o,
+$(COMPILE.p) $(OUTPUT_OPTION) $,
+.r.o,
+$(COMPILE.r) $(OUTPUT_OPTION) $,
+.mod.o,
+$(COMPILE.mod) -o $@ $,
+
+.c.ln,
+$(LINT.c) -C$* $,
+.y.ln,
+$(YACC.y) $ \n rename y_tab.c $@,
+
+.l.ln,
+@$(RM) $*.c\n $(LEX.l) $ $*.c\n$(LINT.c) -i $*.c -o $@\n $(RM) $*.c,
+
#else /* ! VMS */
.o,
@@ -413,19 +471,44 @@ static const char *default_variables[] =
LDLIBS, ,
#endif
+LINK.o, $(LD) $(LDFLAGS),
LINK.obj, $(LD) $(LDFLAGS),
#ifndef GCC_IS_NATIVE
CXXLINK.obj, $(CXXLD) $(LDFLAGS),
COMPILE.cxx, $(CXX) $(CXXFLAGS) $(CPPFLAGS) $(TARGET_ARCH),
#endif
Re: VMS: Fix explicit rules for VMS
On 10/17/2014 7:45 AM, Paul Smith wrote: On Thu, 2014-10-16 at 23:29 -0500, John E. Malmberg wrote: On 10/8/2014 10:14 PM, John E. Malmberg wrote: On 10/7/2014 11:43 PM, Paul Smith wrote: Some of the same formatting notes as before; please check your code against the style of the rest of the code. Reformatted patch to GNU coding standard. Rebuilt to add a fix to remake.c to call gpath_search correctly for a VMS path. OK thanks John. This weekend I'm going to work on getting the series of patches you provided over the last few weeks committed. On the archive patch, I forgot to add the new directory and test script to the Makefile.am file. How do you want me to start packaging up the patches to the test scripts and when do you want me to start? I can add the update to the Makefile.am to one of those. The next thing I am going to work on is getting the functions/call test to pass on VMS. It is mainly failing due to VMS not handling backslashes at the end of lines properly, but I think I will also need to add some code to convert shell syntax to VMS DCL to avoid reworking the test script as much. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Test updated for VMS.
In general, I sometimes needed to tweak both recipes and the text that is printed out. If I can add better shell simulation to the VMS port, and fix up the output, I may be able to remove some of the edits in the future. But I think there may always be some differences. I also file a ticket on general issues that I discovered with the test harness. Right now, only all tests in the features directory are passing, but I do not have time to list the failing tests in the other directories right now. Below is a list of tests that I had to update for VMS: Regards, -John Directory VMS_ROOT:[make.tests.scripts.features] archives.double_colon escape export parallelism patspecific_vars patternrules quoting reinvoke se_explicit se_implicit se_statpat shell_assignment targetvars vpathvpath3 vpathgpath vpathplus Total of 18 files. Directory VMS_ROOT:[make.tests.scripts.functions] dir eval file filter-out foreach origin shellsubstitution valueword Total of 10 files. Directory VMS_ROOT:[make.tests.scripts.misc] general1 general3 general4 Total of 3 files. Directory VMS_ROOT:[make.tests.scripts.options] dash-k dash-l dash-W eval symlinks Total of 5 files. Directory VMS_ROOT:[make.tests.scripts.targets] INTERMEDIATE ONESHELL POSIX Total of 3 files. Directory VMS_ROOT:[make.tests.scripts.variables] automatic GNUMAKEFLAGS private SHELL ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: [bug #43405] [PATCH] pass correct $AR in archives test
On 10/21/2014 5:58 PM, John Malmberg wrote: Additional Item Attachment, bug #43405 (project make): File name: 0001-Do-not-override-AR-macro-on-VMS.patch Size:3 KB ___ Reply to this item at: http://savannah.gnu.org/bugs/?43405 Can this patch be applied to get the archive tests working on VMS again? Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
VMS: bs-nl and command handling fixes.
This patch is a complete rewrite of the VMS command parser. In order to support the existing VMS recipe syntax which is a combination of VMS and Posix shell syntax, VMS had to reparse the action lines to simulate the Posix shell syntax. The existing parsing code was making too many incorrect assumptions about how DCL and POSIX syntax are actually written and needed a complete rewrite. It was also just estimating the size of the buffer to build the command and never doing any bounds checking. This new code knows the VMS DCL limitations and better parses the action lines. While a temporary command file may still be used in some cases, redirection no longer uses temporary files. Before this fix: 43 out of 466 tests fail. After this fix: 21 out of 466 tests fail. Backwards compatibility note: The old parser tolerated some DCL syntax errors that the new parser does not allow. Regards, -John From da68998b0cc648bf7deecebcf5b3039c27dda645 Mon Sep 17 00:00:00 2001 From: John Malmberg wb8...@qsl.net Date: Thu, 27 Nov 2014 21:28:30 -0600 Subject: [PATCH] Fix bs-nl handling for VMS. This fix required a complete rewrite of the command parser vmsjobs.c child_execute_job. The old parser had too many incorrect assumptions about DCL commands and could not be repaired to extended. The parser now more closely parses VMS commands and handles quoted commands and redirection. Command File mode has been improved, but can not fully support bs-nl syntax. * commands.c: vms_comma_separator is now a run-time setting. * function.c: vms_comma_separator is now a run-time setting. * main.c: Add VMS environment variables for run-time settings. * vms_legacy_behavior - Force older behavior. * vms_comma_separator - Commas or spaces for separators. * vms_unix_simulation - Enhanced Posix shell simulation features. * makeint.h: Add VMS run-time option variables. * readme.vms: Update to current behavior. * vms_export_symbol.c: Set max symbol size correctly. * vmsjobs.c: child_execute_job() complete rewrite of VMS comand parsing. --- README.VMS | 150 +-- commands.c |2 +- function.c | 16 +- main.c | 85 +++- makeint.h |6 +- vms_export_symbol.c |6 +- vmsjobs.c | 1303 --- 7 files changed, 1020 insertions(+), 548 deletions(-) diff --git a/README.VMS b/README.VMS index bec405d..71110ee 100644 --- a/README.VMS +++ b/README.VMS @@ -16,18 +16,67 @@ Overview: -*-text-mode-*- The descriptions below are for running GNU make from DCL or equivalent. - Recipe differences: --- GNU Make for OpenVMS can not currently run native Unix make files because of - differences in the implementation that it is not aware of the GNV packages. + differences in the implementation. I am trying to document the current behavior in this section. This is based on the information in the file NEWS. and running the test suite. TODO: More tests are needed to validate and demonstrate the OpenVMS expected behavior. + In some cases the older behavior of GNU Make when run from DCL is not + compatible with standard makefile behavior. + + This behavior can be changed when running GNU Make from DCL by setting + either DCL symbols or logical names of the format GNV$. The settings + are enabled with a string starting with one of '1', 'T', or 'E' for 1, + TRUE, or ENABLE. They are disabled with a '0', 'F', or 'D' for 1, + FALSE, or DISABLE. If they are not explicitly set to one of these + values, then they will be set to their default values. + + The name GNV$MAKE_OLD_VMS when enabled will cause GNU Make to behave as + much as the older method as can be done with out disabling VMS features. + When it is disabled GNU Make have the new behavior which more closely + matches Unix Make behavior. + + The default is currently the old behavior when running GNU Make from DCL. + In the future this may change. When running make from GNV Bash the new + behavior is the default. + + This is a global setting that sets the default behavior for several other + options that can be individually changed. Many of the individual settings + are to make it so that the self tests for GNU Make need less VMS specific + modifications. + + The name GNV$MAKE_COMMA when enabled will cause GNU Make to expect a comma + for a path separator and use a comma for the separator for a list of files. + When disabled, it will cause GNU Make to use a colon for a path separator + and a space for the separator for a list of files. The default is to be + enabled if the GNU Make is set to the older behavior. + + The name GNV$MAKE_SHELL_SIM when enabled will cause GNU Make to try to + simulate a Posix shell more closely. The following behaviors occur: + +* Single quotes are converted to double quotes and any double + quotes
Status of VMS , down to 12 test failures.
This is with the bs-nl patch submitted on November 27, 2014 applied and the features/archives patch submitted October 21, 2014 applied. Since then, I have discovered some of the failures were due to issues with the way the VMS port of Perl is reading the output from the process. The partial line output of fput() is not being read by Perl. As this does not seem to be an issue with Make on VMS, I am adjusting the test scripts to compensate. I also discovered that the VMS port of Perl is creating a DCL symbol c before running the make command in the test. This has a collision with the undefine test, so I needed to change the macro name in the makefile from 'c' to 'cx' to avoid the collision. This bring the test results to: Test run: 457 Pass: 430 skip: 16 fail: 12 The remaining failed tests: options/dash-e 0 of 1 pass - Needs investigation. options/dash-k 2 of 3 pass - Know how to fix - Minor update. options/dash-n 4 of 6 pass - Needs investigation. options/dash-q 7 of 10 pass - Needs investigation. variables/GNUMAKEFLAGS 2 of 2 pass - Needs investigation. variables/MAKEFILES 1 of 2 pass - Needs investigation. variables/MAKEFLAGS 2 of 3 pass - Needs investigation. variables/MAKELEVEL 0 of 1 pass - Needs investigation. variables/MAKE_RESTARTS 2 of 3 pass - Needs investigation. It looks like some problem with environment variables that should be exported are not being seen by the child make programs. Solving that issue should fix the majority of the test failures. Currently skipped scripts: features/export - Expects a Posix shell. features/load - Not implemented on VMS. features/loadapi - Not implemented on VMS. features/output-sync - Not implemented on VMS. features/parallelism - Not implemented on VMS. features/quoting - Expects a Posix shell. features/shell_assignment - Expects a Posix shell. functions/guile - guile not ported to VMS. functions/shell - Expects a Posix shell. options/symlinks - Needs to be implemented on VMS. targets/ONESHELL - Not implemented on VMS. targets/POSIX - Expects a Posix shell. variables/SHELL - Expects a Posix shell. Symbolic links are available on VMS 8.3 and later. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
Re: Status of VMS , down to 12 test failures.
On 12/3/2014 9:01 AM, Paul Smith wrote: On Sun, 2014-11-30 at 19:28 -0600, John E. Malmberg wrote: This is with the bs-nl patch submitted on November 27, 2014 applied and the features/archives patch submitted October 21, 2014 applied. Hi John; thanks for your work here! I'll have some time later this week/this weekend to review and push these changes. Sorry for the delay... Delay is not a problem. Next planned steps: * Add posix exit simulation when the gnv$make_unix_sim is active. VMS exit: 32 bit number. Odd numbers success, even numbers various classes of failures. Posix exit simulation: status 255 transparently encoded to 32 bits status above 255 assumed already in VMS format. * Review the tests that have not been checked in to see what VMS changes can now be backed out. * Find out why the other 11 tests are not seeing the correct environment variables in their test results. Once all the tests are passing the next planned steps are: * Change case sensitive mode to be a run-time option based on the VMS process setting instead of a compile time option. Have to figure out how to get the verification test for that into the test harness * Change compiler flags to be the same as with GNU Awk to fully support dynamic extension loading. /names=(as-is, shorten) /float=(ieee) (except VAX) * Rework the config.h generation to turn on all the options on VMS that should be on, but are not, and then make sure that everything works as expected. This will turn on dynamic extensions and symbolic link support at a minimum. Note that on GNU awk, the config.h.in is checked in to git, and VMS now just reads that to build the config.h with a set of VMS specific command procedures instead of maintaining a VMS specific template. Regards, -John ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
VMS - 442 Tests in 105 Categories No Failures.
I want to write add a few VMS specific tests before submitting a revised patch to the two previously submitted but not applied patches. * Unix shell simulation improved. Now supports variable assignment before command. foo=bar command. * Unix pathname handling improved including autodetection. * VMS handling of empty commands improved. commands.c - VMS FILE_LIST_SEPARATOR now comma or space via runtime setting. function.c - As above, now reports [] or ./ based on VMS CRTL UNIX report setting. job.c - VMS Handle empty commands properly. main.c - Set VMS compatibility options. ':' and '' are also MAP_DIRSEP on VMS. variable.c - Fix enviroment variable lookup. Remove VMS specific @D,%D, etc and use Unix ones. vmsjobs.c - Completely replaced command line parser re-writer. Old code could not be salvaged or repaired. vms_export_symbol.c - Fix DCL Symbol lengths. makeint.h - Support for the above. Regards, -John wb8tyw@qsl.network ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
vmsify/unixify does anyone need the existing broken behavior?
The existing vmsify and unifiy routines are doing completely different translations than what the VMS CRTL is doing, and are also different than just about any other VMS program that I have seen. The code appears to pre-date the VMS CRTL providing equivalent routines, and does not handle logical names or devices correctly at all. Several of the entries in the conversion table, including table lines 3, 4, 5, 9-11 have always been incorrect. Table line 22 is wrong for ODS-5 filenames, and the size of the path name is out of date. ODS-5 paths can be 4095 ASCII, UCS-2, or 8-bit binary characters. What I plan to do is have the routines just call the VMS CRTL conversion routines, and totally junk the broken code. There may be some conversions that the CRTL does not cover, like replacing extra '.' characters to '_', but this should work much better than the existing code. Correct VMS to Unix translations are: foo:bar foo/bar [.foo]bar.bazfoo/bar.baz foo:[bar.baz]x.y /foo/bar/baz/x.y node::foo:[dir] //node/foo/dir/ Of course almost nothing currently used knows that //node is a DECNET address. Regards, -John wb8tyw@qsl.network ___ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
