Re: Wget 1.20.3 v. VMS
From: Darshit Shah > [...] I've gone through it and applied it to the current > Wget repository. Thanks. > I've taken the liberty to make some changes to your patches: [...] That all sounds reasonable to me. > Regarding your troubles with the print functions, [...] I'll try to work up the ambition to do that. > A new release is coming soon with your patches included. Thanks! I noticed that 1.20.3 wasn't ultra-new. I appreciate your getting these changes in before the next version. I'll try to attack that one sooner than I did this one. Thanks again.
Wget 1.20.3 v. VMS
Greetings:
It's been a while since I've tried to put a current version of Wget
onto VMS, but I recently tried 1.20.3, with some success, but with some
changes needed in the main source.
Original and modified files should be available at:
http://antinode.info/ftp/wget/wget-1_20_3a_vms/wget-1_20_3a_mods.zip
Notes follow.
src/hsts.c
time_t on VMS is typically unsigned. (Lazy man's solution to 2038?)
I added "(time_t)" type casts to negative values ("-1"), and changed
tests to avoid complaints like:
return (t < 0 ?
..^
%CC-I-QUESTCOMPARE, In this statement, the unsigned expression "t" is being comp
ared with a relational operator to a constant whose value is not greater than ze
ro. This might not be what you intended.
at line number 224 in file ITS$DKA0:[UTILITY.SOURCE.WGET.wget-1_20_3.src]hsts.c;
1
I believe that it's all compatible with a signed time_t.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
File path name construction using UNIX-only format:
filename = aprintf ("%s/.wget-hsts-test", opt.homedir);
was replaced by a new function which includes (and segregates) the
VMS-specific alternative code:
filename = ajoin_dir_file (opt.homedir, ".wget-hsts-test");
src/init.c
New function, ajoin_dir_file(), to join a directory and file name
(used in hsts.c, init.c, and main.c).
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Removed VMS-specific code using "SYS$LOGIN". (getenv( "HOME) works
on VMS, too, when handled properly.)
src/init.h
Added prototype for new function, ajoin_dir_file().
src/log.c
Disabled check_redirect_output() on VMS (as on Windows).
src/main.c
Changed to use ajoin_dir_file().
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Changed a "`" to "'" in the help text. I assume that you have a
policy on "`", but its use seems inconsistent. I avoid it entirely,
because, in my experience, it's almost always rendered asymmetrically
with respect to "'", hence ugly and distracting. Perhaps I just use bad
fonts.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Added a "use_askpass() not implemented on VMS" message, but did not
actually disable the option.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Added a VMS-specific line (could be more) to the -V/--version report
in print_version(). Typical output at the moment includes the
(run-time) VMS version and the OpenSSL version, if any:
VMS V8.4-2L1, OpenSSL 1.1.1h 22 Sep 2020
src/utils.c
Changed data types in VMS-specific code in fork_to_background() to
agree with changed types in non-VMS code.
Happened to notice:
README
Recursive downloading also works with FTP, where Wget can retrieves a
---^
Other complaints.
I had a bunch of trouble trying to figure out what to do with the GNU
print functions (asprintf(), snprintf(), vasnprintf(), vsnprintf()). I
assume that the UNIX auto-jive copes with this stuff, but I couldn't see
why it would make any sense for vasnprintf() to use snprintf(), and for
snprintf() to use vasnprintf(). It took me a while to diagnose the
resulting stack overflow. I found a solution, but...
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
lib/strerror_r.c
/* On MSVC, there is no snprintf() function, just a _snprintf().
[...]
#if !HAVE_SNPRINTF
[...]
result = _vsnprintf (buf, buflen, format, args);
If the condition for using _vsnprintf() is WINDOWS, then shouldn't
the #if be testing for WINDOWS instead of the apparently unrelated
HAVE_SNPRINTF (which is not defined on VMS)?
I ended up using the following on VMS to get back to the plain
vsnprintf() without editing lib/strerror_r.c:
#define _vsnprintf vsnprintf
But that's not the right way.
If you have any questions/disagreements/requests/suggestions, please
let me know. Thanks, as usual.
Steven M. Schweda s...@antinode.info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.16.3 v. VMS
While I'm thinking about it, ... search!source=hpq=fred: i/o error Cannot write to 'search!source=hpq=fred' (error 0). (Interestingly, in 1.16.1, that last error message was more informative: Cannot write to 'search!source=hpq=fred' (i/o error). but I haven't investigated.) I quick look (at src/http.c) suggests that both messages use strerror (errno). The first one is put out promptly after the failure, and so gets an accurate errno value, but the second one no longer does. I'd guess that somewhere in all the intervening activity, errno gets disturbed (in this case, cleared by some successful operation). Perhaps clearing errno after some successful operation represents a bug in the run-time library on VMS (although a search for errno = 0 in the Wget code will not disappoint), but it might be more reliable to save a bad status value somewhere other than in errno if any serious work (which might affect errno) will be done between the actual failure (like this file open) and any (much later) consumption (like the generation of this Cannot write to message). Just a thought. (I hate errno. VMS has its faults, but its native system services return a status value directly to the consumer; they don't stick it into some common location where it's liable to get lost or damaged. If you forget why something failed, it's _your_ fault.) Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.16.3 v. VMS
From: Giuseppe Scrivano gscriv...@gnu.org I am going to do it and push the patch under your name. Sounds good to me. Sorry about the extra work. Thanks. SMS.
Re: [Bug-wget] Wget 1.16.3 v. VMS
From: Giuseppe Scrivano gscriv...@gnu.org Thanks for the prompt response. could you please send the patch as git format-patch output? Please also add the ChangeLog log style to the commit message. I have no objections to the patch itself. It might be possible, but it may not happen soon. I don't have git installed anywhere, and I don't know how to use it. SMS.
[Bug-wget] Wget 1.16.3 v. VMS
: ; = ? */
- 0, 0, 0, 0, 0, 0, 0, 0, /* @ A B CD E F G */
- 0, 0, 0, 0, 0, 0, 0, 0, /* H I J KL M N O */
- 0, 0, 0, 0, 0, 0, 0, 0, /* P Q R ST U V W */
- 0, 0, 0, 0, W, 0, 0, 0, /* X Y Z [\ ] ^ _ */
- 0, 0, 0, 0, 0, 0, 0, 0, /* ` a b cd e f g */
- 0, 0, 0, 0, 0, 0, 0, 0, /* h i j kl m n o */
- 0, 0, 0, 0, 0, 0, 0, 0, /* p q r st u v w */
- 0, 0, 0, 0, W, 0, 0, C, /* x y z {| } ~ DEL */
+UVWC, VC, VC, VC, VC, VC, VC, VC, /* NUL SOH STX ETX EOT ENQ ACK BEL */
+ VC, VC, VC, VC, VC, VC, VC, VC, /* BS HT LF VT FF CR SO SI */
+ VC, VC, VC, VC, VC, VC, VC, VC, /* DLE DC1 DC2 DC3 DC4 NAK SYN ETB */
+ VC, VC, VC, VC, VC, VC, VC, VC, /* CAN EM SUB ESC FS GS RS US */
+ 0, 0, W, 0, 0, 0, 0, 0, /* SP ! #$ % ' */
+ 0, 0, VW, 0, 0, 0, 0, UW, /* ( ) * +, - . / */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* 0 1 2 34 5 6 7 */
+ 0, 0, W, 0, W, 0, W, VW, /* 8 9 : ; = ? */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* @ A B CD E F G */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* H I J KL M N O */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* P Q R ST U V W */
+ 0, 0, 0, 0, W, 0, 0, 0, /* X Y Z [\ ] ^ _ */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* ` a b cd e f g */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* h i j kl m n o */
+ 0, 0, 0, 0, 0, 0, 0, 0, /* p q r st u v w */
+ 0, 0, 0, 0, W, 0, 0, C, /* x y z {| } ~ DEL */
C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, /* 128-143 */
C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, /* 144-159 */
@@ -1381,10 +1385,13 @@
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
};
#undef U
+#undef V
#undef W
#undef C
#undef UW
-#undef UWC
+#undef UVWC
+#undef VC
+#undef VW
/* FN_PORT_SEP is the separator between host and port in file names
for non-standard port numbers. On Unix this is normally ':', as in
@@ -1393,10 +1400,14 @@
#define FN_PORT_SEP (opt.restrict_files_os != restrict_windows ? ':' : '+')
/* FN_QUERY_SEP is the separator between the file name and the URL
- query, normally '?'. Since Windows cannot handle '?' as part of
+ query, normally '?'. Because VMS and Windows cannot handle '?' in a
file name, we use '@' instead there. */
-#define FN_QUERY_SEP (opt.restrict_files_os != restrict_windows ? '?' : '@')
-#define FN_QUERY_SEP_STR (opt.restrict_files_os != restrict_windows ? ? :
@)
+#define FN_QUERY_SEP \
+ (((opt.restrict_files_os != restrict_vms) \
+ (opt.restrict_files_os != restrict_windows)) ? '?' : '@')
+#define FN_QUERY_SEP_STR \
+ (((opt.restrict_files_os != restrict_vms) \
+ (opt.restrict_files_os != restrict_windows)) ? ? : @)
/* Quote path element, characters in [b, e), as file name, and append
the quoted string to DEST. Each character is quoted as per
@@ -1415,6 +1426,8 @@
int mask;
if (opt.restrict_files_os == restrict_unix)
mask = filechr_not_unix;
+ else if (opt.restrict_files_os == restrict_vms)
+mask = filechr_not_vms;
else
mask = filechr_not_windows;
if (opt.restrict_files_ctrl)
I assume that I've violated some style standard(s), but I'm old and
hard to educate. (git? What's a git?) Please let me know if I
need to do more to persuade you to adopt/adapt a change set like this,
or if there's a better way. Thanks, as always.
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Bug-wget] [PATCH] Bug 40426 follow-up
#define TMP_OUTFILE /tmp/.wget.stdout What happens when a second user/process tries to do the same thing? Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] some remarks about the changed help text
From: Benno Schulenberg Oh, there is an #ifdef __VMS around the -K, --backup-converted, but for my life I cannot see a difference between the then and the else. X_orig v. X.orig? Multiple dots in file names cause problems on (old) ODS2 disks, so it's most convenient not to (try to) add them. Luckily, no one's life depends on this. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] retr-symlinks and directories
From: Matthew Atkinson Would it be reliable enough to try CWD to it [...] I believe that that's the standard method. For example, googlebot seems to do similar things all the time (for plain-old files, not even for symlinks): [...] 5:34.00 User:anonymous logged in ident:google...@google.com from Host:66.249.69.179 5:34.40 User:anonymous ident:google...@google.com status:07649912 CWD dir:robots.txt 5:34.68 User:anonymous ident:google...@google.com status: RETR file:SYS$SYSDEVICE:[ANONYMOUS]robots.txt;1 [...] [...] what is the safest way to restore the original PWD? [...] I haven't tested it anywhere, so I know nothing, but I'd guess that if you ask the server where it is -- ftp_pwd() -- and then feed that result back into ftp_cwd(), then any server should be happy. (That's essentially what the mess in getftp() is doing to deal with a relative path.) [...] And what should be done on error in this case? Exploit Roe v. Wade? Suicide? The only call to ftp_cwd() is wrapped in code which seems to be there to deal with VMS. The mess for dealing with a VMS server is forced by the way Wget (specifically, getftp()) sends a UNIX-like path spec instead of following the RFC (that is, sending the path segments individually). If you ask the server using ftp_pwd(), and then return what it said using ftp_cwd(), then no special handling should be needed for any competent server (or even for a VMS server). Does this need to be done in this case, No. (I believe.) and if so should this be moved into ftp_cwd() or another function so it can be reused? No. ftp_cwd() is a simple, primitive function which should do what it does, namely send a CWD command. The mess in getftp() is specific to the way getftp() works; it's not there because of a flaw in ftp_cwd(). Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] [PATCH 10/14] Do not depend on always defined macros
From: Giuseppe Scrivano gscriv...@gnu.org ops, thanks to have catched it. I've fixed it in my local copy, this is how it looks now: diff --git a/vms/vms.h b/vms/vms.h index d65aeda..6053df1 100644 --- a/vms/vms.h +++ b/vms/vms.h @@ -49,16 +49,6 @@ int utime( const char *path, const struct utimbuf *times); There was also that comment near the beginning: ALP $ gdiff wget-1_14/vms/vms.h_orig wget-1_14/vms/vms.h 10,12d9 * Emergency substitution of stat() for lstat() for VAX and VMS CRTL * before V7.3-1. * It might make sense to fetch one of my 1.15 source kits, and see what else has changed since 1.12. (Probably many things.) I don't know how much sense it makes to include VMS-specific files in the official sources. Without a VMS system, you really can't maintain/test them, and I usually don't do anything until a real release occurs, so it's hard for you to do a release with good VMS stuff in it. And, recent tar kits have not included any VMS-specific files, anyway. SMS.
Re: [Bug-wget] Update VMS Info?
From: Darshit Shah dar...@gmail.com Thanks for the info. There's a bunch of files in vms/ which is always being shipped with the official sources. Define always and shipped (or official sources). I see no vms/ directory (or descrip*.mms files) in the 1.15 tar kit. Nor in the 1.14 tar kit. Nor in the 1.13.4 tar kit. I'm working with the latest git sources. Not sure about the tarball. On the latest git sources, the vms/wget.hlp file doesn't seem to be updated. Hence, I'm assuming that the updated files aren't being shipped at all. I'm working with the official tar kits. Haven't looked at git. The VMS-specific files have changed with every release. If you weren't shipping them in the tar kits, then I didn't send back explicit updates, because it appeared (to me) that there was nothing to update. prohibit_HAVE_MBRTOWC vms/config.h_vms:278:#define HAVE_MBRTOWC 1 maint.mk: do not use \bHAVE_MBRTOWC\b; it is always defined maint.mk:462: recipe for target 'sc_prohibit_HAVE_MBRTOWC' failed make: *** [sc_prohibit_HAVE_MBRTOWC] Error 1 That's interesting, but I don't invent this stuff. As it says at the front of (the current) vms/config.h_vms: /* vms/config.h_vms. VMS-specific config.h for Wget 1.15. */ /* Hand-edited from Tru64. */ /* src/config.h. Generated from config.h.in by configure. */ /* src/config.h.in. Generated from configure.ac by autoheader. */ That is, I ran configure on a Tru64 system (where the compiler most cloesly resembles the VMS compiler), and hand-edited the result. If the Tru64 config.h says: /* Define to 1 if you have the `mbrtowc' function. */ #define HAVE_MBRTOWC 1 (which it does), then you can expect to see something similar in vms/config.h_vms. prohibit_always-defined_macros vms/config.h_vms:#define alloca __ALLOCA maint.mk: define the above via some gnulib .h file maint.mk:848: recipe for target 'sc_prohibit_always-defined_macros' failed make: *** [sc_prohibit_always-defined_macros] Error 1 I'm not likely to try to change some gnulib .h file to clear that one. prohibit_always_true_header_tests [...] vms/config.h_vms:67:/* #define HAVE_ALLOCA_H 1 */ vms/config.h_vms:168:#define HAVE_ERRNO_H 1 vms/config.h_vms:204:#define HAVE_GETOPT_H 1 vms/config.h_vms:241:#define HAVE_INTTYPES_H 1 vms/config.h_vms:360:/* #undef HAVE_STDBOOL_H */ vms/config.h_vms:362:# define HAVE_STDBOOL_H 1 vms/config.h_vms:366:#define HAVE_STDDEF_H 1 vms/config.h_vms:369:/* #define HAVE_STDINT_H 1 */ vms/config.h_vms:375:#define HAVE_STDIO_H 1 vms/config.h_vms:378:#define HAVE_STDLIB_H 1 vms/config.h_vms:399:/* #define HAVE_STRINGS_H 1 */ vms/config.h_vms:401:# define HAVE_STRINGS_H 1 vms/config.h_vms:405:#define HAVE_STRING_H 1 vms/config.h_vms:444:#define HAVE_SYS_IOCTL_H 1 vms/config.h_vms:453:/* #define HAVE_SYS_SELECT_H 1 */ vms/config.h_vms:456:#define HAVE_SYS_SOCKET_H 1 vms/config.h_vms:459:#define HAVE_SYS_STAT_H 1 vms/config.h_vms:462:#define HAVE_SYS_TIME_H 1 vms/config.h_vms:465:#define HAVE_SYS_TYPES_H 1 vms/config.h_vms:477:/* #define HAVE_TERMIOS_H 1 */ vms/config.h_vms:489:#define HAVE_UNISTD_H 1 vms/config.h_vms:510:#define HAVE_WCHAR_H 1 vms/config.h_vms:516:#define HAVE_WCTYPE_H 1 maint.mk: do not test the above HAVE_header_H symbol(s); with the corresponding gnulib module, they are always true maint.mk:803: recipe for target 'sc_prohibit_always_true_header_tests' failed make: *** [sc_prohibit_always_true_header_tests] Error 1 Same as above. If it's in the config.h generated on Tru64, then it's likely to appear in vms/config.h_vms. If you don't want to see these things there, then don't show them to me. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
[Bug-wget] Update VMS Info?
From: Darshit Shah dar...@gmail.com I was going through the VMS specific files in the source and realized that a lot of the documentation seems to be a little outdated. I'm always glad to see that VMS has not been forgotten here. For example the WGET.HLP file still refers to Wget 1.12 as the latest version and explains everything based on it. Maybe it can be updated by whoever is working on a VMS system? I didn't realize that any VMS-specific files were being maintained in the official code. My current (1.15a) source kit includes an updated vms/wget.hlp which should be up-to-date. It's in: http://antinode.info/ftp/wget/wget-1_15a_vms/wget-1_15a_src.zip vms/vms_notes.txt (also in there) includes a list of files changed or added for VMS. Most are in vms/, but decc_ver.c and vms.c are in src/, and there are some descrip*.mms builders in lib/ and src/. There's another request too. `make syntax-check` gives a few issues with the config.h_vms file. Now I'm not sure if those declarations are indeed required or not. If not, can we please get rid of them? Starting with a tar kit, I get: deb4# make syntax-check No version control files detected; skipping syntax check config.h_vms gets updated with every version, so, if you're seeing an old vms/wget.hlp, then I'd guess that you're also seeing an old vms/config.h_vms. But the latest one could be little or no better in this respect. Can you reveal the actual complaints, or do I need to do something with git to get the required version control files, or what? Thanking You, Same here. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Mac OS X wget seems broken?
wget does not download all directories at a URL unless I tack on a sub-directory. Wget and I do not know what all directories at a URL means. And, between my weak psychic powers and a description like [url], I can't tell if Wget is working properly or not. A URL tells Wget which protocol to use, which computer to ask, and what to ask for. Exactly what Wget does depends on the protocol (or scheme, like, say, FTP or HTTP) which the URL specifies. When using FTP, with -r, Wget may get a directory listing, and then it can fetch files recursively. When using HTTP, Wget fetches a Web page, and, with -r, it follows links in the Web page which a Web server returns, If that Web page does not include links to the files of interest, then Wget probably won't find/fetch them. If you want a detailed description of what Wget is doing, add -d to your command. If you want a detailed explanation of why Wget does not work as expected with [url], then you may need to provide some basic information about [url]. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
[Bug-wget] Wget 1.15 --help indentation quirks
While poking at Wget 1.15 on VMS, I noticed some indentation quirks in the --help output (while converting it to a VMS HELP source file), generally related to --long_options (src/main.c): Typical alignment: -S, --server-response print server response. --spider don't download anything. Too far left?: --backups=N before writing file X, rotate up to N backup files. --no-use-server-timestamps don't set the local file's timestamp by the one on the server. --trust-server-names use the name specified by the redirection url last component. --backups=N may fit into 80 columns: --backups=N before writing file X, rotate up to N backup files. --no-use-server-timestamps is too wide to fit easily where it belongs, but that's not my fault. I'd align the hyphens, and move (only) this one explanation a couple of units to the right: --no-use-server-timestamps don't set the local file's timestamp by the one on the server. I haven't yet tried it on a VAX, but with only a little fooling around in the VMS-specific stuff, a build worked on Alpha (so IA64 is likely, too). One quick FTP fetch also worked, so I'm confident that it's not completely broken. (What could go wrong?) Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] New option --no-list-a
I just tested two ftp servers, that I have access to: [...] I saw so far [...] For those seeking more diversity, I run a VMS FTP server using HP's TCPIP IP software package (remarkably lame, but relatively common). One particular directory tree includes several parsing quirks (but not a file named -a): ftp://antinode.info/moz_test/ Simple/stupid FTP clients tend to do better there than Web browsers, but Wget should be able to cope. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget manpage missing options
From: Giuseppe Scrivano g@gnu.org
Yes, I was just waiting for an ACK from Steven if it works on VMS too.
Seems to, if .1 is supposed to be the most recent and .n the
oldest. (VMS versions normally go the other way.) Around here, with a
quick CGI counter, after five runs of:
wgx -d --backups=3 http://alp-l/cgi-bin/counter.com
I got:
alp $ search counter.* count =
**
ALP$DKC0:[SMS.WGET]counter.com;1
count = 5
**
ALP$DKC0:[SMS.WGET]counter.com_1;1
count = 4
**
ALP$DKC0:[SMS.WGET]counter.com_2;1
count = 3
**
ALP$DKC0:[SMS.WGET]counter.com_3;1
count = 2
Seems plausible to me. What could go wrong?
I don't know if the following other VMS-related changes are already
in somewhere else, or if they got lost (at one end or the other), or
what, but, as long as we're all here:
--- lib/tmpdir.c_orig 2013-07-11 15:23:39 -0500
+++ lib/tmpdir.c2013-07-11 22:01:46 -0500
@@ -149,6 +149,15 @@
return -1;
}
+/* 2012-08-18 SMS.
+ * Sadly, on VMS, P_tmpdir is defined as SYS$SCRATCH:, not a
+ * Posix-compatible name, so a VMS-specific scheme (no /) must be used
+ * to concatenate the directory name and the file name.
+ */
+#ifdef __VMS
+ sprintf (tmpl, %.*s%.*sXX, (int) dlen, dir, (int) plen, pfx);
+#else
sprintf (tmpl, %.*s/%.*sXX, (int) dlen, dir, (int) plen, pfx);
+#endif
return 0;
}
(Slightly more compact than before, if you have a before:)
--- src/warc.c_orig 2013-07-04 04:40:52 -0500
+++ src/warc.c 2013-07-12 06:13:39 -0500
@@ -738,8 +738,14 @@
char *new_filename = malloc (base_filename_length + 1 + 5 + 8 + 1);
warc_current_filename = new_filename;
+#ifdef __VMS
+# define WARC_GZ warc-gz
+#else /* def __VMS */
+# define WARC_GZ warc.gz
+#endif /* def __VMS [else] */
+
#ifdef HAVE_LIBZ
- const char *extension = (opt.warc_compression_enabled ? warc.gz : warc);
+ const char *extension = (opt.warc_compression_enabled ? WARC_GZ : warc);
#else
const char *extension = warc;
#endif
@@ -1153,6 +1159,21 @@
if (path_search (filename, 100, opt.warc_tempdir, wget, true) == -1)
return NULL;
+#ifdef __VMS
+ /* 2013-07-12 SMS.
+ * mkostemp()+unlink()+fdopen() scheme causes trouble on VMS, so use
+ * mktemp() to uniquify the (VMS-style) name, and then use a normal
+ * fopen() with a create temp file marked for delete option.
+ */
+ {
+char *tfn;
+
+tfn = mktemp (filename);/* Get unique name from template. */
+if (tfn == NULL)
+ return NULL;
+return fopen (tfn, w+, fop=tmd);/* Create auto-delete temp file. */
+ }
+#else /* def __VMS */
int fd = mkostemp (filename, O_TEMPORARY);
if (fd 0)
return NULL;
@@ -1162,8 +1183,8 @@
return NULL;
#endif
-
return fdopen (fd, wb+);
+#endif /* def __VMS [else] */
}
As always, thanks for the extra VMS-related effort. (Sorry about the
half-baked accusation of half-baked-ness earlier. Well, _something_
was, just not your code. Sigh. But I did catch that bool thing, so I
may not be a total loss.)
SMS.
Re: [Bug-wget] Wget manpage missing options
From: Giuseppe Scrivano g@gnu.org I think the patches should apply without conflicts on 1.14. Could you try to apply the patch I submitted yesterday and see if it works as expected on VMS? If you want I can prepare a new tarball for you. I'd guess that, given that VMS-specific delete() change and code which uses _ instead of . in (all) the constructed names, if it works on a Unix(-like) system, then it should do about as well on VMS. But I could test it. However, after wasting some time looking for the bool backups problem, I'd rather have a complete, consistent kit to work with. If you could (when you get bored) create a tar kit, then I'd be willing to try that. (Thanks.) SMS.
Re: [Bug-wget] Wget manpage missing options
From: Giuseppe Scrivano g@gnu.org Yes Sir, here we go! A freshly baked tarball: http://it.gnu.org/~gscrivano/files/wget-1.14.62-9d54.tar.bz2 Half-baked? Lost a few VMS-specific changes, but the first big problem was: while ((line = read_whole_line (fp)) != NULL) .^ %CC-I-IMPLICITFUNC, In this statement, the identifier read_whole_line is impli citly declared as a function. at line number 588 in file SYS$SYSDEVICE:[UTILITY.SOURCE.WGET.wget-1_14_62-9d54. src]init.c;2 I could declare it (as with secure_getenv()) in vms/config.h_vms if I knew how, but evidence is sparse: ALP $ search [-...]*.h, *.c read_whole_line ** SYS$SYSDEVICE:[UTILITY.SOURCE.WGET.wget-1_14_62-9d54.src]init.c;2 while ((line = read_whole_line (fp)) != NULL) ALP $ I assume that it should be in lib/something.c (which would help at link time, too). SMS.
Re: [Bug-wget] Wget manpage missing options
From: Giuseppe Scrivano g@gnu.org thanks to have tested it, indeed it was working in a different way. I don't think this feature had any user before, considering how broken it is. I assumed that the VMS details were the only remaining mystery, so all I did was look at the generated file names (on VMS), not what was in the files, so almost anything could have happened, and I might not have noticed it. I also assumed that the patch was based on something newer than the original 1.14 source kit I was using, so, even if I had noticed a problem, I probably would not have been suspicious. For example, around here, nothing worked right until I made this change: ALP $ gdiff src/options.h_orig src/options.h 169c169 bool backups; /* Are numeric backups made? */ --- int backups;/* Are numeric backups made? */ When backups can be only zero or one, saying --backups=4 does not have the desired effect. If someone lets me know when the next tar kit appears, then I might be able to do a little more testing (with a little more care). SMS.
Re: [Bug-wget] Wget manpage missing options
From: Giuseppe Scrivano g@gnu.org
G:
Thanks for the e-mail copy. I check the lists.gnu.org/archive pretty
regularly, but the address@hidden stuff there can hide many useful
details.
Tomas Hozza tho...@redhat.com writes:
Options undocumented at all:
--backups
--no-backups
I have two patches here for --backups, one related to VMS and untested.
Could someone please proof-read the documentation bits?
[...]
diff --git a/src/retr.c b/src/retr.c
index 3d51ef9..ab1053e 100644
--- a/src/retr.c
+++ b/src/retr.c
@@ -1194,8 +1194,15 @@ rotate_backups(const char *fname)
for (i = opt.backups; i 1; i--)
{
- sprintf (from, %s.%d, fname, i - 1);
- sprintf (to, %s.%d, fname, i);
+ const char *sep;
+# ifdef __VMS
+ sep = _;
+#else
+ sep = .;
+#endif
+
+ sprintf (from, %s%s%d, fname, sep, i - 1);
+ sprintf (to, %s%s%d, fname, sep, i);
rename (from, to);
}
I haven't tried to get git working here, so I'm looking at original
1.14 source. Did someone already change the rename() code which follows
this loop? I'd use a macro instead of const char *sep;, et al. Also,
for proper operation on VMS, a little more work is needed. Naturally.
Normally, I'd expect a VMS user to rely on the file versioning which
is built into the file system, rather than use any of this backup stuff,
but it can be made to work. My patch (against 1.14) follows.
+@cindex backing up files
+@item --backups=@var{backups}
+Before write a file, back up the original version adding a @samp{.1}
+suffix to the file name. Eventual existing backups will be rotated up
+to @var{backups} files incrementing the suffix by one.
I might say something like:
Before (over)writing a file, back up an existing file by adding a
@samp{.1} suffix (@samp{_1} on VMS) to the file name. Such backup files
are rotated to @samp{.2}, @samp{.3}, and so on, up to @var{backups} (and
lost beyond that).
--- src/retr.c_orig 2012-06-06 06:45:29 -0500
+++ src/retr.c 2013-07-09 17:30:21 -0500
@@ -37,6 +37,9 @@
#include errno.h
#include string.h
#include assert.h
+#ifdef VMS
+# include unixio.h/* For delete(). */
+#endif
#include exits.h
#include utils.h
@@ -1177,7 +1180,16 @@
void
rotate_backups(const char *fname)
{
- int maxlen = strlen (fname) + 1 + numdigit (opt.backups) + 1;
+#ifdef __VMS
+# define SEP _
+# define AVS ;* /* All-version suffix. */
+# define AVSL (sizeof (AVS) - 1)
+#else
+# define SEP .
+# define AVSL 0
+#endif
+
+ int maxlen = strlen (fname) + sizeof (SEP) + numdigit (opt.backups) + AVSL;
char *from = (char *)alloca (maxlen);
char *to = (char *)alloca (maxlen);
struct_stat sb;
@@ -1189,12 +1201,24 @@
for (i = opt.backups; i 1; i--)
{
- sprintf (from, %s.%d, fname, i - 1);
- sprintf (to, %s.%d, fname, i);
+#ifdef VMS
+ /* Delete (all versions of) any existing max-suffix file, to avoid
+ * creating multiple versions of it. (On VMS, rename() will
+ * create a new version of an existing destination file, not
+ * destroy/overwrite it.)
+ */
+ if (i == opt.backups)
+{
+ sprintf (to, %s%s%d%s, fname, SEP, i, AVS);
+ delete (to);
+}
+#endif
+ sprintf (to, %s%s%d, fname, SEP, i);
+ sprintf (from, %s%s%d, fname, SEP, i - 1);
rename (from, to);
}
- sprintf (to, %s.%d, fname, 1);
+ sprintf (to, %s%s%d, fname, SEP, 1);
rename(fname, to);
}
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Syntax for RESTful scripting options
Also, is the --post-file command really needed? Even if one does really need to use POST data from a file, one could always cat the file inside backticks as: wget --options=POST `cat POSTdata` URL Why not go with that? Apparently always means something to you which is different from what it means to me. It may be hard to believe, or you may not care, but some folks use this program on non-UNIX(-like) systems (VMS, for example), where such UNIX(-like) shell tricks are not available. Even on UNIX(-like) systems, some old shells may have limited command-line length. Some users might not want the contents of their file exposed to every user on the system who can run ps. How many reasons do you need not to wreck existing functionality? Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] wget mirroring not syncing FTP directories timestamps
I don't care much about this bug/feature, so I won't argue one way or the other, but I will point out that it often makes little sense to set the modification date-time on a destination directory to that of the source directory. For example, if not all the files in a directory are copied to the destination, then why should the mod date-time on the destination match that of the source, when the directory _contents_ are different? Setting the mod date-time on the destination to some old value can also confuse backup software on the destination system, if it thinks that a recently (created or) modified directory has not been modified recently, so it's not worthy of inclusion in an incremental backup. Strictly speaking, a normal file can be copied to a different file system, but a directory is often (re-) _created_ on the destination file system, not really copied in the same way as a normal file. On VMS, the BACKUP utility normally creates destination directories with the current date-time, not the original. An exception is when using BACKUP /IMAGE, which always copies all files (to a previously empty destination), so a destination directory always has exactly the same contents as the source directory. Because of this file-directory difference, the default behavior of the next version of Info-ZIP UnZip is expected be to restore the mod date-time on an extracted file, but _not_ on an extracted (recreated) directory, but the user will have the option to get directory date-times restored as well as file date-times (or neither). VMS BACKUP knows a /IMAGE save set when it sees one, but UnZip has no way of knowing if an archived directory is complete, so it can't reliably decide whether it makes sense to restore directory date-times. Thus the user needs to specify the behavior. (The behavior of currently released UnZip is a mess, depending on the local OS.) If Wget can't be sure, either, then it may make some sense to have a separate option to cause directory date-times to be preserved, and let the user decide. Just a thought. Why not move the action of setting of mtime point until after leaving the directory? Given the existence of symbolic links (or if, for some other reason, the sender is not always well-organized in the way it sequences the copied files and directories), then the only safe time to set directory attributes is after all the files have been processed. I remember nothing, and I'm too lazy to look at the Wget code, but the same kind of problem arises with directory permissions. You can't create a read-only destination directory, and then proceed to populate it with files. Such permissions must be set after all the files have been processed. (That all the files is all the files in that directory, but if a symlink can redirect a file from somewhere else, then all the files is really all the files.) As usual, everything's complicated Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.11.4 and FTP Server from Windows Server 2008 R2
Is this a known problem? I know nothing, but ... Wget 1.11.4 [...] Not even close to the latest version. wget --dont-remove-listing -r -l1 ftp://192.168.154.9/ Adding -d to that command might provide some interesting info. 192.168.154.9 is not a server which anyone else can reach, which makes exploration difficult. --dont-remove-listing might be helpful if my psychic powers were strong enough that I could see the listing file, but they're not. If you want someone else to investigate this problem, then you may need to provide some information about the listing, or else access to a server which exhibits the problem. But I'd start with a newer version of wget. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] alpha release (1.13.4.56-620c)
It's not really your fault, but the GNU regex code (lib/regcomp.c) has an annoying portability problem, [...] The gnulib folks have fixed this regcomp.c problem. I don't much like the solution they adopted, but it does get past more compilers than the old code. http://lists.gnu.org/archive/html/bug-gnulib/2012-05/msg00276.html Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] alpha release (1.13.4.56-620c)
From: Giuseppe Scrivano gscriv...@gnu.org By using gnulib we can assume these headers _always_ exist. In case the system doesn't provide them, then gnulib will offer a replacement. The problem here is that you can't use configure, right? Yes. In general: no shell, no compatible utilities, ..., no hope. if that is the problem, you can submit a patch which re-introduces these macros. It might not be elegant, but if it solves this problem then there are no problems to leave them there, for systems using configure they are just a no-op. Do you mean like this?: [...] ALP $ gdiff -du src/connect.c_orig src/connect.c [...] http://lists.gnu.org/archive/html/bug-wget/2012-05/msg00086.html Or is something else needed? Providing such a patch helps more than complaining about portability of GNU programs, I think. Complaining is very enjoyable, though. Of course, it's more enjoyable when it's effective (as it generally is here). Apparently, complaining about the fundamental GNU regex problem gets one advice like get the compiler fixed: http://lists.gnu.org/archive/html/bug-gnulib/2012-03/msg00154.html Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] alpha release (1.13.4.56-620c) (was: [PATCH] gnutls.c: fix infinite read timeout)
It's not really your fault, but the GNU regex code (lib/regcomp.c)
has an annoying portability problem, because what could easily have been
a compile-time decision was postponed until run-time. A good compiler
complains about loss of significant bits when BITSET_WORD_BITS 64:
if (BITSET_WORD_BITS == 64)
{
dfa-word_char[0] = UINT64_C (0x03ff);
dfa-word_char[1] = UINT64_C (0x07fe87fe);
Not fatal, however, only annoying. (And, I claim, stupid. And
relatively easy to fix, unless you _really_ like it the way it is, which
seems to be the case for the GNU regex maintainer.)
Actually, on VAX (where a 64-bit integer type in unavailable), it's
fatal:
CC /decc /include = ([], [.VAX], [-.SRC.VAX], [-.VMS]) /prefix_library_entri
es = (all_entries, except = (getopt, optarg, opterr, optind, optopt))/objec
t = [.VAX]REGEX.OBJ /define = (VMS, _POSIX_EXIT, OS_TYPE=VMS VAX V7.3 , -
ENABLE_DEBUG , HAVE_LIBSSL, ENABLE_NTLM ) [-.LIB]REGEX.C
dfa-word_char[0] = UINT64_C (0x03ff);
..^
%CC-E-INTCONST, Ill-formed integer constant.
At line number 963 in SYS$SYSDEVICE:[UTILITY.SOURCE.WGET.WGET-1_
13_4_56-620C.LIB]REGCOMP.C;1.
dfa-word_char[1] = UINT64_C (0x07fe87fe);
..^
%CC-E-INTCONST, Ill-formed integer constant.
At line number 964 in SYS$SYSDEVICE:[UTILITY.SOURCE.WGET.WGET-1_
13_4_56-620C.LIB]REGCOMP.C;1.
I assume that I'll find more problems as I try to get the new code
into the VMS builders, but those are the ones which popped up first.
I'm such a pessimist. Actually, with a little adjustment to the
VMS-specific files, building on Alpha (and, I assume, on IA64 -- What
could go wrong?) seems to be ok. Even more amazingly, I seem to have
found a truly stupid-looking work-around for that
should-have-been-compile-time conditionality problem in the GNU regex
code on VAX:
# ifdef __VAX
# define UINT64_C(c) 0
Apparently, at least in some cases, one piece of truly stupid code can
cancel out another.
HAVE_FSEEKO and HAVE_FTELLO (also unavailable on VAX) seem to get
about as much respect as HAVE_SYS_SELECT_H and HAVE_SYS_SOCKET_H, but I
should be able to deal with that, too.
I haven't done any serious testing (just fetched a Web page on
Alpha), but, except for the problems previously reported, builds on VMS
should be possible.
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] alpha release (1.13.4.56-620c) (was: [PATCH] gnutls.c: fix infinite read timeout)
From: Ray Satiro raysat...@yahoo.com [...] your wget gnulib dir [...] You may be confusing GNU with portable. They're not even similar. All the GNU-autojunk in the world does me approximately no good at all. sys_select.in.h, sys_socket.in.h, and the like, are useful only if someone can turn them into actual header files, and on VMS, I, not some GNU-autojunk, am the one who would need to do it. And, as I said, I don't even have a convenient way to use a sys directory (without wrecking access to existing sys/XXX.h stuff). [...] they are always required.. aren't they? Obviously not, or we wouldn't be having this discussion. For Wget 1.13.4 on VMS, I added these modifications to src/connect.c, because, as I recall, the argument that HAVE_SYS_SOCKET existed for some reason was unsuccessful. I never understood why, so I thought that I'd take another run at it this time. Perhaps my mind is simply too weak for this stuff, but I can't help thinking that if the #define directives for HAVE_SYS_SELECT_H and HAVE_SYS_SOCKET_H in the generated src/config.h are correct, then actually using these macros to guard the relevant #include directives should be harmless, at worst. But what do I know? Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 on True64 OSF1 Beta
-Olimit 2000 ;; *) CFLAGS=$CFLAGS -O ;; [...] That was fun _and_ educational. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 on True64 OSF1 Beta
[...] So, the smart thing to do may be to add a definition of _SOCKADDR_LEN where _POSIX_PII_SOCKET is being defined. (Perhaps _SOCKADDR_LEN should replace _POSIX_PII_SOCKET in these spots, but I don't have any old versions of Tru64, or any non-Tru64 OSF/1 systems, so I can't run any serious tests on that. As I showed, adding a definition of _SOCKADDR_LEN seems to solve the problem on Tru64 V5.1B. I don't know what would break where if the definition of _POSIX_PII_SOCKET were removed, and I'm not eager to find out.) Also in ./config.h here: [...] /* Define to 1 if `sa_len' is a member of `struct sockaddr'. */ #define HAVE_STRUCT_SOCKADDR_SA_LEN 1 [...] This is true, but no one seems to care about the corresponding details of struct sockaddr_in. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
[Various other changes/fixes affecting VMS] Still wondering. For the curious, a set of patches should be available at: http://antinode.info/ftp/wget/wget-1_13_4/1_13_4_1.dru That includes the changes to rehabilitate --preserve-permissions. News: The --help output now has one line which exceeds 80 characters: --random-wait wait from 0.5*WAIT...1.5*WAIT secs between retr ievals. Note, too, that, to a casual reader, it's not entirely clear what WAIT refers to here. -w, --wait=SECONDSwait SECONDS between retrievals. --waitretry=SECONDS wait 1..SECONDS between retries of a retrieval. Possibly SECONDS from one of those? Might be easier (or messier) if --random-wait could take a time value, too. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
From: Giuseppe Scrivano gscriv...@gnu.org http://antinode.info/ftp/wget/wget-1_13_4/1_13_4_1.dru can you please include a ChangeLog entry for each of them? --- ChangeLog._orig 2011-09-13 03:08:59 -0500 +++ ChangeLog. 2011-10-07 14:29:59 -0500 @@ -1,3 +1,38 @@ +2011-10-07 Steven Schweda s...@antinode.info + + * connect.c: Add HAVE_SYS_SELECT_H and HAVE_SYS_SOCKET_H conditions + on includes of sys/select.h and sys/socket.h, respectively. + + * ftp.c (getftp): Move BIN_TYPE_TRANSFER macro into VMS-specific + section. On VMS, use Stream_LF attributes for listing files. Pass + BIN_TYPE_FILE to fopen_excl() instead of constant-everywhere true. + + * ftp.c (ftp_retrieve_list): Restore lost test of opt.preserve_perm + (--preserve-permissions) on the chmod() operation. + + * init.c, main.c: Remove deprecated from opt.preserve_perm + (--preserve-permissions). + + * init.c (initialize): Use distinct messages for errors in C macro + SYSTEM_WGETRC and environment-variable SYSTEM_WGETRC. Avoid use of + C macro SYSTEM_WGETRC when it's not defined. + + * log.c (log_close): Avoid closing logfp when it's stderr. + + * main.c (print_help): Restore --preserve-permissions. + + * main.c (main): Avoid using a negative value of longindex as a + subscript (for long_options[]) when searching for --config. + + * main.c (main): Exit the program using exit() instead of return. + (VMS handles these differently, and exit() is better.) + + * openssl.c (ssl_init): Add type cast (SSL_METHOD *) to newly const + meth argument to accommodate OpenSSL version 0.9.8, where that + argument is not const in the OpenSSL function (SSL_CTX_new). + + * utils.c (fopen_excl): Comment typography. + 2011-09-13 Giuseppe Scrivano gscriv...@gnu.org * ftp.c (ftp_retrieve_glob): Propagate correctly the `res' error Note that in openssl.c, accomodate should be accommodate. (Not the only place in the code, but this one would be new and my fault.) The leading-tab indentation in src/ChangeLog seems to have some exceptions. And, of course, there are various changes to the VMS builders and other accessory files, but I don't know how these will be handled. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
Assuming that I eventually get satisfactory VMS builders constructed, is there any interest in including the VMS-specific files in the main wget distribution kit, or should I assume that the victim will need to obtain and unpack a suplementary VMS-specific kit (scattered VMS builders, vms/ directory) onto a normal wget source directory? (Or a complete VMS-compatible kit, as in the past.) Still wondering. The packaging affects the (VMS-specific) installation instructions, or I would care less. [Various other changes/fixes affecting VMS] Still wondering. News: The --help output now has one line which exceeds 80 characters: --random-wait wait from 0.5*WAIT...1.5*WAIT secs between retr ievals. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
And a (very) minor typo: ALP $ gdiff -u src/utils.c_orig src/utils.c --- src/utils.c_orig2011-08-29 03:01:24 -0500 +++ src/utils.c 2011-10-02 23:22:29 -0500 @@ -769,8 +769,7 @@ open_id = 13; fd = open( fname, /* File name. */ flags, /* Flags. */ - 0777,/* Mode for default protection. -*/ + 0777,/* Mode for default protection. */ rfm=stmlf, /* Stream_LF. */ OPEN_OPT_ARGS); /* Access callback. */ } Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
Two more things, both in src/ftp.c. Part of some VMS-specific
stuff (BIN_TYPE_FILE) got lost/discarded somewhere along the line.
Also, while fooling around, I noticed that .listing files were being
created with binary instead of text attributes. Wget itself didn't seem
to care, but, with --no-remove-listing, for example, the resulting
(persistent) .listing files were unsuitable for use by normal text
editors.
--- ftp.c_orig 2011-09-13 03:05:12 -0500
+++ ftp.c 2011-09-30 16:36:31 -0500
@@ -1152,13 +1152,25 @@
Elsewhere, define a constant binary flag.
Isn't it nice to have distinct text and binary file types?
*/
-# define BIN_TYPE_TRANSFER (type_char != 'A')
+/* 2011-09-30 SMS.
+ Added listing files to the set of non-binary (text, Stream_LF)
+ files. (Wget works either way, but other programs, like, say, text
+ editors, work better on listing files which have text attributes.)
+ Now we use binary attributes for a binary (IMAGE) transfer,
+ unless --ftp-stmlf was specified, and we always use non-binary
+ (text, Stream_LF) attributes for a listing file, or for an ASCII
+ transfer.
+ Tidied the VMS-specific BIN_TYPE_xxx macros, and changed the call to
+ fopen_excl() (restored?) to use BIN_TYPE_FILE instead of true.
+*/
#ifdef __VMS
+# define BIN_TYPE_TRANSFER (type_char != 'A')
+# define BIN_TYPE_FILE \
+ ((!(cmd DO_LIST)) BIN_TYPE_TRANSFER (opt.ftp_stmlf == 0))
# define FOPEN_OPT_ARGS fop=sqo, acc, acc_cb, open_id
# define FOPEN_OPT_ARGS_BIN ctx=bin,stm, rfm=fix, mrs=512 FOPEN_OPT_ARGS
-# define BIN_TYPE_FILE (BIN_TYPE_TRANSFER (opt.ftp_stmlf == 0))
#else /* def __VMS */
-# define BIN_TYPE_FILE 1
+# define BIN_TYPE_FILE true
#endif /* def __VMS [else] */
if (restval !(con-cmd DO_LIST))
@@ -1217,7 +1229,7 @@
}
else
{
- fp = fopen_excl (con-target, true);
+ fp = fopen_excl (con-target, BIN_TYPE_FILE);
if (!fp errno == EEXIST)
{
/* We cannot just invent a new name and use it (which is
Note that the directive:
# define BIN_TYPE_TRANSFER (type_char != 'A')
was outside the #ifdef __VMS block, but BIN_TYPE_TRANSFER is used only
on VMS. This change would move it inside (where it belongs). The
fopen_excl() is generic, but on VMS it needs the variable (on VMS)
BIN_TYPE_FILE argument, not a constant true. (Which is why
BIN_TYPE_FILE exists.)
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
[...]
2 24135 confval = long_options[longindex].val;
Apparently, referencing long_options[-1] causes more trouble in a VMS
environment than it does on a typical UNIX(-like) system. [...]
I still haven't looked at the details here, but I did notice that the
_second_ getopt_long() loop does things like:
/* If LONGINDEX is unchanged, it means RET is referring a short
option. */
if (longindex == -1)
rather than jumping right in and using longindex as a subscript.
Perhaps the first getopt_long() loop should be so careful.
Other helpful changes:
ALP $ gdiff -u src/log.c_orig src/log.c
--- src/log.c_orig 2011-07-29 08:43:44 -0500
+++ src/log.c 2011-09-29 08:43:17 -0500
@@ -573,14 +573,14 @@
}
}
-/* Close LOGFP, inhibit further logging and free the memory associated
- with it. */
+/* Close LOGFP (only if we opened it, not if it's stderr), inhibit
+ further logging and free the memory associated with it. */
void
log_close (void)
{
int i;
- if (logfp)
+ if (logfp (logfp != stderr))
fclose (logfp);
logfp = NULL;
inhibit_logging = true;
(This is particularly helpful if one is putting in debugging code like
fprintf( stderr, [...]), but closing stderr seems like a poor idea in
any case.)
ALP $ gdiff -u src/main.c_orig src/main.c
--- src/main.c_orig 2011-09-06 08:50:11 -0500
+++ src/main.c 2011-09-29 09:15:31 -0500
@@ -1467,7 +1480,7 @@
xfree (url[i]);
cleanup ();
- return get_exit_status ();
+ exit (get_exit_status ());
}
#endif /* TESTING */
On VMS, return x from main() and exit(x) are not equivalent, and
only exit(x) can deal with status codes other than EXIT_FAILURE and
EXIT_SUCCESS in a reasonable way. (exit(x) is used in many other
places, so I wouldn't expect any damage from using it in one more.)
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
[...]
2 24135 confval = long_options[longindex].val;
Apparently, referencing long_options[-1] causes more trouble in a VMS
environment than it does on a typical UNIX(-like) system. [...]
I still haven't looked at the details here, [...]
Now I have. Adding a test on longindex seems to fix the problem
without causing any obvious (to me) damage:
ALP $ gdiff -u src/main.c_orig src/main.c
--- src/main.c_orig 2011-09-06 08:50:11 -0500
+++ src/main.c 2011-09-29 14:28:28 -0500
@@ -945,8 +945,8 @@
init_switches ();
- /* This seperate getopt_long is needed to find the user config
- and parse it before the other user options. */
+ /* This separate getopt_long is needed to find the user config file
+ option (--config) and parse it before the other user options. */
longindex = -1;
int retconf;
bool use_userconfig = false;
@@ -957,20 +957,25 @@
int confval;
bool userrc_ret = true;
struct cmdline_option *config_opt;
- confval = long_options[longindex].val;
- config_opt = option_data[confval ~BOOLEAN_NEG_MARKER];
- if (strcmp (config_opt-long_name, config) == 0)
-{
- userrc_ret = run_wgetrc (optarg);
- use_userconfig = true;
-}
- if (!userrc_ret)
+
+ /* There is no short option for --config. */
+ if (longindex = 0)
{
- printf (Exiting due to error in %s\n, optarg);
- exit (2);
+ confval = long_options[longindex].val;
+ config_opt = option_data[confval ~BOOLEAN_NEG_MARKER];
+ if (strcmp (config_opt-long_name, config) == 0)
+{
+ userrc_ret = run_wgetrc (optarg);
+ use_userconfig = true;
+}
+ if (!userrc_ret)
+{
+ printf (Exiting due to error in %s\n, optarg);
+ exit (2);
+}
+ else
+break;
}
- else
-break;
}
/* If the user did not specify a config, read the system wgetrc and
~/.wgetrc. */
[Plus the same return - exit() change as reported earlier.]
I still have some VMS-specific builder changes to make, but the
changes to src/connect.c, src/log.c, and src/main.c seem to be all
that I've needed in the common code (so far). For completeness:
ALP $ gdiff -u src/connect.c_orig src/connect.c
--- src/connect.c_orig 2011-08-07 15:55:23 -0500
+++ src/connect.c 2011-09-29 09:23:37 -0500
@@ -36,8 +36,13 @@
#include unistd.h
#include assert.h
-#include sys/socket.h
-#include sys/select.h
+#ifdef HAVE_SYS_SOCKET_H
+# include sys/socket.h
+#endif /* def HAVE_SYS_SOCKET_H */
+
+#ifdef HAVE_SYS_SELECT_H
+# include sys/select.h
+#endif /* def HAVE_SYS_SELECT_H */
#ifndef WINDOWS
# ifdef __VMS
Assuming that I eventually get satisfactory VMS builders constructed,
is there any interest in including the VMS-specific files in the main
wget distribution kit, or should I assume that the victim will need to
obtain and unpack a suplementary VMS-specific kit (scattered VMS
builders, vms/ directory) onto a normal wget source directory? (Or a
complete VMS-compatible kit, as in the past.)
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
I still have some VMS-specific builder changes to make, but the changes to src/connect.c, src/log.c, and src/main.c seem to be all that I've needed in the common code (so far). [...] Well, not quite. In src/openssl.c:ssl_init(), there's this: SSL_METHOD const *meth; which causes a problem with some older OpenSSL kits (0.9.8j, for example), where the argument to SSL_CTX_new() was not declared as const (as it is in a newer OpenSSL kit, like, say, 1.0.0e): CC /float = ieee_float /include = ([], [.ALPHAL], [-.LIB], [-.LIB.ALPHAL], [-. VMS]) /prefix_library_entries = (all_entries, except = (utime , __UTC_UTIME)) /object = [.ALPHAL]OPENSSL.OBJ /define = (VMS, _POSIX_EXIT , ENABLE_DEBUG , _LARGEFILE , HAVE_SSL, ENABLE_NTLM ) [-.SRC]OPENSSL.C ssl_ctx = SSL_CTX_new (meth); .^ %CC-W-NOTCONSTQUAL, In this statement, the referenced type of the pointer value meth is const, but the referenced type of the target of this assignment is not . at line number 204 in file ALP2$DKC0:[utility.source.WGET.wget-1_13_4.src]openss l.c;1 Naturally, the latest HP-supplied SSL kit is based on some 0.9.8 kit, so this would likely affect more users than one might expect. I don't know what the preferred solution would be. Adding a type cast seems to fix the problem here: ALP $ gdiff -u src/openssl.c_orig src/openssl.c --- src/openssl.c_orig 2011-08-29 03:01:24 -0500 +++ src/openssl.c 2011-09-29 17:40:09 -0500 @@ -201,7 +201,9 @@ abort (); } - ssl_ctx = SSL_CTX_new (meth); + /* The type cast below accomodates older OpenSSL versions (0.9.8) + where SSL_CTX_new() is declared without a const argument. */ + ssl_ctx = SSL_CTX_new ((SSL_METHOD *)meth); if (!ssl_ctx) goto error; but I haven't tried it in any other environments. And there's still that error message in src/init.c which uses SYSTEM_WGETRC, whether or not SYSTEM_WGETRC is defined. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
And there's still that error message in src/init.c which uses
SYSTEM_WGETRC, whether or not SYSTEM_WGETRC is defined.
One possible solution:
--- src/init.c_orig 2011-08-19 05:06:20 -0500
+++ src/init.c 2011-09-30 00:44:16 -0500
@@ -597,21 +597,34 @@
variable has been set. For internal testing purposes only! */
env_sysrc = getenv (SYSTEM_WGETRC);
if (env_sysrc file_exists_p (env_sysrc))
-ok = run_wgetrc (env_sysrc);
+{
+ ok = run_wgetrc (env_sysrc);
+ /* If there are any problems parsing the system wgetrc file, tell
+ the user and exit */
+ if (! ok)
+{
+ fprintf (stderr, _(\
+Parsing system wgetrc file (env SYSTEM_WGETRC) failed. Please check\n\
+'%s',\n\
+or specify a different file using --config.\n), env_sysrc);
+ exit (2);
+}
+}
/* Otherwise, if SYSTEM_WGETRC is defined, use it. */
#ifdef SYSTEM_WGETRC
else if (file_exists_p (SYSTEM_WGETRC))
ok = run_wgetrc (SYSTEM_WGETRC);
-#endif
/* If there are any problems parsing the system wgetrc file, tell
the user and exit */
if (! ok)
{
fprintf (stderr, _(\
-Parsing system wgetrc file failed, please check '%s'. \
-Or specify a different file using --config\n), SYSTEM_WGETRC);
+Parsing system wgetrc file failed. Please check\n\
+'%s',\n\
+or specify a different file using --config.\n), SYSTEM_WGETRC);
exit (2);
}
+#endif
/* Override it with your own, if one exists. */
file = wgetrc_file_name ();
if (!file)
Aside from working when SYSTEM_WGETRC is not defined, it provides
distinct messages for the two different ways the system wgetrc file
might have been specified.
ALP $ define /user_mode system_wgetrc sys$manager:.wgetrcx
ALP $ wgx --version
WGET: Syntax error in sys$manager:.wgetrcx at line 1.
Parsing system wgetrc file (env SYSTEM_WGETRC) failed. Please check
'sys$manager:.wgetrcx',
or specify a different file using --config.
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.13.4 v. VMS -- Various problems
From: Micah Cowan mi...@cowan.name
On 09/28/2011 06:39 AM, Steven M. Schweda wrote:
From: Micah Cowanmi...@cowan.name
In this case, the logic that does a rename of snprintf seems to be at
the end of vasnprintf.h rather than directly in snprintf.c.
Those aren't the droids you're looking for. Try lib/stdio.in.h
(which I also try very hard to ignore).
Yes, but lib/stdio.in.h isn't included directly by snprintf.c, as
vasnprintf.h is. The bit I'm talking about is indeed in vasnprintf.h,
which is what actually does the rename of the link name.
What, exactly, do you see where, exactly? All I see in
lib/vasnprintf.h is stuff like:
# define vasnprintf rpl_vasnprintf
# define asnprintf rpl_asnprintf
and neither vasnprintf nor asnprintf looks quite like snprintf to
me. But what do I know?
On the ACCVIO front, the first problem seems to occur with any short
option on the command line. My sophisticated diagnostics (fprintf())
suggest that there may be a problem in the command-line parsing in
src/main.c:main(). This seems to be the bad section (worse than just
the spelling of separate):
[...]
/* This seperate getopt_long is needed to find the user config
and parse it before the other user options. */
longindex = -1;
int retconf;
bool use_userconfig = false;
/* SMSd. */
fprintf( stderr, Pre-getopt_long(1) loop. argc = %d, longindex = %d.\n,
argc, longindex);
while ((retconf = getopt_long (argc, argv,
short_options, long_options, longindex)) != -1)
{
int confval;
bool userrc_ret = true;
struct cmdline_option *config_opt;
/* SMSd. */
fprintf( stderr, getopt_long(1) loop. longindex = %d, retconf = %d.\n,
longindex, retconf);
confval = long_options[longindex].val;
/* SMSd. */
fprintf( stderr, getopt_long(1) loop. confval = %d.\n, confval);
[...]
With only a long option, I get output like the following:
ALP $ wgx --hel
Pre-getopt_long(1) loop. argc = 2, longindex = -1.
getopt_long(1) loop. longindex = 68, retconf = 47.
getopt_long(1) loop. confval = 47.
GNU Wget 1.13.4a, a non-interactive network retriever.
[...]
With a short option, I get output like the following:
ALP $ wgx -h
Pre-getopt_long(1) loop. argc = 2, longindex = -1.
getopt_long(1) loop. longindex = -1, retconf = 104.
%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=0028
FFFC, PC=000E0E90, PS=001B
%TRACE-F-TRACEBACK, symbolic stack dump follows
imagemoduleroutine line rel PC abs PC
WGET MAIN no_prefix 24135 0E90 000E0E90
WGET MAIN 23537 000E0E90
WGET MAIN __main 24076 0A1C 000E0A1C
WGET 0 002839E0 002839E0
0 8037FCE4 8037FCE4
%TRACE-I-END, end of TRACE stack dump
Listing line 24135 is:
2 24135 confval = long_options[longindex].val;
Apparently, referencing long_options[-1] causes more trouble in a VMS
environment than it does on a typical UNIX(-like) system. It worked on
a Tru64 system, for example:
urtx# src/wget -h
Pre-getopt_long(1) loop. argc = 2, longindex = -1.
getopt_long(1) loop. longindex = -1, retconf = 104.
getopt_long(1) loop. confval = 0.
GNU Wget 1.13.4, a non-interactive network retriever.
[...]
But longindex is still -1 in this case, and this seems to me to be a
risky thing to use as a subscript in long_options[]. (But I'm always
open to a good counter-argument.)
Steven M. Schweda sms@antinode-info
382 South Warwick Street(+1) 651-699-9818
Saint Paul MN 55105-2547
[Bug-wget] Wget 1.13.4 v. VMS -- Various problems
It's still early, but here are the initial complaints... lib/snprintf.c now ignores HAVE_SNPRINTF. In previous wget versions, I could compile snprintf.c and not get a redundant snprintf() if HAVE_SNPRINTF was defined (%LINK-W-MULDEF, symbol DECC$TXSNPRINTF multiply defined). I can supply a conditional jacket for snprintf.c, but should I need to (now)? src/connect.c does #include sys/select.h and #include sys/socket.h with no regard for HAVE_SYS_SELECT_H or HAVE_SYS_SOCKET_H. I have no select.h (in sys or anywhere else), and was hoping that not defining HAVE_SYS_SELECT_H would have some effect. In src/init.c initialize(), the if (! ok) message thinks that it can print SYSTEM_WGETRC whether or not SYSTEM_WGETRC is defined. An old OpenSSL (0.9.8j) was too old. (1.0.0e worked better.) I haven't looked yet, but is the minimum required OpenSSL version documented anywhere? Or would some magic have happened if I had been using the auto-configure stuff? Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget for OpenVMS error
I am using OpenVMS( Version: V7.3-2) [...] So, that's on an Alpha? Could you please help me to resolve the error? I might be able to tell you something if I could see the actual command instead of the mess I see in the mailing list archive, but if you are actually specifying anything like https://ABC.com/*;, then you can expect trouble. Wildcards like * don't work with the HTTP[S] schemes, only with a scheme like FTP. (When you're talking to an FTP server, then you should be able to get a directory listing. The same is not true for an HTTP[S] server.) Wget does have a -r (--recursive) option, which might do what you want (but that will depend on what you want, and I don't know what that is). Apostrophes also mean something to DCL, so if you're really using those (as in --http-user='xxx'), then you could be in still more trouble. Have you tried using this wget program to do anything simpler than this? Did that work? Send some plain-text e-mail showing the actual wget command to the address below (more or less), and I'll see what I can do. Also: show symbol wget wget -V And, of course, with a host specification like abc.com, I can't do any testing, so you might also look into the -d (--debug) option if you want to see what the program is actually doing. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] new alpha tarball wget-1.12-2460.tar.bz2
From: Micah Cowan mi...@cowan.name Looks like it's still there, at least in the development sources repository. [...] That's a mercy. Good news is always appreciated. To get those, you need to use the Bazaar SCM's client (bzr); once you obtain that, you can use bzr branch http://bzr.savannah.gnu.org/r/wget/trunk to obtain the dev sources. It is not currently set up for inclusion in official source release distributions (I was probably waiting for a final version of the stuff); but that's easily fixable when everything's ready, by adding the vms/ dir's contents to the EXTRA_DIST variable in Makefile.am. Since these are dev sources, you'll have to have GNU automake and autoconf, and (now) git to obtain gnulib... if you have all the prereqs (not sure what the complete set is), then you run the bootstrap script. Every procedure which begins with once you obtain that is a problem around here, which is why life would be better in all ways if the VMS stuff were in the main stream, so that when a kit like this appeared, I might be able to do something with it directly. I probably could fire up some UNIX (-like) system, and start to pull on that GNU thread, which might eventually end in success, but it's the long way around in my environment. On the other hand, a quick look at this tar kit suggests that it's not really so ready-to-use as I had expected. (lib/getopt.in.h, for example, is littered with @s.) But it sounded good when I read the announcement. Who's the chicken and who's the egg in this situation? P.S.: I know that all the serious folks in the world have all the GNU infrastructure in place, but wouldn't a clever repository-access system be able to grind out a ready-to-use distribution kit upon user request? Just a thought. Steven M. Schweda sms@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Calling wget programmatically from C/C++ code,
From: Chitra Nanda Subject: [Bug-wget] Calling wget programmatically from C/C++ code I want to know if I can call wget programmatically from c/c++ program. I don't want to use system(wget). [...] Have you looked at cURL/libcurl? http://curl.haxx.se/ That would seem to be closer to what you want than wget. (And it seems to be reasonably portable.) As usual, you might get better advice if you revealed the actual problem which you are trying to solve, instead of asking how to implement a particular solution (like, say, callable wget) which may be sub-optimal (or nonexistent). Steven M. Schweda s...@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] collectd application
Any better way to do it? There are better ways to report the problem. You could start by identifying your wget program and your system type: wget --version Then you could supply a real test which someone else could run. the file.png was corrupt Corrupt _how_? Did you look at the file's contents? My psychic powers are too weak to tell me what's wrong with it, and collectd.example.com is not responding when I try to run your example command. Steven M. Schweda s...@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: [Bug-wget] Wget 1.12 v. VMS
From: Micah Cowan Could you also add one to cover whatever new work was done to src/vms.c? Forgot about those. --- src/ChangeLog;-12009-10-09 15:16:51 -0500 +++ src/ChangeLog 2009-10-09 17:10:42 -0500 @@ -3,6 +3,13 @@ * utils.c: Added characters : \ | to the list of normal caret-escaped (ODS5) characters in the char_prop[] table. + * vms.c: Added functions vms_basename(), used to extract the + bare executable name from argv[0], and vms_getpass(), a + VMS-specific replacement for GNU getpass(). + +vms.c: Added global string constants: compilation_string + (NULL), link_string (NULL), and version_string (real). + 2009-09-24 Micah Cowan mi...@cowan.name * vms.c: Moved from vms/vms.c. I'll investigate this. Thanks for pointing it out. Always happy to complain. SMS.
Re: [Bug-wget] Wget 1.12 v. VMS
I figured that I'd suck down a fresh source kit soon, and whip up some patches against the current stuff, and also assemble a kit of new VMS-specific files (source and builders) for incorporation. [...] http://antinode.info/ftp/wget/mainline-f5754f5d165c_v1.zip Starting with a normal source tree, delete the whole vms/ directory, and then overlay that Zip archive onto the tree. Contents: lib/DESCRIP_DEPS.MMSNew. lib/DESCRIP_MODS.MMSNew. lib/DESCRIP_SRC.MMS New. md5/DESCRIP_DEPS.MMSNew. md5/DESCRIP_MODS.MMSNew. md5/DESCRIP_SRC.MMS New. src/decc_ver.c Moved from vms/. src/DESCRIP_DEPS.MMSNew. src/DESCRIP_MODS.MMSNew. src/DESCRIP_SRC.MMS New. These two could be patched instead of replaced: src/main.c --version report changes. src/vms.c Misc. Most of these are greatly changed, so replacement might beat patching. (New header files lack copyrights, et c.): vms/alloca.h New. vms/COLLECT_DEPS.COM vms/config.h_vms vms/CONFIG_EXTRACT.COM New. vms/DESCRIP.MMS vms/DESCRIP_MKDEPS.MMS vms/DESCRIP_SRC.MMS Unchanged. vms/DESCRIP_SRC_CMN.MMS vms/DESCRIP_SRC_FLAGS.MMSUnchanged. vms/stdint.h New. vms/vms.h vms/vms_ip.h Unchanged. vms/VMS_NOTES.TXT vms/WGET.HLP vms/WGET_MULTINET.OPTUnchanged. vms/WGET_SSL_HP.OPT vms/WGET_SSL_O.OPT Changed, renamed from: vms/WGET_SSL.OPT. Should be gone now: vms/decc_ver.c Moved to src/. vms/DESCRIP_CONFIG.MMS Deleted. vms/DESCRIP_DEPS.MMS Deleted. vms/VMS-WGET.COM Deleted. vms/vms_name_fix.sh Deleted. vms/WGET.OPT Deleted. I'd also like to see if there's an easy way to auto-generate a VMS help file from the wget -h output, but that may be a while. That looks unlikely ever to happen. I did manually update vms/WGET.HLP, however. With these changes, a normal VMS user could almost fetch a 1.12.1-devel kit and build it, but not quite. If real src/build_info.c and src/css.c were in there, all would be easy. In the useless info department, I tried building the new code on some of my older VMS (VAX) versions, but anything before V7.0 looks problematic. I have a gap between V6.2 (which is too old) and V7.3 (which is ok), so I can't be sure just where the C RTL gets adequate, but it does look as if anything from the past decade has some chance of working. (V7.3 seems to have appeared around mid-2001.) SMS.
Re: [Bug-wget] Wget 1.12 v. VMS
From: Micah Cowan
Steven M. Schweda wrote:
7a. Lacking Unix-like builders, I stuck some place-holders for
compilation_string and link_string into src/vms.c:
Is there no equivalent that could be used in the VMS builders? If not,
the NULL check seems reasonable to me.
I know of nothing. The compiler options are revealed in a compiler
listing file, and the linker options may be revealed in a linker map
file, but none of that stuff would be easy to obtain, or, I claim, of
particular interest to the user.
And, speaking of build-related info, would it be useful on other
system types to include an indicator of large-file support somewhere in
this stuff? (Or has large-file support become so common that no one
else worries about it now?) A sizeof off_t test would be good enough
for me.
It can't hurt.
You say that now.
And, as the author of much sloppy code, I hate to complain -- Well, I
enjoy it, but I'm reluctant -- but in src/main.c:
[...]
/* defined in version.c */
extern char *version_string;
extern char *compilation_string;
extern char *system_getrc;
extern char *link_string;
/* defined in build_info.c */
extern char *compiled_features[];
[...]
Isn't this why the Great Sprit gave us header files? Especially, for
example, when the real compiled_features is declared differently
(src/build_info.c):
const char* (compiled_features[])
(And do those parentheses really do anything? My cdecl says no.)
At the moment, what I have looks like this:
[...]
extern const char *compiled_features[]; /* I fixed only this one. */
[...]
int i;
/* Changes begin. */
int j;
int line_length;
const char *runtime_features[] = { /, NULL, NULL };
const char **features[] = { compiled_features, runtime_features, NULL };
/* Fill runtime_features[]. Be sure to leave an array-terminating NULL. */
if (sizeof (off_t) = 8)
runtime_features[1] = +large-file;
else
runtime_features[1] = -large-file;
printf (_(GNU Wget %s built on %s.\n\n), version_string, OS_TYPE);
/* compiled_features is a char*[]. We limit the characters per
line to MAX_CHARS_PER_LINE and prefix each line with a constant
number of spaces for proper alignment. */
printf (_(Compile-time/Run-time features:\n));
line_length = MAX_CHARS_PER_LINE - TABULATION;
printf (%*c, TABULATION, ' ');
for (j = 0; features[j] != NULL; )
{
for (i = 0; features[j][i] != NULL; )
{
int len = strlen (features[j][i]) + 1;
line_length -= len;
if (line_length 0)
{
printf (\n%*c, TABULATION, ' ');
line_length = MAX_CHARS_PER_LINE - TABULATION - len;
}
printf (%s , features[j][i]);
i++;
}
j++;
}
printf (\n);
Here, the result looks like this:
ALP $ wgxl --version
GNU Wget 1.12 built on VMS Alpha V7.3-2.
Compile-time/Run-time features:
+digest +ipv6 -nls +ntlm +opie +md5/builtin +https -gnutls +openssl
-iri / +large-file
Wgetrc:
SYS$LOGIN:.wgetrc (user)
Copyright (C) 2009 Free Software Foundation, Inc.
[...]
I haven't checked it for portability problems (what could go wrong?)
but I claim that this code actually does limit the line length properly
(although I didn't trim that trailing space), and it really is indented,
as the comment suggests. It was so easy to toss in that / separator
between the two classes of items, that I couldn't resist, but it's easy
to remove, if desired. Lots of formatting changes could be made, of
course. And the new array could be relegated to some other backwater,
like src/build_info.c, or some new module.
I figured that I'd suck down a fresh source kit soon, and whip up
some patches against the current stuff, and also assemble a kit of new
VMS-specific files (source and builders) for incorporation. More VMS
builder changes may be needed, but I think that I have something usable
now, and it might be nice to get it into the mainstream, so that an
adventurous VMS user could try it. (I assume that a normal victim will
have trouble where I don't.)
I'd also like to see if there's an easy way to auto-generate a VMS
help file from the wget -h output, but that may be a while.
Opinions welcome, as always.
SMS.
Re: [Bug-wget] Wget 1.12 v. VMS
From: Micah Cowan (I didn't improve the vsnprintf() prototype, because it needs va_list, which not universally available. A realistic snprintf() seems to be harmless around here, however.) Not universally available? It's part of the C standard, since 1989. And it surely must exist on your platform, since if that #if clause is firing, then so is the one in snprintf.c, whose snprintf implementation uses vsnprintf (and therefore va_list), and apparently doesn't fail to compile. Poor phrasing on my part. src/sysdep.h didn't already pull in a declaration of va_list, so merely dropping in the realistic prototype for vsnprintf() caused more problems than it solved, and I didn't want to start tossing in #include directives to pull in stdarg.h or varargs.h or I_don't_know_what_else.h, which might not be portable. (But _you_ can.) I'll fix both (the va_list will require stdarg.h as well). Using a non-prototype function declaration to refer to a function that uses variable parameters is expressly forbidden by the C standard (though I suppose that would only apply to snprintf). Sounds good to me. 4. That VAX C compiler is also dismayed by the (const char *) type-cast in the dummy gettext() macro in lib/gettext.h and src/gettext.h. For example: [...] Actually, it looks like this is the type cast doing its job: according to comments in gettext.h, the cast is to prevent callers from trying to use gettext's return value inappropriately. host_errstr() should have been returning a const char*, which would make this message go away. I've fixed host_errstr()'s return type. I was afraid to try that, but it sounds plausible. Generally, if the (older, lamer) VAX compiler complains while the (newer, spiffier) Alpha and IA64 compilers don't, I just figure that it's a compiler problem, and start looking for a work-around. Is vms/getopt.h just copied from lib/getopt.in.h? I assumed it was, and made this change. The new builders have been copying lib/getopt.in.h to getopt.h (in a product directory, not vms/), and getting it from there. I could keep doing that. Or, if it's easy and not odious, and you'd like to supply it as vms/getopt.h, then I could rip out a few lines. Not having to deal with what happens to a multi-dot file name on an ODS2 disk would be ok with me, and I believe that that's the only one. So far, the builders seem to be working with MMS on VAX (V7.3) and Alpha (V7.3-2), with ODS2 and ODS5 disks. I need to try things on IA64, and with MMK instead of MMS, and with no SSL, HP SSL, and OpenSSL, and then on some older OS/compiler versions, but there seems to be some hope. (For 1.12.1 or something, anyway.) Cool. :) I'll say. I'm just starting to try to make the psychological adjustment to not having to host a separate VMS-suitable distribution kit myself. I hope that I can cope. There should still be a market for pre-fab VMS executable kits, so I don't have to go cold-turkey, but it'll be a jolt. SMS.
Re: [Bug-wget] Re: Steven Schweda's VMS patch in mainline
From: Keisial [...] Steven, note that all those headers are .h.in files, but have no m4 variables, so just copying them to the .h name works. Yup. I'm (slowly) waking up. I've been away from this stuff for too long. (And my brain is old and tired. (And I'm lazy.)) The VMS builders are complicated by the fact that a file name like xxx.h.in is too exotic for an ODS2 file system, and whether it gets stored as XXX.H_IN or XXX_H.IN (or who knows what else) depends on who unpacks the source kit. I intend to try to find an auto-sensing scheme to cope with the more likely possibilities, but it'll be a while. Everything's complicated. I am making a little progress, however. I'm reorganizing the VMS builders to deal with the more complex source tree structure. Not done, but I did get far enough to get to an interesting run-time message problem. Don't know if it's been noticed and/or fixed already, but: ALP $ wgxl --no-check-certificate https://alp-l --2009-09-21 23:24:34-- https://alp-l/ Resolving alp-l... 10.0.0.9 Connecting to alp-l|10.0.0.9|:443... connected. WARNING: cannot verify `/C=US/ST=MN/L=Saint Paul/O=Antinode/CN=antinode.info/ema iladdress=webmas...@antinode.info''s certificate, issued by `/C=US/ST=MN/L=Saint Paul/O=Antinode/CN=antinode.info/emailaddress=webmas...@antinode.info': Self-signed certificate encountered. WARNING: certificate common name `alp-l' doesn't match requested host name `alp- l'. HTTP request sent, awaiting response... 200 OK Length: 1009 [text/html] Saving to: `index.html' 100%[==] 1,009 --.-K/s in 0s 2009-09-21 23:24:35 (19.7 KB/s) - `index.html' saved [1009/1009] I thought that certificate common name `alp-l' matched requested host name `alp-l' pretty well, but it's the result of a misleading message. Actually, as a diagnostic printf() revealed: common_name: antinode.info, host: alp-l. I know nothing, but in src/openssl.c, this puts out the meassage: logprintf (LOG_NOTQUIET, _(\ %s: certificate common name %s doesn't match requested host name %s.\n), severity, quote (common_name), quote (host)); and I'm guessing that the fancy quote() function is insufficiently fancy, and is using the same buffer for both invocations in that statement, so the second name (host: alp-l) appears in both places. Should someone be auto-incrementing (modulo something) an n somewhere? Steven M. Schweda s...@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
[Bug-wget] Re: Steven Schweda's VMS patch in mainline
From: Micah Cowan mi...@cowan.name What more GNU environment do you need now, that you didn't before? For a starter, flex. (A fresher version of which solved those warning: implicit function declaration: complaints about css.c, by the way.) Development sources have always required automake/autoconf setups, that's not new. I guess you wrote the patch against the released 1.11 sources, so you may not have had to deal with that then, but... But everything's awful, and getting worse. I've grown to expect that. Neither autoconf nor automake will be required to build published sources, of course... I've been playing with the thought of automated distro-packaging for dev sources, so people can try out the development sources without having to install the extra GNU stuff... of course, actually developing on the sources will still require them, if one has to modify the build environment itself... That'd be better for me, of course. I woke up my IA64 system with some Debian thing on one of its disks, and that was less painful. Automatic detection of OpenSSL might be nice, too. I don't know how difficult that might be. If libiconv.so requires libgcc_s.so.1, it should probably have been added to the LIBICONV substitution variable. Maybe you can peek at the config.log to see why it apparently wasn't detected as necessary. I'll see if I can track it down when I get bored (unless the weather turns warm again). (Although, a powerful argument could be made for the proposition that any project which relies on a VMS guy for Solaris support might be considered somewhere between defective and deficient.) Steven M. Schweda s...@antinode-info 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
