RE: Large file support
From: Hrvoje Niksic [mailto:[EMAIL PROTECTED] Sent: Thursday, February 24, 2005 10:50 PM To: Maciej W. Rozycki Cc: Herold Heiko; wget@sunsite.dk Subject: Re: Large file support Maciej W. Rozycki [EMAIL PROTECTED] writes: Doesn't GCC work for this target? It does, in the form of Cygwin and MingW. But Heiko was using MS VC before, and we have catered to broken compilers before, so it doesn't hurt to try. Also, Cygwin requires a large installed environment. It may be possible to link statically and adapt everything in order to produce a single (large...) binary but that would be rather an ugly hack. MingW could be better, I really don't use it personally but I know the (standalone, portable) mame win32 binary is compiled with that. Nevertheless, Visual C++ is the de facto standard on the windows platforms, so supporting it if possible renders (self compiled) wget accessible to a larger audience. Now, if this means hacking the source too much I think the more recent vc++ only path shoule be tried first before throwing away vc++ support altogether. Said that, in retr.c simplifying the int rdsize line did not solve, but I tried the following, we have: #ifndef MIN # define MIN(i, j) ((i) = (j) ? (i) : (j)) #endif int rdsize = exact ? MIN (toread - sum_read, dlbufsize) : dlbufsize; double tmout = opt.read_timeout; Commenting out the double tmout... line removes the compiler error, OR exact ? (toread... (without MIN!) does compile, OR commenting out the #ifndef MIN..#endif does compile (in other words, MIN is already defined somewhere, how can I discover where?), however changing any occurence of MIN to XXXTEST still generates the compiler error, as Something simple like this int rdsize = 1; double tmout = opt.read_timeout; compiles, as does this int rdsize = dlbufsize; double tmout = opt.read_timeout; or this int rdsize = toread - sum_read; double tmout = opt.read_timeout; while this one int rdsize; rdsize = 1; double tmout = opt.read_timeout; fails with retr.c(263) : error C2143: syntax error : missing ';' before 'type' retr.c(269) : error C2065: 'tmout' : undeclared identifier (where line 263 is the double tmout=...). Any suggestions for other tests ? Heiko -- -- PREVINET S.p.A. www.previnet.it -- Heiko Herold [EMAIL PROTECTED] [EMAIL PROTECTED] -- +39-041-5907073 ph -- +39-041-5907472 fax
Re: Large file support
Herold Heiko [EMAIL PROTECTED] writes: Said that, in retr.c simplifying the int rdsize line did not solve, but I tried the following, we have: #ifndef MIN # define MIN(i, j) ((i) = (j) ? (i) : (j)) #endif int rdsize = exact ? MIN (toread - sum_read, dlbufsize) : dlbufsize; double tmout = opt.read_timeout; Commenting out the double tmout... line removes the compiler error, OR exact ? (toread... (without MIN!) does compile, OR commenting out the #ifndef MIN..#endif does compile (in other words, MIN is already defined somewhere, how can I discover where?), The whole point of #ifndef MIN ... #endif is to *use* the compiler-provided MIN where available. Dealing with a compiler crash is tricky, as it seems that every change has a potential of causing it to occur. Could you maybe tone down or remove the optimization flags when compiling retr.c?
RE: Large file support
On Fri, 25 Feb 2005, Herold Heiko wrote: Doesn't GCC work for this target? It does, in the form of Cygwin and MingW. But Heiko was using MS VC before, and we have catered to broken compilers before, so it doesn't hurt to try. Also, Cygwin requires a large installed environment. It may be possible to link statically and adapt everything in order to produce a single (large...) binary but that would be rather an ugly hack. MingW could be better, I really don't use it personally but I know the (standalone, portable) mame win32 binary is compiled with that. OK, thanks for the clarification. I'm not into non-*nix environments at all and with a *nix compiler you can normally drop in GCC as a replacement for the system C compiler without touching anything else, including the assembler, the linker and any libraries, be it shared or static (that's the point of having an ABI, after all). I'm not sure about other GCC languages, though -- they may require their own run-time libraries to be used instead of system ones; well, the Java compiler certainly does. ;-) Nevertheless, Visual C++ is the de facto standard on the windows platforms, so supporting it if possible renders (self compiled) wget accessible to a larger audience. Now, if this means hacking the source too much I think the more recent vc++ only path shoule be tried first before throwing away vc++ support altogether. Well, instead of scratching the head, how about filing a bug report? This way you should get a fix for the compiler and perhaps a description of a possible workaround for broken versions that we could use for wget. Commenting out the double tmout... line removes the compiler error, OR exact ? (toread... (without MIN!) does compile, OR commenting out the #ifndef MIN..#endif does compile (in other words, MIN is already defined somewhere, how can I discover where?), however changing any FYI, unless it's predefined, with GCC you'd use something like `gcc -E -dD foo.c' and search the output for the macro name. Maciej
RE: Large file support
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Maciej W. Rozycki Well, instead of scratching the head, how about filing a bug report? Ha :), would be nice. I suppose that would mean calling PSS, which (if things didn't change) means an immediate billing on your credit card (to be refunded later if there really was a problem). Anyway in this case the answer would probably be upgrade to VS.Net 2002. But see my other email... disabling optimization as Hrvoje suggested does work around the problem, and indeed I found some Microsoft articles suggesting exactly that. Heiko -- -- PREVINET S.p.A. www.previnet.it -- Heiko Herold [EMAIL PROTECTED] [EMAIL PROTECTED] -- +39-041-5907073 ph -- +39-041-5907472 fax
RE: Large file support
From: Hrvoje Niksic [mailto:[EMAIL PROTECTED] Sent: Friday, February 25, 2005 1:29 PM The whole point of #ifndef MIN ... #endif is to *use* the compiler-provided MIN where available. Dealing with a compiler crash is tricky, as it seems that every change has a potential of causing it to occur. Could you maybe tone down or remove the optimization flags when compiling retr.c? Does solve, in fact I found some MS articles suggesting the same thing. Attached patch does work around the problem by disabling optimization selectively.. I was able to retrieve a 2.5GB file with ftp. I tried to put a copy in the usual place (sunsite.dk by [EMAIL PROTECTED]), got autentication required - I suppose the password has changed. If still possible I'd like to put them there, for now a copy for general testing is at http://xoomer.virgilio.it/hherold/ Heiko -- -- PREVINET S.p.A. www.previnet.it -- Heiko Herold [EMAIL PROTECTED] [EMAIL PROTECTED] -- +39-041-5907073 ph -- +39-041-5907472 fax
RE: Large file support
On Fri, 25 Feb 2005, Herold Heiko wrote: Well, instead of scratching the head, how about filing a bug report? Ha :), would be nice. I suppose that would mean calling PSS, which (if things didn't change) means an immediate billing on your credit card (to be refunded later if there really was a problem). Huh? It's you who should actually charge them for doing bug discovery for them. Anyway in this case the answer would probably be upgrade to VS.Net 2002. I don't know how that actually relates to version 6 you've mentioned, but you should just obtain a bugfix release. But see my other email... disabling optimization as Hrvoje suggested does work around the problem, and indeed I found some Microsoft articles suggesting exactly that. Well, with the major version number of 6, you'd expect the product be already past its infancy, wouldn't you? Now I've heard some horror stories about C compilers crashing or producing bad code in early nineties (leading to many people avoiding any optimisation settings at all), but I've thought these times are over. Anyway, with that level of vendor (non-)support, you should really consider switching to GCC, no matter how much hassle with libraries it involves. It's just an effort to be done once and *will* pay back. Maciej
Re: Large file support
Herold Heiko [EMAIL PROTECTED] writes: Does solve, in fact I found some MS articles suggesting the same thing. Attached patch does work around the problem by disabling optimization selectively.. I was able to retrieve a 2.5GB file with ftp. In other words, large files now work on Windows? I must admit, that was almost too easy. :-) Now could someone try this with Borland and/or Watcom and MingW? I'm pretty sure I broke them in some places, but it's near impossible to fix it without having the compilers available for testing. I tried to put a copy in the usual place (sunsite.dk by [EMAIL PROTECTED]), got autentication required - I suppose the password has changed. If still possible I'd like to put them there, Mauro, are you aware of the password change?
O_EXCL under Windows?
Is there a way to get the functionality of open(..., O_CREAT|O_EXCL) under Windows? For those who don't know, O_EXCL opens the file exclusively, guaranteeing that the file we're opening will not be overwritten. (Note that it's not enough to check that the file doesn't exist before opening it; it can spring into existence between the check and the open.) I cannot find a way to do the equivalent on Windows. Does O_EXCL just work on Windows? A search for O_EXCL on MSDN doesn't reveal any results.
Re: O_EXCL under Windows?
Hrvoje Niksic wrote: Is there a way to get the functionality of open(..., O_CREAT|O_EXCL) under Windows? For those who don't know, O_EXCL opens the file exclusively, guaranteeing that the file we're opening will not be overwritten. (Note that it's not enough to check that the file doesn't exist before opening it; it can spring into existence between the check and the open.) This works with MingW and MSVC. Watcom acts a bit odd if you use fdopen (fd, w+) afterwards. A snippet from my contribution to libnet (works across processes of course): #include fcntl.h #include share.h #include io.h int open_flags = O_WRONLY | O_CREAT | O_TRUNC; /* possibly drop the O_WRONLY for r/w */ int fd = sopen (file_name, open_flags | O_BINARY | _O_SEQUENTIAL, SH_DENYWR, S_IREAD | S_IWRITE); _O_SEQUENTIAL is just to tell the cache manager to stop wasting memory. --gv
