portsnap oddities
Hello all, I'm getting the following error when I use portsnap lately on one of my servers: [mesh:/home/gary]# portsnap fetch Looking up portsnap.FreeBSD.org mirrors... 2 mirrors found. Fetching snapshot tag from portsnap2.FreeBSD.org... done. Fetching snapshot metadata... done. Updating from Thu 3 Aug 2006 04:45:32 EST to Sat 5 Aug 2006 11:06:54 EST. Fetching 0 metadata patches. done. Applying metadata patches... done. Fetching 0 metadata files... done. Fetching 2 patches.. done. Applying patches... done. Fetching 2 new ports or files... gunzip: stdin: not in gzip format snapshot is corrupt. I have now removed /var/db/portsnap and started from scratch 3 times. This fixes the problem, but a few days later, I get the same story. I haven't had any problems on any of the other servers (although they have older kernels), so I'm guessing this could be a problem with portsnap from this kernel build or it's related to something else. FreeBSD mesh.lhshoses.com.au 6.1-STABLE FreeBSD 6.1-STABLE #0: Fri Jul 28 13:29:26 EST 2006 root@:/usr/obj/usr/src/sys/MESH i386 Anyone got any ideas? Thanks Gary ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap oddities
Gary Newcombe wrote: I'm getting the following error when I use portsnap lately on one of my servers: [snip] Fetching 2 new ports or files... gunzip: stdin: not in gzip format snapshot is corrupt. I have now removed /var/db/portsnap and started from scratch 3 times. This fixes the problem, but a few days later, I get the same story. I haven't had any problems on any of the other servers (although they have older kernels), so I'm guessing this could be a problem with portsnap from this kernel build or it's related to something else. First, the obvious thing to check: Are you running out of disk space on /var ? Second, please run 'portsnap --debug fetch' and send me the output; this will give me a chance of identifying the problem. Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: portsnap oddities
Sure, the /var slice is 2Gb and has plenty of room. [mesh:/home/gary]# portsnap --debug fetch Looking up portsnap.FreeBSD.org mirrors... 2 mirrors found. Fetching snapshot tag from portsnap1.FreeBSD.org... latest.ssl100% of 256 B 512 kBps done. Fetching snapshot metadata... 6eb9d6e9f93dd6b8e2a88bdf9d394b7e77f75795f073a4100% of 299 B 650 kBps done. Updating from Thu 3 Aug 2006 04:45:32 EST to Sat 5 Aug 2006 11:06:54 EST. Fetching 0 metadata patches... done. Applying metadata patches... done. Fetching 0 metadata files... done. Fetching 2 patches... /usr/libexec/phttpget portsnap1.FreeBSD.org bp/7c2d57a2388d4d5cd20e935c57727b5019fbdf06210ebf9b8f0b7c01bf072db5-ad06d1f7 b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd bp/352d16ab1731729d4542c9c126034d6f27ce2830f297effb8831e6eb6a46cb31-ad3d5100 1a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3 http://portsnap1.FreeBSD.org/bp/7c2d57a2388d4d5cd20e935c57727b5019fbdf06210e bf9b8f0b7c01bf072db5-ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d 059f1b8fd: 200 OK http://portsnap1.FreeBSD.org/bp/352d16ab1731729d4542c9c126034d6f27ce2830f297 effb8831e6eb6a46cb31-ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee83799 48a44aae3: 200 OK done. Applying patches... bspatch: Corrupt patch sha256: NEW: No such file or directory [: =: unexpected operator bspatch: Corrupt patch sha256: NEW: No such file or directory [: =: unexpected operator done. Fetching 2 new ports or files... /usr/libexec/phttpget portsnap1.FreeBSD.org f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz f/ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz http://portsnap1.FreeBSD.org/f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166 564e61666d059f1b8fd.gz: 200 OK http://portsnap1.FreeBSD.org/f/ad3d51001a264245eab5894cece6c902d073841143e9f fc7ee8379948a44aae3.gz: 200 OK gunzip: stdin: not in gzip format snapshot is corrupt. Thanks Colin, Gary -Original Message- From: Colin Percival [mailto:[EMAIL PROTECTED] Sent: Saturday, 5 August 2006 12:21 PM To: Gary Newcombe Cc: freebsd-questions@freebsd.org Subject: Re: portsnap oddities Gary Newcombe wrote: I'm getting the following error when I use portsnap lately on one of my servers: [snip] Fetching 2 new ports or files... gunzip: stdin: not in gzip format snapshot is corrupt. I have now removed /var/db/portsnap and started from scratch 3 times. This fixes the problem, but a few days later, I get the same story. I haven't had any problems on any of the other servers (although they have older kernels), so I'm guessing this could be a problem with portsnap from this kernel build or it's related to something else. First, the obvious thing to check: Are you running out of disk space on /var ? Second, please run 'portsnap --debug fetch' and send me the output; this will give me a chance of identifying the problem. Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap oddities
Gary Newcombe wrote: Fetching 2 new ports or files... /usr/libexec/phttpget portsnap1.FreeBSD.org f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz f/ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz http://portsnap1.FreeBSD.org/f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166 564e61666d059f1b8fd.gz: 200 OK http://portsnap1.FreeBSD.org/f/ad3d51001a264245eab5894cece6c902d073841143e9f fc7ee8379948a44aae3.gz: 200 OK gunzip: stdin: not in gzip format snapshot is corrupt. Strange. I've checked on portsnap1.freebsd.org, and those files are definitely intact. Are you using an HTTP proxy? It's possible that it might have cached a broken version of those files. Could you look in /var/db/portsnap and tell me how large those two files are? Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: portsnap oddities
Yes, nail on the head methinks. This server is behind a proxy and portsnap works fine with it disabled. With combination of advproxy, havp and privoxy: [mesh:/var/db/portsnap]# l *[3d].gz -rw-r--r-- 1 root wheel64B 5 Aug 12:51 ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz -rw-r--r-- 1 root wheel64B 5 Aug 12:51 ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz Without: [mesh:/var/db/portsnap]# portsnap --debug fetch Looking up portsnap.FreeBSD.org mirrors... 2 mirrors found. Fetching snapshot tag from portsnap1.FreeBSD.org... latest.ssl100% of 256 B 685 kBps done. Fetching snapshot metadata... d82061f1c680d235d7c08c340e9c25e42b4a133e2ca1c1100% of 299 B 533 kBps done. Updating from Thu 3 Aug 2006 04:45:32 EST to Sat 5 Aug 2006 11:22:38 EST. Fetching 0 metadata patches... done. Applying metadata patches... done. Fetching 0 metadata files... done. Fetching 2 patches... /usr/libexec/phttpget portsnap1.FreeBSD.org bp/7c2d57a2388d4d5cd20e935c57727b5019fbdf06210ebf9b8f0b7c01bf072db5-ad06d1f7 b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd bp/352d16ab1731729d4542c9c126034d6f27ce2830f297effb8831e6eb6a46cb31-ad3d5100 1a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3 http://portsnap1.FreeBSD.org/bp/7c2d57a2388d4d5cd20e935c57727b5019fbdf06210e bf9b8f0b7c01bf072db5-ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d 059f1b8fd: 200 OK http://portsnap1.FreeBSD.org/bp/352d16ab1731729d4542c9c126034d6f27ce2830f297 effb8831e6eb6a46cb31-ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee83799 48a44aae3: 200 OK done. Applying patches... done. Fetching 0 new ports or files... done. [mesh:/var/db/portsnap]# l *[3d].gz -rw-r--r-- 1 root wheel64B 5 Aug 13:32 ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz -rw-r--r-- 1 root wheel64B 5 Aug 13:32 ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz So the files did seem to be intact initially anyway? Just clearing the cache for the proxy didn't seem to solve the problem btw. Thanks, Gary -Original Message- From: Colin Percival [mailto:[EMAIL PROTECTED] Sent: Saturday, 5 August 2006 1:24 PM To: Gary Newcombe Cc: freebsd-questions@freebsd.org Subject: Re: portsnap oddities Gary Newcombe wrote: Fetching 2 new ports or files... /usr/libexec/phttpget portsnap1.FreeBSD.org f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz f/ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz http://portsnap1.FreeBSD.org/f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166 564e61666d059f1b8fd.gz: 200 OK http://portsnap1.FreeBSD.org/f/ad3d51001a264245eab5894cece6c902d073841143e9f fc7ee8379948a44aae3.gz: 200 OK gunzip: stdin: not in gzip format snapshot is corrupt. Strange. I've checked on portsnap1.freebsd.org, and those files are definitely intact. Are you using an HTTP proxy? It's possible that it might have cached a broken version of those files. Could you look in /var/db/portsnap and tell me how large those two files are? Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap oddities
I might just add that I had similar corruption issues using portsnap behind a transparent squid proxy. Not 100% of the time corruption, but fairly often. Changed the router rules to not proxy for this server and all has been well. Scott On Aug 4, 2006, at 11:49 PM, Gary Newcombe wrote: Yes, nail on the head methinks. This server is behind a proxy and portsnap works fine with it disabled. With combination of advproxy, havp and privoxy: [mesh:/var/db/portsnap]# l *[3d].gz -rw-r--r-- 1 root wheel64B 5 Aug 12:51 ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz -rw-r--r-- 1 root wheel64B 5 Aug 12:51 ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz Without: [mesh:/var/db/portsnap]# portsnap --debug fetch Looking up portsnap.FreeBSD.org mirrors... 2 mirrors found. Fetching snapshot tag from portsnap1.FreeBSD.org... latest.ssl100% of 256 B 685 kBps done. Fetching snapshot metadata... d82061f1c680d235d7c08c340e9c25e42b4a133e2ca1c1100% of 299 B 533 kBps done. Updating from Thu 3 Aug 2006 04:45:32 EST to Sat 5 Aug 2006 11:22:38 EST. Fetching 0 metadata patches... done. Applying metadata patches... done. Fetching 0 metadata files... done. Fetching 2 patches... /usr/libexec/phttpget portsnap1.FreeBSD.org bp/7c2d57a2388d4d5cd20e935c57727b5019fbdf06210ebf9b8f0b7c01bf072db5- ad06d1f7 b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd bp/352d16ab1731729d4542c9c126034d6f27ce2830f297effb8831e6eb6a46cb31- ad3d5100 1a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3 http://portsnap1.FreeBSD.org/bp/ 7c2d57a2388d4d5cd20e935c57727b5019fbdf06210e bf9b8f0b7c01bf072db5- ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d 059f1b8fd: 200 OK http://portsnap1.FreeBSD.org/bp/ 352d16ab1731729d4542c9c126034d6f27ce2830f297 effb8831e6eb6a46cb31- ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee83799 48a44aae3: 200 OK done. Applying patches... done. Fetching 0 new ports or files... done. [mesh:/var/db/portsnap]# l *[3d].gz -rw-r--r-- 1 root wheel64B 5 Aug 13:32 ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz -rw-r--r-- 1 root wheel64B 5 Aug 13:32 ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz So the files did seem to be intact initially anyway? Just clearing the cache for the proxy didn't seem to solve the problem btw. Thanks, Gary -Original Message- From: Colin Percival [mailto:[EMAIL PROTECTED] Sent: Saturday, 5 August 2006 1:24 PM To: Gary Newcombe Cc: freebsd-questions@freebsd.org Subject: Re: portsnap oddities Gary Newcombe wrote: Fetching 2 new ports or files... /usr/libexec/phttpget portsnap1.FreeBSD.org f/ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz f/ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz http://portsnap1.FreeBSD.org/f/ ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166 564e61666d059f1b8fd.gz: 200 OK http://portsnap1.FreeBSD.org/f/ ad3d51001a264245eab5894cece6c902d073841143e9f fc7ee8379948a44aae3.gz: 200 OK gunzip: stdin: not in gzip format snapshot is corrupt. Strange. I've checked on portsnap1.freebsd.org, and those files are definitely intact. Are you using an HTTP proxy? It's possible that it might have cached a broken version of those files. Could you look in /var/db/ portsnap and tell me how large those two files are? Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions- [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: portsnap oddities
Gary Newcombe wrote: Yes, nail on the head methinks. This server is behind a proxy and portsnap works fine with it disabled. With combination of advproxy, havp and privoxy: [mesh:/var/db/portsnap]# l *[3d].gz -rw-r--r-- 1 root wheel64B 5 Aug 12:51 ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz -rw-r--r-- 1 root wheel64B 5 Aug 12:51 ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz Without: [...] Fetching 2 patches... [...] done. Applying patches... done. Fetching 0 new ports or files... done. [mesh:/var/db/portsnap]# l *[3d].gz -rw-r--r-- 1 root wheel64B 5 Aug 13:32 ad06d1f7b82db9ebcb496e7d48a754932622f1c8d6166564e61666d059f1b8fd.gz -rw-r--r-- 1 root wheel64B 5 Aug 13:32 ad3d51001a264245eab5894cece6c902d073841143e9ffc7ee8379948a44aae3.gz So the files did seem to be intact initially anyway? Everything seems to be working fine now. You can delete those two files; they were left behind because I forgot to handle the case of 'file download failed, portsnap gets run again, and then patch download succeeds'. The correct versions of the files are stored in the /var/db/portsnap/files/ directory. Just clearing the cache for the proxy didn't seem to solve the problem btw. It's possible that your cache gets confused by pipelined HTTP. It wouldn't be the first time that has happened... Colin Percival ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
