On 12/1/19 5:12 PM, Ken Moffat via blfs-dev wrote:
This long epistle was going to be a note that Archive::Zip fails one of its tests in my build of LFS and BLFS from last night, but I now think that our build of unzip has been broken (for odd zip archives which include a file compressed with bzip2) since at least BLFS-7.8 and perhaps longer.The Archive::Zip tests reported - t/25_traversal.t .............. ok # Failed test 'testZip ok after t/data/bzip.zip to /tmp/testout-T1RY5.zip' # at t/26_bzip2.t line 33. # got: '20736' # expected: '0' ziptest said: Archive: /tmp/testout-T1RY5.zip skipping: README need PK compat. v4.6 (can do v4.5) Caution: zero files tested in /tmp/testout-T1RY5.zip. 1 file skipped because of unsupported compression or encoding. t/26_bzip2.t .................. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests This is new after 1.64 version, there seem to be a number of reports. https://github.com/redhotpenguin/perl-Archive-Zip/issues/70 mentions this on various platforms, and apparently various cpantest builders are reporting failures. Aggregate issue is https://github.com/redhotpenguin/perl-Archive-Zip/issues/67 but that has not been updated for a month. And the implication is that something in our environment might be odd. I now agree, our unzip cannot extract archives containing a file compressed with bzip2. The failing test was added in September. Its heading says # This test uses an archive, bzip.zip, that contains a member that uses bzip2 compression. # The test is checking that the bzip2 member will pass-through to a new zip file without # causing corruption. # Before this fix when you ran "unzip -t" on the newly created archive file it would report # that the fip zipe was corrupted. # # See https://github.com/redhotpenguin/perl-Archive-Zip/issues/26 for more details. From metacpan testers, 1006 pass, 403 fail. Two linux failures on 5.30.0 http://matrix.cpantesters.org/?dist=Archive-Zip%201.67;os=linux;perl=5.30.0;reports=1 Both on recent x86_64, the first report seems to fail in many tests including this one, the second (from Arch) seems to only fail this one. Not sure at this point if the failure is reproducible by other BLFS users. At the moment I have not found where the 'Need PK Compat' message comes from, but google suggests it might be coming from unzip. Ah, maybe my build of unzip is borked - sh unix/configure "cc" "-I. -Ibzip2 -DUNIX " "bzip2" [...] Check for bzip2 sources in unzip's bzip2 subdirectory -- bzip2 sources not found - no bzip2 support All my builds of unzip going back to BLFS-7.8 seem to lack bzip2 support according to the logs. Before that we apparently built unzip slightly differently and my logs are not verbose, with no indication whether or not bzip2 was picked up (and those old systems are long gone). Maybe this did not matter until Archive-Zip added this test. I see that gentoo are using debian patches for unzip (apparently for security), but also passing -DUSE_BZIP2 in CPPFLAGS but I can't get that to work. Looking at fedora, they have a stack of patches for various CVE fixes, as well as for non UTF-8 encodings. Their changelog in the specfile says * Mon Dec 10 2012 Michal Luscon <[email protected]> 6.0-7 - Resolves: #884679 - zip files with bzip2 compression https://bugzilla.redhat.com/show_bug.cgi?id=884679 They had a configure patch, and apparently an error crept in while rebasing it from a previous version. Summary of that bug report: | Today I encountered a zip which uses bzip2 compression and unzip | could not handle it. A quick look into the package shows that likely | a mistake snook in while rebasing the unzip-6.0-bzip2-configure.patch | from a previous unzip version. Looking at Arch, they add some patches and they add -DUSE_BZIP2 but apparently that did not work for the cpan tester. So, it seems that there are three sets of deficiencies in our build: 1. Files with non-ASCII filenames : we recommend using WinZip under Wine, or unpacking bsdtar and then using convmv. I guess that fedora's many patches perhaps cover this, but does anyone have an example zip file they can share for testing ? 2. A plethora of CVE and related security fixes. 3. Not supporting members of a zip archive which have been compressed with bzip2. I'm inclined to look at taking all of fedora's patches. The configure patch is just one line to change but in all there are 25 of them totalling 94KB. I'll list the details from fedora's specfile - # Not sent to upstream. Patch1: unzip-6.0-bzip2-configure.patch # Upstream plans to do this in zip (hopefully also in unzip). Patch2: unzip-6.0-exec-shield.patch # Upstream plans to do similar thing. Patch3: unzip-6.0-close.patch # Details in rhbz#532380. # Reported to upstream: http://www.info-zip.org/board/board.pl?m-1259575993/ Patch4: unzip-6.0-attribs-overflow.patch # Not sent to upstream, as it's Fedora/RHEL specific. # Modify the configure script to accept var LFLAGS2 so linking can be configurable # from the spec file. In addition '-s' is still removed as before Patch5: unzip-6.0-configure.patch Patch6: unzip-6.0-manpage-fix.patch # Update match.c with recmatch() from zip 3.0's util.c # This also resolves the license issue in that old function. # Original came from here: https://projects.parabolagnulinux.org/abslibre.git/plain/libre/unzip-libre/match.patch Patch7: unzip-6.0-fix-recmatch.patch # Update process.c Patch8: unzip-6.0-symlink.patch # change using of macro "case_map" by "to_up" Patch9: unzip-6.0-caseinsensitive.patch # downstream fix for "-Werror=format-security" # upstream doesn't want hear about this option again Patch10: unzip-6.0-format-secure.patch Patch11: unzip-6.0-valgrind.patch Patch12: unzip-6.0-x-option.patch Patch13: unzip-6.0-overflow.patch Patch14: unzip-6.0-cve-2014-8139.patch Patch15: unzip-6.0-cve-2014-8140.patch Patch16: unzip-6.0-cve-2014-8141.patch Patch17: unzip-6.0-overflow-long-fsize.patch # Fix heap overflow and infinite loop when invalid input is given (#1260947) Patch18: unzip-6.0-heap-overflow-infloop.patch # support non-{latin,unicode} encoding Patch19: unzip-6.0-alt-iconv-utf8.patch Patch20: unzip-6.0-alt-iconv-utf8-print.patch Patch21: 0001-Fix-CVE-2016-9844-rhbz-1404283.patch # restore unix timestamp accurately Patch22: unzip-6.0-timestamp.patch # fix possible heap based stack overflow in passwd protected files Patch23: unzip-6.0-cve-2018-1000035-heap-based-overflow.patch Patch24: unzip-6.0-cve-2018-18384.patch # covscan issues Patch25: unzip-6.0-COVSCAN-fix-unterminated-string.patch Clearly, patches 19-21 attempt to fix non-ASCII filenames. And FWIW their rpm build is # IZ_HAVE_UXUIDGID is needed for right functionality of unzip -X # NOMEMCPY solve problem with memory overlapping - decomression is slowly, # but successfull. make -f unix/Makefile CF_NOOPT="-I. -DUNIX $RPM_OPT_FLAGS -DNOMEMCPY -DIZ_HAVE_UXUIDGID -DNO_LCHMOD" \ LFLAGS2="%{?__global_ldflags}" generic_gcc %{?_smp_mflags} But I'm unconvinced that we need to worry about unzip -X, and I'm not sure about the non-ASCII filenames (in the absence of test data).
Wow. That's a lot of things for a package we rarely use. I'd say that if you can use Fedora's patches and then the archive-zip test passes, than that's good enough for us.
-- Bruce -- http://lists.linuxfromscratch.org/listinfo/blfs-dev FAQ: http://www.linuxfromscratch.org/blfs/faq.html Unsubscribe: See the above information page
