Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package wimlib for openSUSE:Factory checked in at 2021-07-05 22:23:18 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/wimlib (Old) and /work/SRC/openSUSE:Factory/.wimlib.new.2625 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "wimlib" Mon Jul 5 22:23:18 2021 rev:5 rq:904132 version:1.13.4 Changes: -------- --- /work/SRC/openSUSE:Factory/wimlib/wimlib.changes 2020-12-12 20:32:47.129878404 +0100 +++ /work/SRC/openSUSE:Factory/.wimlib.new.2625/wimlib.changes 2021-07-05 22:24:05.721204085 +0200 @@ -1,0 +2,13 @@ +Mon Jul 5 07:36:38 UTC 2021 - Dirk M??ller <dmuel...@suse.com> + +- update to 1.13.4: + * wimsplit now prints progress messages regularly rather than just once + per WIM part. + + * Added support for a data recovery mode which causes files to be + extracted even if they are corrupted. The option is --recover-data for + wimapply and wimextract, and WIMLIB_EXTRACT_FLAG_RECOVER_DATA for the + library. Note that this option won't help with all types of corruption; + some types of corruption will still cause a fatal error. + +------------------------------------------------------------------- Old: ---- wimlib-1.13.3.tar.gz New: ---- wimlib-1.13.4.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ wimlib.spec ++++++ --- /var/tmp/diff_new_pack.AMR21n/_old 2021-07-05 22:24:06.213200279 +0200 +++ /var/tmp/diff_new_pack.AMR21n/_new 2021-07-05 22:24:06.213200279 +0200 @@ -1,7 +1,7 @@ # # spec file for package wimlib # -# Copyright (c) 2020 SUSE LLC +# Copyright (c) 2021 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -18,10 +18,10 @@ %define so_version 15 Name: wimlib -Version: 1.13.3 +Version: 1.13.4 Release: 0 Summary: Library to extract, create, modify, and mount WIM files -License: GPL-3.0-or-later AND LGPL-3.0-or-later AND CC0-1.0 +License: CC0-1.0 AND GPL-3.0-or-later AND LGPL-3.0-or-later Group: Development/Libraries/C and C++ URL: https://wimlib.net Source: https://wimlib.net/downloads/wimlib-%{version}.tar.gz ++++++ wimlib-1.13.3.tar.gz -> wimlib-1.13.4.tar.gz ++++++ ++++ 13040 lines of diff (skipped) ++++ retrying with extended exclude list diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/Makefile.am new/wimlib-1.13.4/Makefile.am --- old/wimlib-1.13.3/Makefile.am 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/Makefile.am 2021-04-19 06:35:01.000000000 +0200 @@ -200,7 +200,7 @@ $(LIBFUSE_CFLAGS) \ $(LIBCRYPTO_CFLAGS) -libwim_la_LDFLAGS = $(AM_LDFLAGS) -version-info 33:0:18 +libwim_la_LDFLAGS = $(AM_LDFLAGS) -version-info 34:0:19 libwim_la_LIBADD = \ $(PTHREAD_LIBS) \ @@ -340,7 +340,8 @@ tests/security_descriptor_1.base64 \ tests/security_descriptor_1.bin \ tests/security_descriptor_2.base64 \ - tests/security_descriptor_2.bin + tests/security_descriptor_2.bin \ + tests/wims if WINDOWS_NATIVE_BUILD # Tests are run manually for Windows builds. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/NEWS new/wimlib-1.13.4/NEWS --- old/wimlib-1.13.3/NEWS 2020-10-27 04:53:59.000000000 +0100 +++ new/wimlib-1.13.4/NEWS 2021-04-19 06:35:01.000000000 +0200 @@ -1,3 +1,13 @@ +Version 1.13.4: + wimsplit now prints progress messages regularly rather than just once + per WIM part. + + Added support for a data recovery mode which causes files to be + extracted even if they are corrupted. The option is --recover-data for + wimapply and wimextract, and WIMLIB_EXTRACT_FLAG_RECOVER_DATA for the + library. Note that this option won't help with all types of corruption; + some types of corruption will still cause a fatal error. + Version 1.13.3: On Windows, improved performance of capturing an entire drive in some cases. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/README new/wimlib-1.13.4/README --- old/wimlib-1.13.3/README 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/README 2021-04-19 06:35:01.000000000 +0200 @@ -1,6 +1,6 @@ INTRODUCTION -This is wimlib version 1.13.3 (October 2020). wimlib is a C library for +This is wimlib version 1.13.4 (April 2021). wimlib is a C library for creating, modifying, extracting, and mounting files in the Windows Imaging Format (WIM files). wimlib and its command-line frontend 'wimlib-imagex' provide a free and cross-platform alternative to Microsoft's WIMGAPI, ImageX, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/config.h.in new/wimlib-1.13.4/config.h.in --- old/wimlib-1.13.3/config.h.in 2020-10-27 04:54:51.000000000 +0100 +++ new/wimlib-1.13.4/config.h.in 2021-04-19 06:35:44.000000000 +0200 @@ -72,9 +72,6 @@ /* Define to 1 if you have the <machine/endian.h> header file. */ #undef HAVE_MACHINE_ENDIAN_H -/* Define to 1 if you have the <memory.h> header file. */ -#undef HAVE_MEMORY_H - /* Define to 1 if you have the `mempcpy' function. */ #undef HAVE_MEMPCPY @@ -109,6 +106,9 @@ /* Define to 1 if you have the <stdint.h> header file. */ #undef HAVE_STDINT_H +/* Define to 1 if you have the <stdio.h> header file. */ +#undef HAVE_STDIO_H + /* Define to 1 if you have the <stdlib.h> header file. */ #undef HAVE_STDLIB_H @@ -185,7 +185,9 @@ your system. */ #undef PTHREAD_CREATE_JOINABLE -/* Define to 1 if you have the ANSI C header files. */ +/* Define to 1 if all of the C90 standard headers exist (not just the ones + required in a freestanding environment). This macro is provided for + backward compatibility; new code need not use it. */ #undef STDC_HEADERS /* Version number of package */ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/configure.ac new/wimlib-1.13.4/configure.ac --- old/wimlib-1.13.3/configure.ac 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/configure.ac 2021-04-19 06:35:01.000000000 +0200 @@ -1,6 +1,6 @@ ############################################################################### -AC_INIT([wimlib], [1.13.3], [https://wimlib.net/forums/]) +AC_INIT([wimlib], [1.13.4], [https://wimlib.net/forums/]) AC_CONFIG_SRCDIR([src/wim.c]) AC_CONFIG_MACRO_DIR([m4]) AC_CONFIG_AUX_DIR([build-aux]) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/mkwinpeimg.1 new/wimlib-1.13.4/doc/man1/mkwinpeimg.1 --- old/wimlib-1.13.3/doc/man1/mkwinpeimg.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/mkwinpeimg.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH MKWINPEIMG "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH MKWINPEIMG "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME mkwinpeimg \- Make a customized bootable image of Windows PE .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimapply.1 new/wimlib-1.13.4/doc/man1/wimapply.1 --- old/wimlib-1.13.3/doc/man1/wimapply.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimapply.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMAPPLY "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMAPPLY "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimapply \- Apply a WIM image .SH SYNOPSIS @@ -355,15 +355,22 @@ In addition, wimlib has a hardcoded list of files for which it knows, for compatibility with the Windows bootloader, to override the requested compression format. +.TP +\fB--recover-data\fR +If a file is corrupted (its stored hash doesn't match its actual hash, or some +parts of it can't be decompressed), extract the corrupted file anyway with a +warning, rather than aborting with an error. This may be useful to recover data +if a WIM archive was corrupted. Note that recovering data is not guaranteed to +succeed, as it depends on the type of corruption that occurred. .SH NOTES \fIData integrity\fR: WIM files include checksums of file data. To detect accidental (non-malicious) data corruption, wimlib calculates the checksum of every file it extracts and issues an error if it does not have the expected -value. (This default behavior seems equivalent to the \fB/verify\fR option of -ImageX.) In addition, a WIM file can include an integrity table (extra -checksums) over the raw data of the entire WIM file. For performance reasons -wimlib does not check the integrity table by default, but the \fB--check\fR -option can be passed to make it do so. +value, unless the \fB--recover-data\fR option is given. (This default behavior +seems equivalent to the \fB/verify\fR option of ImageX.) In addition, a WIM +file can include an integrity table (extra checksums) over the raw data of the +entire WIM file. For performance reasons wimlib does not check the integrity +table by default, but the \fB--check\fR option can be passed to make it do so. .PP \fIESD files\fR: wimlib can extract files from solid-compressed WIMs, or "ESD" (.esd) files, just like from normal WIM (.wim) files. However, Microsoft diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimcapture.1 new/wimlib-1.13.4/doc/man1/wimcapture.1 --- old/wimlib-1.13.3/doc/man1/wimcapture.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimcapture.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMCAPTURE "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMCAPTURE "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimcapture, wimappend \- Capture or append a WIM image .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimdelete.1 new/wimlib-1.13.4/doc/man1/wimdelete.1 --- old/wimlib-1.13.3/doc/man1/wimdelete.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimdelete.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMDELETE "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMDELETE "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimdelete \- Delete an image from a WIM archive .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimdir.1 new/wimlib-1.13.4/doc/man1/wimdir.1 --- old/wimlib-1.13.3/doc/man1/wimdir.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimdir.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMDIR "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMDIR "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimdir \- List the files contained in a WIM image .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimexport.1 new/wimlib-1.13.4/doc/man1/wimexport.1 --- old/wimlib-1.13.3/doc/man1/wimexport.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimexport.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMEXPORT "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMEXPORT "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimexport \- Export image(s) from a WIM archive .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimextract.1 new/wimlib-1.13.4/doc/man1/wimextract.1 --- old/wimlib-1.13.3/doc/man1/wimextract.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimextract.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMEXTRACT "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMEXTRACT "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimextract \- Extract files from a WIM image .SH SYNOPSIS @@ -152,6 +152,9 @@ .TP \fB--compact\fR=\fIFORMAT\fR See the documentation for this option to \fBwimapply\fR(1). +.TP +\fB--recover-data\fR +See the documentation for this option to \fBwimapply\fR(1). .SH NOTES See \fBwimapply\fR(1) for information about what data and metadata are extracted on UNIX-like systems versus on Windows. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wiminfo.1 new/wimlib-1.13.4/doc/man1/wiminfo.1 --- old/wimlib-1.13.3/doc/man1/wiminfo.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wiminfo.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMINFO "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMINFO "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wiminfo \- Display or change information about a WIM file or image .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimjoin.1 new/wimlib-1.13.4/doc/man1/wimjoin.1 --- old/wimlib-1.13.3/doc/man1/wimjoin.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimjoin.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMJOIN "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMJOIN "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimjoin\- Join a split WIM into a standalone WIM .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimlib-imagex.1 new/wimlib-1.13.4/doc/man1/wimlib-imagex.1 --- old/wimlib-1.13.3/doc/man1/wimlib-imagex.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimlib-imagex.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMLIB-IMAGEX 1 "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMLIB-IMAGEX 1 "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimlib-imagex \- Extract, create, modify, or mount a WIM archive .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimmount.1 new/wimlib-1.13.4/doc/man1/wimmount.1 --- old/wimlib-1.13.3/doc/man1/wimmount.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimmount.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMMOUNT "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMMOUNT "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimmount, wimmountrw, wimunmount \- Mount or unmount a WIM image .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimoptimize.1 new/wimlib-1.13.4/doc/man1/wimoptimize.1 --- old/wimlib-1.13.3/doc/man1/wimoptimize.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimoptimize.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMOPTIMIZE "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMOPTIMIZE "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimoptimize \- Optimize a WIM archive .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimsplit.1 new/wimlib-1.13.4/doc/man1/wimsplit.1 --- old/wimlib-1.13.3/doc/man1/wimsplit.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimsplit.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMSPLIT "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMSPLIT "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimsplit \- Split a WIM archive into multiple parts .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimupdate.1 new/wimlib-1.13.4/doc/man1/wimupdate.1 --- old/wimlib-1.13.3/doc/man1/wimupdate.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimupdate.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMUPDATE "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMUPDATE "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimupdate \- Update a WIM image .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/doc/man1/wimverify.1 new/wimlib-1.13.4/doc/man1/wimverify.1 --- old/wimlib-1.13.3/doc/man1/wimverify.1 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/doc/man1/wimverify.1 2021-04-19 06:35:01.000000000 +0200 @@ -1,4 +1,4 @@ -.TH WIMVERIFY "1" "October 2020" "wimlib 1.13.3" "User Commands" +.TH WIMVERIFY "1" "April 2021" "wimlib 1.13.4" "User Commands" .SH NAME wimverify \- Verify a WIM archive .SH SYNOPSIS diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/include/wimlib/blob_table.h new/wimlib-1.13.4/include/wimlib/blob_table.h --- old/wimlib-1.13.3/include/wimlib/blob_table.h 2020-02-21 05:14:39.000000000 +0100 +++ new/wimlib-1.13.4/include/wimlib/blob_table.h 2021-04-07 05:55:50.000000000 +0200 @@ -141,6 +141,9 @@ /* 1 iff the SHA-1 message digest of this blob is unknown. */ u16 unhashed : 1; + /* 1 iff this blob has failed its checksum. */ + u16 corrupted : 1; + /* Temporary fields used when writing blobs; set as documented for * prepare_blob_list_for_write(). */ u16 unique_size : 1; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/include/wimlib/ntfs_3g.h new/wimlib-1.13.4/include/wimlib/ntfs_3g.h --- old/wimlib-1.13.3/include/wimlib/ntfs_3g.h 2017-09-20 04:55:16.000000000 +0200 +++ new/wimlib-1.13.4/include/wimlib/ntfs_3g.h 2021-04-06 21:48:32.000000000 +0200 @@ -11,7 +11,8 @@ extern int read_ntfs_attribute_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb); + const struct consume_chunk_callback *cb, + bool recover_data); extern struct ntfs_location * clone_ntfs_location(const struct ntfs_location *loc); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/include/wimlib/resource.h new/wimlib-1.13.4/include/wimlib/resource.h --- old/wimlib-1.13.3/include/wimlib/resource.h 2017-09-20 04:55:16.000000000 +0200 +++ new/wimlib-1.13.4/include/wimlib/resource.h 2021-04-07 05:32:59.000000000 +0200 @@ -273,6 +273,7 @@ #define VERIFY_BLOB_HASHES 0x1 #define COMPUTE_MISSING_BLOB_HASHES 0x2 #define BLOB_LIST_ALREADY_SORTED 0x4 +#define RECOVER_DATA 0x8 extern int read_blob_list(struct list_head *blob_list, size_t list_head_offset, @@ -280,18 +281,19 @@ extern int read_blob_with_cbs(struct blob_descriptor *blob, - const struct read_blob_callbacks *cbs); + const struct read_blob_callbacks *cbs, bool recover_data); extern int read_blob_with_sha1(struct blob_descriptor *blob, - const struct read_blob_callbacks *cbs); + const struct read_blob_callbacks *cbs, bool recover_data); extern int extract_blob_prefix_to_fd(struct blob_descriptor *blob, u64 size, struct filedes *fd); extern int -extract_blob_to_fd(struct blob_descriptor *blob, struct filedes *fd); +extract_blob_to_fd(struct blob_descriptor *blob, struct filedes *fd, + bool recover_data); /* Miscellaneous blob functions. */ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/include/wimlib/win32.h new/wimlib-1.13.4/include/wimlib/win32.h --- old/wimlib-1.13.3/include/wimlib/win32.h 2017-09-20 04:55:16.000000000 +0200 +++ new/wimlib-1.13.4/include/wimlib/win32.h 2021-04-06 21:48:32.000000000 +0200 @@ -25,7 +25,8 @@ extern int read_windows_file_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb); + const struct consume_chunk_callback *cb, + bool recover_data); extern int win32_global_init(int init_flags); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/include/wimlib.h new/wimlib-1.13.4/include/wimlib.h --- old/wimlib-1.13.3/include/wimlib.h 2020-10-27 04:49:10.000000000 +0100 +++ new/wimlib-1.13.4/include/wimlib.h 2021-04-19 06:35:01.000000000 +0200 @@ -11,7 +11,7 @@ /** * @mainpage * - * This is the documentation for the library interface of wimlib 1.13.3, a C + * This is the documentation for the library interface of wimlib 1.13.4, a C * library for creating, modifying, extracting, and mounting files in the * Windows Imaging (WIM) format. This documentation is intended for developers * only. If you have installed wimlib and want to know how to use the @b @@ -411,7 +411,7 @@ #define WIMLIB_MINOR_VERSION 13 /** Patch version of the library (for example, the 5 in 1.2.5). */ -#define WIMLIB_PATCH_VERSION 3 +#define WIMLIB_PATCH_VERSION 4 #ifdef __cplusplus extern "C" { @@ -649,7 +649,8 @@ * to ::wimlib_progress_info.write_streams. This message may be * received many times while the WIM file is being written or appended * to with wimlib_write(), wimlib_overwrite(), or wimlib_write_to_fd(). - */ + * Since wimlib v1.13.4 it will also be received when a split WIM part + * is being written by wimlib_split(). */ WIMLIB_PROGRESS_MSG_WRITE_STREAMS = 12, /** Per-image metadata is about to be written to the WIM file. @p info @@ -825,7 +826,8 @@ /** The number of bytes of file data that have been written so * far. This starts at 0 and ends at @p total_bytes. This * number is the uncompressed size; the actual size may be lower - * due to compression. */ + * due to compression. See @p completed_compressed_bytes for + * the compressed size. */ uint64_t completed_bytes; /** The number of distinct file data "blobs" that have been @@ -848,6 +850,10 @@ /** This is currently broken and will always be 0. */ uint32_t completed_parts; + + /** Since wimlib v1.13.4: Like @p completed_bytes, but counts + * the compressed size. */ + uint64_t completed_compressed_bytes; } write_streams; /** Valid on messages ::WIMLIB_PROGRESS_MSG_SCAN_BEGIN, @@ -1922,6 +1928,10 @@ * wimlib_extract_paths() when passed multiple paths. */ #define WIMLIB_EXTRACT_FLAG_NTFS 0x00000001 +/** Since wimlib v1.13.4: Don't consider corrupted files to be an error. Just + * extract them in whatever form we can. */ +#define WIMLIB_EXTRACT_FLAG_RECOVER_DATA 0x00000002 + /** UNIX-like systems only: Extract UNIX-specific metadata captured with * ::WIMLIB_ADD_FLAG_UNIX_DATA. */ #define WIMLIB_EXTRACT_FLAG_UNIX_DATA 0x00000020 @@ -4305,7 +4315,9 @@ * If a progress function is registered with @p wim, then for each split WIM * part that is written it will receive the messages * ::WIMLIB_PROGRESS_MSG_SPLIT_BEGIN_PART and - * ::WIMLIB_PROGRESS_MSG_SPLIT_END_PART. + * ::WIMLIB_PROGRESS_MSG_SPLIT_END_PART. Since wimlib v1.13.4 it will also + * receive ::WIMLIB_PROGRESS_MSG_WRITE_STREAMS messages while writing each part; + * these messages will report the progress of the current part only. */ extern int wimlib_split(WIMStruct *wim, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/programs/imagex.c new/wimlib-1.13.4/programs/imagex.c --- old/wimlib-1.13.3/programs/imagex.c 2020-05-22 08:31:25.000000000 +0200 +++ new/wimlib-1.13.4/programs/imagex.c 2021-04-19 05:13:26.000000000 +0200 @@ -6,7 +6,7 @@ */ /* - * Copyright (C) 2012-2018 Eric Biggers + * Copyright (C) 2012-2021 Eric Biggers * * This program is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -202,6 +202,7 @@ IMAGEX_PRESERVE_DIR_STRUCTURE_OPTION, IMAGEX_REBUILD_OPTION, IMAGEX_RECOMPRESS_OPTION, + IMAGEX_RECOVER_DATA_OPTION, IMAGEX_RECURSIVE_OPTION, IMAGEX_REF_OPTION, IMAGEX_RPFIX_OPTION, @@ -239,6 +240,7 @@ {T("include-invalid-names"), no_argument, NULL, IMAGEX_INCLUDE_INVALID_NAMES_OPTION}, {T("wimboot"), no_argument, NULL, IMAGEX_WIMBOOT_OPTION}, {T("compact"), required_argument, NULL, IMAGEX_COMPACT_OPTION}, + {T("recover-data"), no_argument, NULL, IMAGEX_RECOVER_DATA_OPTION}, {NULL, 0, NULL, 0}, }; @@ -336,6 +338,7 @@ {T("preserve-dir-structure"), no_argument, NULL, IMAGEX_PRESERVE_DIR_STRUCTURE_OPTION}, {T("wimboot"), no_argument, NULL, IMAGEX_WIMBOOT_OPTION}, {T("compact"), required_argument, NULL, IMAGEX_COMPACT_OPTION}, + {T("recover-data"), no_argument, NULL, IMAGEX_RECOVER_DATA_OPTION}, {NULL, 0, NULL, 0}, }; @@ -1163,6 +1166,31 @@ last_scan_progress = *scan; } } + +static struct wimlib_progress_info_split last_split_progress; + +static void +report_split_progress(uint64_t bytes_completed_in_part) +{ + uint64_t completed_bytes = last_split_progress.completed_bytes + + bytes_completed_in_part; + unsigned percent_done = TO_PERCENT(completed_bytes, + last_split_progress.total_bytes); + unsigned unit_shift; + const tchar *unit_name; + + unit_shift = get_unit(last_split_progress.total_bytes, &unit_name); + imagex_printf(T("\rSplitting WIM: %"PRIu64" %"TS" of " + "%"PRIu64" %"TS" (%u%%) written, part %u of %u"), + completed_bytes >> unit_shift, + unit_name, + last_split_progress.total_bytes >> unit_shift, + unit_name, + percent_done, + last_split_progress.cur_part_number, + last_split_progress.total_parts); +} + /* Progress callback function passed to various wimlib functions. */ static enum wimlib_progress_status imagex_progress_func(enum wimlib_progress_msg msg, @@ -1175,6 +1203,12 @@ switch (msg) { case WIMLIB_PROGRESS_MSG_WRITE_STREAMS: + if (last_split_progress.total_bytes != 0) { + /* wimlib_split() in progress; use the split-specific + * progress message. */ + report_split_progress(info->write_streams.completed_compressed_bytes); + break; + } { static bool started; if (!started) { @@ -1330,26 +1364,9 @@ } break; case WIMLIB_PROGRESS_MSG_SPLIT_BEGIN_PART: - percent_done = TO_PERCENT(info->split.completed_bytes, - info->split.total_bytes); - unit_shift = get_unit(info->split.total_bytes, &unit_name); - imagex_printf(T("Writing \"%"TS"\" (part %u of %u): %"PRIu64" %"TS" of " - "%"PRIu64" %"TS" (%u%%) written\n"), - info->split.part_name, - info->split.cur_part_number, - info->split.total_parts, - info->split.completed_bytes >> unit_shift, - unit_name, - info->split.total_bytes >> unit_shift, - unit_name, - percent_done); - break; case WIMLIB_PROGRESS_MSG_SPLIT_END_PART: - if (info->split.completed_bytes == info->split.total_bytes) { - imagex_printf(T("Finished writing split WIM part %u of %u\n"), - info->split.cur_part_number, - info->split.total_parts); - } + last_split_progress = info->split; + report_split_progress(0); break; case WIMLIB_PROGRESS_MSG_UPDATE_END_COMMAND: switch (info->update.command->op) { @@ -1733,6 +1750,9 @@ if (ret) goto out_free_refglobs; break; + case IMAGEX_RECOVER_DATA_OPTION: + extract_flags |= WIMLIB_EXTRACT_FLAG_RECOVER_DATA; + break; default: goto out_usage; } @@ -3280,6 +3300,9 @@ if (ret) goto out_free_refglobs; break; + case IMAGEX_RECOVER_DATA_OPTION: + extract_flags |= WIMLIB_EXTRACT_FLAG_RECOVER_DATA; + break; default: goto out_usage; } @@ -4019,6 +4042,7 @@ goto out; ret = wimlib_split(wim, argv[1], part_size, write_flags); + tprintf(T("\nFinished splitting \"%"TS"\"\n"), argv[0]); wimlib_free(wim); out: return ret; @@ -4469,7 +4493,7 @@ " [--check] [--ref=\"GLOB\"] [--no-acls] [--strict-acls]\n" " [--no-attributes] [--rpfix] [--norpfix]\n" " [--include-invalid-names] [--wimboot] [--unix-data]\n" -" [--compact=FORMAT]\n" +" [--compact=FORMAT] [--recover-data]\n" ), [CMD_CAPTURE] = T( @@ -4502,8 +4526,8 @@ " %"TS" WIMFILE IMAGE [(PATH | @LISTFILE)...]\n" " [--check] [--ref=\"GLOB\"] [--dest-dir=CMD_DIR]\n" " [--to-stdout] [--no-acls] [--strict-acls]\n" -" [--no-attributes] [--include-invalid-names]\n" -" [--no-globs] [--nullglob] [--preserve-dir-structure]\n" +" [--no-attributes] [--include-invalid-names] [--no-globs]\n" +" [--nullglob] [--preserve-dir-structure] [--recover-data]\n" ), [CMD_INFO] = T( @@ -4587,7 +4611,7 @@ static const tchar * const fmt = T( "wimlib-imagex " PACKAGE_VERSION " (using wimlib %"TS")\n" -"Copyright (C) 2012-2018 Eric Biggers\n" +"Copyright (C) 2012-2021 Eric Biggers\n" "License GPLv3+; GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.\n" "This is free software: you are free to change and redistribute it.\n" "There is NO WARRANTY, to the extent permitted by law.\n" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/programs/mkwinpeimg new/wimlib-1.13.4/programs/mkwinpeimg --- old/wimlib-1.13.3/programs/mkwinpeimg 2020-10-27 04:54:57.000000000 +0100 +++ new/wimlib-1.13.4/programs/mkwinpeimg 2021-04-19 06:35:44.000000000 +0200 @@ -20,7 +20,7 @@ script_name="$(basename "$0")" PREFIX_REG="::" -WIMLIB_VERSION=1.13.3 +WIMLIB_VERSION=1.13.4 calc_columns () { STAT_COL=80 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/src/extract.c new/wimlib-1.13.4/src/extract.c --- old/wimlib-1.13.3/src/extract.c 2018-07-21 22:32:01.000000000 +0200 +++ new/wimlib-1.13.4/src/extract.c 2021-04-19 06:35:01.000000000 +0200 @@ -72,6 +72,7 @@ /* Keep in sync with wimlib.h */ #define WIMLIB_EXTRACT_MASK_PUBLIC \ (WIMLIB_EXTRACT_FLAG_NTFS | \ + WIMLIB_EXTRACT_FLAG_RECOVER_DATA | \ WIMLIB_EXTRACT_FLAG_UNIX_DATA | \ WIMLIB_EXTRACT_FLAG_NO_ACLS | \ WIMLIB_EXTRACT_FLAG_STRICT_ACLS | \ @@ -310,7 +311,9 @@ && (blob->out_refcnt)) { wim_reshdr_to_desc_and_blob(&reshdr, ctx->wim, &rdesc, blob); - ret = read_blob_with_sha1(blob, cbs); + ret = read_blob_with_sha1(blob, cbs, + ctx->extract_flags & + WIMLIB_EXTRACT_FLAG_RECOVER_DATA); blob_unset_is_located_in_wim_resource(blob); if (ret) return ret; @@ -504,18 +507,39 @@ for (u32 i = 0; i < orig_blob->out_refcnt; i++) { tmpfile_blob.inline_blob_extraction_targets[0] = targets[i]; - ret = read_blob_with_cbs(&tmpfile_blob, cbs); + ret = read_blob_with_cbs(&tmpfile_blob, cbs, false); if (ret) return ret; } return 0; } +static void +warn_about_corrupted_file(struct wim_dentry *dentry, + const struct wim_inode_stream *stream) +{ + WARNING("Corruption in %s\"%"TS"\"! Extracting anyway since data recovery mode is enabled.", + stream_is_unnamed_data_stream(stream) ? "" : "alternate stream of ", + dentry_full_path(dentry)); +} + static int end_extract_blob(struct blob_descriptor *blob, int status, void *_ctx) { struct apply_ctx *ctx = _ctx; + if ((ctx->extract_flags & WIMLIB_EXTRACT_FLAG_RECOVER_DATA) && + !status && blob->corrupted) { + const struct blob_extraction_target *targets = + blob_extraction_targets(blob); + for (u32 i = 0; i < blob->out_refcnt; i++) { + struct wim_dentry *dentry = + inode_first_extraction_dentry(targets[i].inode); + + warn_about_corrupted_file(dentry, targets[i].stream); + } + } + if (unlikely(filedes_valid(&ctx->tmpfile_fd))) { filedes_close(&ctx->tmpfile_fd); if (!status) @@ -560,10 +584,15 @@ if (ctx->extract_flags & WIMLIB_EXTRACT_FLAG_FROM_PIPE) { return read_blobs_from_pipe(ctx, &wrapper_cbs); } else { + int flags = VERIFY_BLOB_HASHES; + + if (ctx->extract_flags & WIMLIB_EXTRACT_FLAG_RECOVER_DATA) + flags |= RECOVER_DATA; + return read_blob_list(&ctx->blob_list, offsetof(struct blob_descriptor, extraction_list), - &wrapper_cbs, VERIFY_BLOB_HASHES); + &wrapper_cbs, flags); } } @@ -574,11 +603,13 @@ * unnamed data stream only. */ static int extract_dentry_to_stdout(struct wim_dentry *dentry, - const struct blob_table *blob_table) + const struct blob_table *blob_table, int extract_flags) { struct wim_inode *inode = dentry->d_inode; struct blob_descriptor *blob; struct filedes _stdout; + bool recover = (extract_flags & WIMLIB_EXTRACT_FLAG_RECOVER_DATA); + int ret; if (inode->i_attributes & (FILE_ATTRIBUTE_REPARSE_POINT | FILE_ATTRIBUTE_DIRECTORY | @@ -598,15 +629,23 @@ } filedes_init(&_stdout, STDOUT_FILENO); - return extract_blob_to_fd(blob, &_stdout); + ret = extract_blob_to_fd(blob, &_stdout, recover); + if (ret) + return ret; + if (recover && blob->corrupted) + warn_about_corrupted_file(dentry, + inode_get_unnamed_data_stream(inode)); + return 0; } static int extract_dentries_to_stdout(struct wim_dentry **dentries, size_t num_dentries, - const struct blob_table *blob_table) + const struct blob_table *blob_table, + int extract_flags) { for (size_t i = 0; i < num_dentries; i++) { - int ret = extract_dentry_to_stdout(dentries[i], blob_table); + int ret = extract_dentry_to_stdout(dentries[i], blob_table, + extract_flags); if (ret) return ret; } @@ -1446,7 +1485,8 @@ if (extract_flags & WIMLIB_EXTRACT_FLAG_TO_STDOUT) { ret = extract_dentries_to_stdout(trees, num_trees, - wim->blob_table); + wim->blob_table, + extract_flags); goto out; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/src/ntfs-3g_capture.c new/wimlib-1.13.4/src/ntfs-3g_capture.c --- old/wimlib-1.13.3/src/ntfs-3g_capture.c 2018-05-02 05:38:31.000000000 +0200 +++ new/wimlib-1.13.4/src/ntfs-3g_capture.c 2021-04-07 05:56:07.000000000 +0200 @@ -117,7 +117,8 @@ int read_ntfs_attribute_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, + bool recover_data) { const struct ntfs_location *loc = blob->ntfs_loc; ntfs_volume *vol = loc->volume->vol; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/src/resource.c new/wimlib-1.13.4/src/resource.c --- old/wimlib-1.13.3/src/resource.c 2017-09-20 04:55:17.000000000 +0200 +++ new/wimlib-1.13.4/src/resource.c 2021-04-07 06:31:47.000000000 +0200 @@ -83,6 +83,34 @@ u64 size; }; +static int +decompress_chunk(const void *cbuf, u32 chunk_csize, u8 *ubuf, u32 chunk_usize, + struct wimlib_decompressor *decompressor, bool recover_data) +{ + int res = wimlib_decompress(cbuf, chunk_csize, ubuf, chunk_usize, + decompressor); + if (likely(res == 0)) + return 0; + + if (recover_data) { + WARNING("Failed to decompress data! Continuing anyway since data recovery mode is enabled."); + + /* Continue on with *something*. In the worst case just use a + * zeroed buffer. But, try to fill as much of it with + * decompressed data as we can. This works because if the + * corruption isn't located right at the beginning of the + * compressed chunk, wimlib_decompress() may write some correct + * output at the beginning even if it fails later. */ + memset(ubuf, 0, chunk_usize); + (void)wimlib_decompress(cbuf, chunk_csize, ubuf, + chunk_usize, decompressor); + return 0; + } + ERROR("Failed to decompress data!"); + errno = EINVAL; + return WIMLIB_ERR_DECOMPRESSION; +} + /* * Read data from a compressed WIM resource. * @@ -98,6 +126,9 @@ * the data being read. Each call provides the next chunk of the requested * data, uncompressed. Each chunk will be nonempty and will not cross * range boundaries but otherwise will be of unspecified size. + * @recover_data + * If a chunk can't be fully decompressed due to being corrupted, continue + * with whatever data can be recovered rather than return an error. * * Possible return values: * @@ -114,7 +145,8 @@ read_compressed_wim_resource(const struct wim_resource_descriptor * const rdesc, const struct data_range * const ranges, const size_t num_ranges, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, + bool recover_data) { int ret; u64 *chunk_offsets = NULL; @@ -446,17 +478,12 @@ goto read_error; if (read_buf == cbuf) { - ret = wimlib_decompress(cbuf, - chunk_csize, - ubuf, - chunk_usize, - decompressor); - if (unlikely(ret)) { - ERROR("Failed to decompress data!"); - ret = WIMLIB_ERR_DECOMPRESSION; - errno = EINVAL; + ret = decompress_chunk(cbuf, chunk_csize, + ubuf, chunk_usize, + decompressor, + recover_data); + if (unlikely(ret)) goto out_cleanup; - } } cur_read_offset += chunk_csize; @@ -592,7 +619,8 @@ static int read_partial_wim_resource(const struct wim_resource_descriptor *rdesc, const u64 offset, const u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, + bool recover_data) { if (rdesc->flags & (WIM_RESHDR_FLAG_COMPRESSED | WIM_RESHDR_FLAG_SOLID)) @@ -604,7 +632,8 @@ .offset = offset, .size = size, }; - return read_compressed_wim_resource(rdesc, &range, 1, cb); + return read_compressed_wim_resource(rdesc, &range, 1, cb, + recover_data); } /* Uncompressed resource */ @@ -626,7 +655,7 @@ return read_partial_wim_resource(blob->rdesc, blob->offset_in_res + offset, size, - &cb); + &cb, false); } static int @@ -643,15 +672,15 @@ .func = noop_cb, }; return read_partial_wim_resource(rdesc, 0, - rdesc->uncompressed_size, &cb); + rdesc->uncompressed_size, &cb, false); } static int read_wim_blob_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, bool recover_data) { return read_partial_wim_resource(blob->rdesc, blob->offset_in_res, - size, cb); + size, cb, recover_data); } /* This function handles reading blob data that is located in an external file, @@ -664,7 +693,8 @@ * encrypted), so Windows uses its own code for its equivalent case. */ static int read_file_on_disk_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, + bool recover_data) { int ret; int raw_fd; @@ -684,7 +714,8 @@ #ifdef WITH_FUSE static int read_staging_file_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, + bool recover_data) { int raw_fd; struct filedes fd; @@ -708,7 +739,8 @@ * already located in an in-memory buffer. */ static int read_buffer_prefix(const struct blob_descriptor *blob, - u64 size, const struct consume_chunk_callback *cb) + u64 size, const struct consume_chunk_callback *cb, + bool recover_data) { if (unlikely(!size)) return 0; @@ -717,7 +749,8 @@ typedef int (*read_blob_prefix_handler_t)(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb); + const struct consume_chunk_callback *cb, + bool recover_data); /* * Read the first @size bytes from a generic "blob", which may be located in any @@ -728,11 +761,12 @@ * Returns 0 on success; nonzero on error. A nonzero value will be returned if * the blob data cannot be successfully read (for a number of different reasons, * depending on the blob location), or if @cb returned nonzero in which case - * that error code will be returned. + * that error code will be returned. If @recover_data is true, then errors + * decompressing chunks in WIM resources will be ignored. */ static int read_blob_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, bool recover_data) { static const read_blob_prefix_handler_t handlers[] = { [BLOB_IN_WIM] = read_wim_blob_prefix, @@ -751,7 +785,7 @@ wimlib_assert(blob->blob_location < ARRAY_LEN(handlers) && handlers[blob->blob_location] != NULL); wimlib_assert(size <= blob->size); - return handlers[blob->blob_location](blob, size, cb); + return handlers[blob->blob_location](blob, size, cb, recover_data); } struct blob_chunk_ctx { @@ -775,7 +809,7 @@ * callbacks (all of which are optional). */ int read_blob_with_cbs(struct blob_descriptor *blob, - const struct read_blob_callbacks *cbs) + const struct read_blob_callbacks *cbs, bool recover_data) { int ret; struct blob_chunk_ctx ctx = { @@ -792,7 +826,7 @@ if (unlikely(ret)) return ret; - ret = read_blob_prefix(blob, blob->size, &cb); + ret = read_blob_prefix(blob, blob->size, &cb, recover_data); return call_end_blob(blob, ret, cbs); } @@ -807,7 +841,7 @@ .func = bufferer_cb, .ctx = &buf, }; - return read_blob_prefix(blob, blob->size, &cb); + return read_blob_prefix(blob, blob->size, &cb, false); } /* Retrieve the full uncompressed data of the specified blob. A buffer large @@ -955,6 +989,7 @@ struct hasher_context *ctx = _ctx; sha1_init(&ctx->sha_ctx); + blob->corrupted = 0; return call_begin_blob(blob, &ctx->cbs); } @@ -977,8 +1012,8 @@ } static int -report_sha1_mismatch_error(const struct blob_descriptor *blob, - const u8 actual_hash[SHA1_HASH_SIZE]) +report_sha1_mismatch(struct blob_descriptor *blob, + const u8 actual_hash[SHA1_HASH_SIZE], bool recover_data) { tchar expected_hashstr[SHA1_HASH_SIZE * 2 + 1]; tchar actual_hashstr[SHA1_HASH_SIZE * 2 + 1]; @@ -989,6 +1024,8 @@ sprint_hash(blob->hash, expected_hashstr); sprint_hash(actual_hash, actual_hashstr); + blob->corrupted = 1; + if (blob_is_in_file(blob)) { ERROR("A file was concurrently modified!\n" " Path: \"%"TS"\"\n" @@ -997,18 +1034,21 @@ blob_file_path(blob), expected_hashstr, actual_hashstr); return WIMLIB_ERR_CONCURRENT_MODIFICATION_DETECTED; } else if (blob->blob_location == BLOB_IN_WIM) { + #ifdef ENABLE_ERROR_MESSAGES const struct wim_resource_descriptor *rdesc = blob->rdesc; - ERROR("A WIM resource is corrupted!\n" - " WIM file: \"%"TS"\"\n" - " Blob uncompressed size: %"PRIu64"\n" - " Resource offset in WIM: %"PRIu64"\n" - " Resource uncompressed size: %"PRIu64"\n" - " Resource size in WIM: %"PRIu64"\n" - " Resource flags: 0x%x%"TS"\n" - " Resource compression type: %"TS"\n" - " Resource compression chunk size: %"PRIu32"\n" - " Expected SHA-1: %"TS"\n" - " Actual SHA-1: %"TS"\n", + + (recover_data ? wimlib_warning : wimlib_error)( + T("A WIM resource is corrupted!\n" + " WIM file: \"%"TS"\"\n" + " Blob uncompressed size: %"PRIu64"\n" + " Resource offset in WIM: %"PRIu64"\n" + " Resource uncompressed size: %"PRIu64"\n" + " Resource size in WIM: %"PRIu64"\n" + " Resource flags: 0x%x%"TS"\n" + " Resource compression type: %"TS"\n" + " Resource compression chunk size: %"PRIu32"\n" + " Expected SHA-1: %"TS"\n" + " Actual SHA-1: %"TS"\n"), rdesc->wim->filename, blob->size, rdesc->offset_in_wim, @@ -1020,6 +1060,9 @@ rdesc->compression_type), rdesc->chunk_size, expected_hashstr, actual_hashstr); + #endif /* ENABLE_ERROR_MESSAGES */ + if (recover_data) + return 0; return WIMLIB_ERR_INVALID_RESOURCE_HASH; } else { ERROR("File data was concurrently modified!\n" @@ -1058,7 +1101,8 @@ } else if ((ctx->flags & VERIFY_BLOB_HASHES) && unlikely(!hashes_equal(hash, blob->hash))) { - ret = report_sha1_mismatch_error(blob, hash); + ret = report_sha1_mismatch(blob, hash, + ctx->flags & RECOVER_DATA); goto out_next_cb; } ret = 0; @@ -1071,10 +1115,11 @@ * SHA-1 message digest of the blob. */ int read_blob_with_sha1(struct blob_descriptor *blob, - const struct read_blob_callbacks *cbs) + const struct read_blob_callbacks *cbs, bool recover_data) { struct hasher_context hasher_ctx = { - .flags = VERIFY_BLOB_HASHES | COMPUTE_MISSING_BLOB_HASHES, + .flags = VERIFY_BLOB_HASHES | COMPUTE_MISSING_BLOB_HASHES | + (recover_data ? RECOVER_DATA : 0), .cbs = *cbs, }; struct read_blob_callbacks hasher_cbs = { @@ -1083,7 +1128,7 @@ .end_blob = hasher_end_blob, .ctx = &hasher_ctx, }; - return read_blob_with_cbs(blob, &hasher_cbs); + return read_blob_with_cbs(blob, &hasher_cbs, recover_data); } static int @@ -1091,7 +1136,8 @@ struct blob_descriptor *last_blob, size_t blob_count, size_t list_head_offset, - const struct read_blob_callbacks *sink_cbs) + const struct read_blob_callbacks *sink_cbs, + bool recover_data) { struct data_range *ranges; bool ranges_malloced; @@ -1141,7 +1187,7 @@ }; ret = read_compressed_wim_resource(first_blob->rdesc, ranges, - blob_count, &cb); + blob_count, &cb, recover_data); if (ranges_malloced) FREE(ranges); @@ -1178,7 +1224,8 @@ * For all blobs being read that have already had SHA-1 message * digests computed, calculate the SHA-1 message digest of the read * data and compare it with the previously computed value. If they - * do not match, return WIMLIB_ERR_INVALID_RESOURCE_HASH. + * do not match, return WIMLIB_ERR_INVALID_RESOURCE_HASH (unless + * RECOVER_DATA is also set, in which case just issue a warning). * * COMPUTE_MISSING_BLOB_HASHES * For all blobs being read that have not yet had their SHA-1 @@ -1188,6 +1235,9 @@ * BLOB_LIST_ALREADY_SORTED * @blob_list is already sorted in sequential order for reading. * + * RECOVER_DATA + * Don't consider corrupted blob data to be an error. + * * The callback functions are allowed to delete the current blob from the list * if necessary. * @@ -1273,14 +1323,15 @@ ret = read_blobs_in_solid_resource(blob, blob_last, blob_count, list_head_offset, - sink_cbs); + sink_cbs, + flags & RECOVER_DATA); if (ret) return ret; continue; } } - ret = read_blob_with_cbs(blob, sink_cbs); + ret = read_blob_with_cbs(blob, sink_cbs, flags & RECOVER_DATA); if (unlikely(ret && ret != BEGIN_BLOB_STATUS_SKIP_BLOB)) return ret; } @@ -1314,19 +1365,20 @@ .func = extract_chunk_to_fd, .ctx = fd, }; - return read_blob_prefix(blob, size, &cb); + return read_blob_prefix(blob, size, &cb, false); } /* Extract the full uncompressed contents of the specified blob to the specified * file descriptor. This checks the SHA-1 message digest. */ int -extract_blob_to_fd(struct blob_descriptor *blob, struct filedes *fd) +extract_blob_to_fd(struct blob_descriptor *blob, struct filedes *fd, + bool recover_data) { struct read_blob_callbacks cbs = { .continue_blob = extract_blob_chunk_to_fd, .ctx = fd, }; - return read_blob_with_sha1(blob, &cbs); + return read_blob_with_sha1(blob, &cbs, recover_data); } /* Calculate the SHA-1 message digest of a blob and store it in @blob->hash. */ @@ -1335,7 +1387,7 @@ { static const struct read_blob_callbacks cbs = { }; - return read_blob_with_sha1(blob, &cbs); + return read_blob_with_sha1(blob, &cbs, false); } /* diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/src/split.c new/wimlib-1.13.4/src/split.c --- old/wimlib-1.13.3/src/split.c 2018-05-02 05:38:31.000000000 +0200 +++ new/wimlib-1.13.4/src/split.c 2021-04-02 05:59:52.000000000 +0200 @@ -96,7 +96,6 @@ for (part_number = 1; part_number <= swm_info->num_parts; part_number++) { int part_write_flags; - wimlib_progress_func_t progfunc; if (part_number != 1) { tsprintf(swm_name_buf + swm_base_name_len, @@ -118,8 +117,6 @@ if (part_number != 1) part_write_flags |= WIMLIB_WRITE_FLAG_NO_METADATA; - progfunc = orig_wim->progfunc; - orig_wim->progfunc = NULL; ret = write_wim_part(orig_wim, progress.split.part_name, WIMLIB_ALL_IMAGES, @@ -129,7 +126,6 @@ swm_info->num_parts, &swm_info->parts[part_number - 1].blob_list, guid); - orig_wim->progfunc = progfunc; if (ret) return ret; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/src/win32_capture.c new/wimlib-1.13.4/src/win32_capture.c --- old/wimlib-1.13.3/src/win32_capture.c 2020-06-02 06:10:11.000000000 +0200 +++ new/wimlib-1.13.4/src/win32_capture.c 2021-04-06 21:48:32.000000000 +0200 @@ -488,7 +488,8 @@ * described by @blob. */ int read_windows_file_prefix(const struct blob_descriptor *blob, u64 size, - const struct consume_chunk_callback *cb) + const struct consume_chunk_callback *cb, + bool recover_data) { const struct windows_file *file = blob->windows_file; diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/src/write.c new/wimlib-1.13.4/src/write.c --- old/wimlib-1.13.3/src/write.c 2019-04-14 06:58:03.000000000 +0200 +++ new/wimlib-1.13.4/src/write.c 2021-04-19 06:35:01.000000000 +0200 @@ -301,7 +301,8 @@ static int do_write_blobs_progress(struct write_blobs_progress_data *progress_data, - u64 complete_size, u32 complete_count, bool discarded) + u64 complete_size, u64 complete_compressed_size, + u32 complete_count, bool discarded) { union wimlib_progress_info *progress = &progress_data->progress; int ret; @@ -316,6 +317,8 @@ } } else { progress->write_streams.completed_bytes += complete_size; + progress->write_streams.completed_compressed_bytes += + complete_compressed_size; progress->write_streams.completed_streams += complete_count; } @@ -713,7 +716,9 @@ * output reference count to the duplicate blob * in the former case. */ ret = do_write_blobs_progress(&ctx->progress_data, - blob->size, 1, true); + blob->size, + blob->size, + 1, true); list_del(&blob->write_blobs_list); list_del(&blob->blob_table_list); if (new_blob->will_be_in_output_wim) @@ -762,7 +767,7 @@ if (filedes_seek(out_fd, begin_offset) == -1) return 0; - ret = extract_blob_to_fd(blob, out_fd); + ret = extract_blob_to_fd(blob, out_fd, false); if (ret) { /* Error reading the uncompressed data. */ if (out_fd->offset == begin_offset && @@ -867,8 +872,7 @@ { int ret; struct blob_descriptor *blob; - u32 completed_blob_count; - u32 completed_size; + u32 completed_blob_count = 0; blob = list_entry(ctx->blobs_being_compressed.next, struct blob_descriptor, write_blobs_list); @@ -915,8 +919,6 @@ ctx->cur_write_blob_offset += usize; - completed_size = usize; - completed_blob_count = 0; if (ctx->write_resource_flags & WRITE_RESOURCE_FLAG_SOLID) { /* Wrote chunk in solid mode. It may have finished multiple * blobs. */ @@ -973,7 +975,7 @@ } } - return do_write_blobs_progress(&ctx->progress_data, completed_size, + return do_write_blobs_progress(&ctx->progress_data, usize, csize, completed_blob_count, false); write_error: @@ -1287,15 +1289,18 @@ blob->rdesc->raw_copy_ok = 1; list_for_each_entry(blob, raw_copy_blobs, write_blobs_list) { + u64 compressed_size = 0; + if (blob->rdesc->raw_copy_ok) { /* Write each solid resource only one time. */ ret = write_raw_copy_resource(blob->rdesc, out_fd); if (ret) return ret; blob->rdesc->raw_copy_ok = 0; + compressed_size = blob->rdesc->size_in_wim; } ret = do_write_blobs_progress(progress_data, blob->size, - 1, false); + compressed_size, 1, false); if (ret) return ret; } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/tests/test-imagex-capture_and_apply new/wimlib-1.13.4/tests/test-imagex-capture_and_apply --- old/wimlib-1.13.3/tests/test-imagex-capture_and_apply 2018-07-21 19:21:18.000000000 +0200 +++ new/wimlib-1.13.4/tests/test-imagex-capture_and_apply 2021-04-07 05:46:27.000000000 +0200 @@ -142,6 +142,20 @@ . $srcdir/tests/common_tests.sh +# Test the data recovery mode +__msg "Testing data recovery mode" +for file in corrupted_file_1.wim corrupted_file_2.wim; do + rm -rf out.dir + wimapply $srcdir/tests/wims/$file 1 out.dir 2>/dev/null && \ + error "Applying $file in default mode unexpectedly succeeded" + rm -rf out.dir + wimapply --recover-data $srcdir/tests/wims/$file 1 out.dir || \ + error "Applying $file in data recovery mode unexpectedly failed" + if [ ! -e out.dir/file ]; then + error "Recovered file not found" + fi +done + # Make sure exclusion list works __msg "Testing default capture configuration file" touch in.dir/hiberfil.sys diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' --exclude Makefile.in --exclude configure --exclude config.guess --exclude '*.pot' --exclude mkinstalldirs --exclude aclocal.m4 --exclude config.sub --exclude depcomp --exclude install-sh --exclude ltmain.sh old/wimlib-1.13.3/tests/wims/README new/wimlib-1.13.4/tests/wims/README --- old/wimlib-1.13.3/tests/wims/README 1970-01-01 01:00:00.000000000 +0100 +++ new/wimlib-1.13.4/tests/wims/README 2021-04-07 06:21:48.000000000 +0200 @@ -0,0 +1,27 @@ +Some fun files: + +corrupted_file_1.wim: This WIM contains a file whose SHA-1 digest doesn't match. + +corrupted_file_2.wim: This WIM contains a file that fails to decompress. + +cyclic.wim: This WIM has an image with a cyclic directory structure and should be +detected as invalid. + +duplicate_names.wim: This WIM has an image with multiple files with the same +name in the same directory, and should be detected as invalid. + +dotdot.wim: This WIM has an image with the path +/../../../../../../../../../../../../../../../../etc/passwd, and should be +detected as invalid. (Fun fact: WIMGAPI is dumb and will extract .. files, and +requires running with Admin rights, so given a malicious WIM file it will +overwrite arbitrary files on the target drive.) + +longpaths.wim: This WIM has an image with a path longer than MAX_PATH on Windows. +This should still be extracted successfully. + +empty_dacl.wim: This WIM has an image containing file with a security +descriptor having an empty DACL. This is valid and should be extracted +successfully. + +linux_xattrs_old.wim: Includes Linux xattrs in old format +(TAG_WIMLIB_LINUX_XATTRS) Binary files old/wimlib-1.13.3/tests/wims/corrupted_file_1.wim and new/wimlib-1.13.4/tests/wims/corrupted_file_1.wim differ Binary files old/wimlib-1.13.3/tests/wims/corrupted_file_2.wim and new/wimlib-1.13.4/tests/wims/corrupted_file_2.wim differ Binary files old/wimlib-1.13.3/tests/wims/cyclic.wim and new/wimlib-1.13.4/tests/wims/cyclic.wim differ Binary files old/wimlib-1.13.3/tests/wims/dotdot.wim and new/wimlib-1.13.4/tests/wims/dotdot.wim differ Binary files old/wimlib-1.13.3/tests/wims/duplicate_names.wim and new/wimlib-1.13.4/tests/wims/duplicate_names.wim differ Binary files old/wimlib-1.13.3/tests/wims/empty_dacl.wim and new/wimlib-1.13.4/tests/wims/empty_dacl.wim differ Binary files old/wimlib-1.13.3/tests/wims/linux_xattrs_old.wim and new/wimlib-1.13.4/tests/wims/linux_xattrs_old.wim differ Binary files old/wimlib-1.13.3/tests/wims/longpaths.wim and new/wimlib-1.13.4/tests/wims/longpaths.wim differ
