This patch series contains the initial VHDX log parsing, replay, write support, and image creation.
=== v7 changes === https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v7-upstream Rebased to latest qemu/master (picked up vhdx r/o tests, migration blocker) Patch 8/19: * validate log descriptor_count (Stefan) * fix typos in comments (Stefan) * Removed unneccessary initialization (Stefan) * Replay log prior to metadata (Stefan) * In vhdx_log_flush(), call bdrv_flush() prior to zeroing out the log guid in the header. * In vhdx_close(), set freed pointers to NULL Patch 9/19: * correct logic for region overlap (Stefan) Patch 10/19: * add missing goto exit in error case (Stefan) * add bdrv_flush() to ensure data is stable on disk (Stefan) Patch 11/19: * fixed typos in comments (Stefan) * QEMU coding style changes (Stefan) * rename bat_entry to bat_entry_le for clarity (Stefan) * Add PAYLOAD_BLOCK_ZERO explicit zero padding for protocols that do not support zero init (Stefan) * rename PAYLOAD_BLOCK_FULL_PRESENT to PAYLOAD_BLOCK_FULLY_PRESENT (Stefan) Patch 13/19: * Fixed typo in commit message (Stefan) Patch 19/19: * New, adds qemu-io test for log replay of data sector v6 Patch 17/20: * Dropped (already upstream) v6 Patch 18/20: * Dropped (already upstream) === v6 changes === https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v6-upstream Rebased to latest qemu/master: Patch 16/20: .bdrv_create() propagates Error, and bdrv_unref() used instead of bdrv_delete(). Patch 17 & 18 are already included in another series: [PATCH v3 0/3] qemu-iotests with sample images, vhdx test, cleanup They are included here to provide a base for patches 19 & 20. If the above series is applied before this series, then patches 17 and 18 can be ignored. Patch 19/20: In qemu-io tests _make_test_img(), filter out vhdx-specific options for .bdrv_create(). Patch 20/20: Add VHDX write test case to 064. === v5 changes === v5 is also available for testing from: https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v5-upstream Most of the patches from v4 -> v5 are the same, but there are a few differences and a few new patches. Here is a summary of which patches are different and/or new: Patch highlights: Patch 7 just some minor code movement, in prep for changes in patch 8 Patch 8 incorporates review feedback from Stefan, for the previous Patch 7 in v4. Patch 9 adds region checking for log, region table, and metadata tables, per suggestion from Stefan. Patch 10 minor change from changes made in 8/16 (vhdx_guid_is_zero() is gone) Patch 12 is just some minor housekeeping, to get rid of bit shifting that doesn't need to happen. === v4 changes === v4 patches are available from github as well, on branch vhdx-write-v4-upstream: https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v4-upstream https://github.com/codyprime/qemu-kvm-jtc.git Those in the midst of reviewing v3, don't fear - the only changes with v4 is the addition of patches on the end of the series (patches 10-13). These patches enable creating VHDX images. Image files created have been (briefly & lightly) tested on Hyper-V running on Windows Server 2012. Some of the new patches could be squashed with earlier patches in the series, but I refrained from doing so, since some of the patches have already been reviewed, and others are in the midst of review. I want to make it as easy as possible on those currently reviewing. There is nothing critical that needs to be pushed into the earlier patches. New patches: Patch 10: Breaks out some more endian translation functions (likely squashable into patch 5) Patch 11: Break out some operations into seperate helper functions Patch 12: More comment typos and header fixes in vhdx.h (likely squashable into patch 1) Patch 13: Adds .bdrv_create() for vhdx. VHDX images are can be created for Fixed or Dynamic images. Patches 1-9 are unchanged. === end v4 changelog === === v3 changes === Thank you Kevin & Stefan for the feedback; incoporated in v3: Patch 1: --- nil --- Patch 2: * use sizeof(crc) instead of 4 * remove outdated comment * use sizeof(MSGUID) instead of 16 * direct assignment of guid structs rather than memcpy * rename 'rw' to 'generate_data_write_guid' * use offsetof() instead of 4 * comment typos * add missing error checking * MSGUID is now QEMU_PACKED * configure enable for vhdx is now correct and not braindead Patch 3: --- nil --- Patch 4: * code style fixes * removed unused struct (VHDXLogEntryInfo) Patch 5: * more direct assignment of guid rather than memcpy * order of operation in export/import the same now * became less generous with newlines (bah-humbug!) Patch 6: * more direct assignment of guid rather than memcpy * add error check in vhdx_user_visible_write(), now returns int Patch 7: * check error return now of vhdx_user_visible_write() Patch 8: * check error return now of vhdx_user_visible_write() * vhdx_log_write_sectors() uses bdrv_pwrite() vs bdrv_pwrite_sync() * more direct assignment of guid rather than memcpy * use offsetof() instead of 4 Patch 9: --- nil --- === end v3 changelog === v2 changes: Incorporated Fam's review feedback - Thank you Fam for the feedback === end v2 changelog === This will allow an existing log in a VHDX image to be replayed (e.g., a VHDX image from a Hyper-V host that crashed). In addition, metadata writes are enabled through the log. This allows write support to be enabled for VHDX, as the BAT can be updated safely via the log journal. These exact patches are available from github, for testing: https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v2-upstream The latest vhdx work (including anything beyond these patches, such as backing/parent file support) can be found at: https://github.com/codyprime/qemu-kvm-jtc/tree/jtc-vhdx-latest Jeff Cody (19): block: vhdx - minor comments and typo correction. block: vhdx - add header update capability. block: vhdx code movement - VHDXMetadataEntries and BDRVVHDXState to header. block: vhdx - log support struct and defines block: vhdx - break endian translation functions out block: vhdx - update log guid in header, and first write tracker block: vhdx code movement - move vhdx_close() above vhdx_open() block: vhdx - log parsing, replay, and flush support block: vhdx - add region overlap detection for image files block: vhdx - add log write support block: vhdx write support block: vhdx - remove BAT file offset bit shifting block: vhdx - move more endian translations to vhdx-endian.c block: vhdx - break out code operations to functions block: vhdx - fix comment typos in header, fix incorrect struct fields block: vhdx - add .bdrv_create() support block: vhdx - update _make_test_img() to filter out vhdx options block: qemu-iotests for vhdx, add write test support block: vhdx qemu-iotest - log replay of data sector block/Makefile.objs | 2 +- block/vhdx-endian.c | 216 ++++ block/vhdx-log.c | 1007 ++++++++++++++++ block/vhdx.c | 1203 +++++++++++++++++--- block/vhdx.h | 178 ++- configure | 24 + tests/qemu-iotests/064 | 11 + tests/qemu-iotests/064.out | 14 + tests/qemu-iotests/065 | 67 ++ tests/qemu-iotests/065.out | 8 + tests/qemu-iotests/common | 1 - tests/qemu-iotests/common.rc | 4 +- tests/qemu-iotests/group | 1 + .../sample_images/iotest-dirtylog-10G-4M.vhdx.bz2 | Bin 0 -> 4490 bytes 14 files changed, 2552 insertions(+), 184 deletions(-) create mode 100644 block/vhdx-endian.c create mode 100644 block/vhdx-log.c create mode 100755 tests/qemu-iotests/065 create mode 100644 tests/qemu-iotests/065.out create mode 100644 tests/qemu-iotests/sample_images/iotest-dirtylog-10G-4M.vhdx.bz2 -- 1.8.3.1
