Reviewed by: Alex Reece <[email protected]> Reviewed by: George Wilson <[email protected]> Reviewed by: John Kennedy <[email protected]> Reviewed by: Prakash Surya <[email protected]>
This project allows top-level vdevs to be removed from the storage pool with "zpool remove", reducing the total amount of storage in the pool. This operation copies all allocated regions of the device to be removed onto other devices, recording the mapping from old to new location. After the removal is complete, read and free operations to the removed (now "indirect") vdev must be remapped and performed at the new location on disk. The indirect mapping table is kept in memory whenever the pool is loaded, so there is minimal performance overhead when doing operations on the indirect vdev. The size of the in-memory mapping table will be reduced when its entries become "obsolete" because they are no longer used by any block pointers in the pool. An entry becomes obsolete when all the blocks that use it are freed. An entry can also become obsolete when all the snapshots that reference it are deleted, and the block pointers that reference it have been "remapped" in all filesystems/zvols (and clones). Whenever an indirect block is written, all the block pointers in it will be "remapped" to their new (concrete) locations if possible. This process can be accelerated by using the "zfs remap" command to proactively rewrite all indirect blocks that reference indirect (removed) vdevs. Note that when a device is removed, we do not verify the checksum of the data that is copied. This makes the process much faster, but if it were used on raidz vdevs, it would be possible to copy the wrong data, when we have the correct data on e.g. the other side of the raidz. Therefore, raidz devices can not be removed. You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/482 -- Commit Summary -- * 7614 zfs device evacuation/removal -- File Changes -- M usr/src/cmd/mdb/common/modules/zfs/zfs.c (24) M usr/src/cmd/truss/codes.c (2) M usr/src/cmd/zdb/zdb.c (696) M usr/src/cmd/zfs/zfs_main.c (25) M usr/src/cmd/zpool/zpool_main.c (171) M usr/src/cmd/ztest/ztest.c (91) M usr/src/common/zfs/zfeature_common.c (16) M usr/src/common/zfs/zfeature_common.h (2) M usr/src/common/zfs/zfs_deleg.c (3) M usr/src/common/zfs/zfs_deleg.h (3) M usr/src/common/zfs/zfs_prop.c (2) M usr/src/lib/libzfs/common/libzfs.h (5) M usr/src/lib/libzfs/common/libzfs_dataset.c (18) M usr/src/lib/libzfs/common/libzfs_pool.c (68) M usr/src/lib/libzfs/common/libzfs_util.c (7) M usr/src/lib/libzfs/common/mapfile-vers (4) M usr/src/lib/libzfs_core/common/libzfs_core.c (10) M usr/src/lib/libzfs_core/common/libzfs_core.h (1) M usr/src/lib/libzfs_core/common/mapfile-vers (1) M usr/src/man/man1m/zfs.1m (13) M usr/src/man/man1m/zpool.1m (56) M usr/src/man/man5/zpool-features.5 (45) M usr/src/pkg/manifests/system-test-zfstest.mf (37) M usr/src/test/test-runner/cmd/run (3) M usr/src/test/zfs-tests/include/libtest.shlib (57) M usr/src/test/zfs-tests/runfiles/delphix.run (12) A usr/src/test/zfs-tests/runfiles/stress.run (23) M usr/src/test/zfs-tests/tests/functional/bootfs/bootfs_001_pos.ksh (4) M usr/src/test/zfs-tests/tests/functional/bootfs/bootfs_002_neg.ksh (4) M usr/src/test/zfs-tests/tests/functional/cli_root/zpool_get/zpool_get.cfg (3) M usr/src/test/zfs-tests/tests/functional/features/async_destroy/async_destroy_001_pos.ksh (27) A usr/src/test/zfs-tests/tests/functional/removal/Makefile (64) A usr/src/test/zfs-tests/tests/functional/removal/cleanup.ksh (23) A usr/src/test/zfs-tests/tests/functional/removal/removal.kshlib (155) A usr/src/test/zfs-tests/tests/functional/removal/removal_all_vdev.ksh (39) A usr/src/test/zfs-tests/tests/functional/removal/removal_check_space.ksh (44) A usr/src/test/zfs-tests/tests/functional/removal/removal_condense_export.ksh (86) A usr/src/test/zfs-tests/tests/functional/removal/removal_multiple_indirection.ksh (93) A usr/src/test/zfs-tests/tests/functional/removal/removal_remap.ksh (123) A usr/src/test/zfs-tests/tests/functional/removal/removal_remap_deadlists.ksh (73) A usr/src/test/zfs-tests/tests/functional/removal/removal_reservation.ksh (68) A usr/src/test/zfs-tests/tests/functional/removal/removal_sanity.ksh (39) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_add.ksh (52) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_create_fs.ksh (39) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_dedup.ksh (49) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_export.ksh (49) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_ganging.ksh (53) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_remap.ksh (38) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_remove.ksh (38) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_scrub.ksh (38) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_send.ksh (40) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_send_recv.ksh (41) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_snapshot.ksh (39) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_write.ksh (34) A usr/src/test/zfs-tests/tests/functional/removal/removal_with_zdb.ksh (57) A usr/src/test/zfs-tests/tests/functional/removal/remove_mirror.ksh (48) A usr/src/test/zfs-tests/tests/functional/removal/remove_raidz.ksh (48) A usr/src/test/zfs-tests/tests/stress/races/Makefile (39) A usr/src/test/zfs-tests/tests/stress/races/remove_file_while_remap.ksh (75) M usr/src/uts/common/Makefile.files (4) M usr/src/uts/common/fs/zfs/arc.c (2) M usr/src/uts/common/fs/zfs/bpobj.c (28) M usr/src/uts/common/fs/zfs/dbuf.c (138) M usr/src/uts/common/fs/zfs/ddt.c (8) M usr/src/uts/common/fs/zfs/dmu.c (124) M usr/src/uts/common/fs/zfs/dmu_objset.c (107) M usr/src/uts/common/fs/zfs/dmu_tx.c (17) M usr/src/uts/common/fs/zfs/dnode.c (3) M usr/src/uts/common/fs/zfs/dsl_dataset.c (229) M usr/src/uts/common/fs/zfs/dsl_deadlist.c (19) M usr/src/uts/common/fs/zfs/dsl_destroy.c (54) M usr/src/uts/common/fs/zfs/dsl_dir.c (50) M usr/src/uts/common/fs/zfs/dsl_pool.c (64) M usr/src/uts/common/fs/zfs/dsl_scan.c (71) M usr/src/uts/common/fs/zfs/metaslab.c (629) M usr/src/uts/common/fs/zfs/range_tree.c (18) M usr/src/uts/common/fs/zfs/spa.c (590) M usr/src/uts/common/fs/zfs/spa_config.c (12) M usr/src/uts/common/fs/zfs/spa_misc.c (75) M usr/src/uts/common/fs/zfs/space_map.c (136) M usr/src/uts/common/fs/zfs/space_reftree.c (4) M usr/src/uts/common/fs/zfs/sys/bpobj.h (4) M usr/src/uts/common/fs/zfs/sys/dbuf.h (2) M usr/src/uts/common/fs/zfs/sys/dmu.h (11) M usr/src/uts/common/fs/zfs/sys/dnode.h (1) M usr/src/uts/common/fs/zfs/sys/dsl_dataset.h (30) M usr/src/uts/common/fs/zfs/sys/dsl_deadlist.h (2) M usr/src/uts/common/fs/zfs/sys/dsl_deleg.h (3) M usr/src/uts/common/fs/zfs/sys/dsl_dir.h (4) M usr/src/uts/common/fs/zfs/sys/dsl_pool.h (5) M usr/src/uts/common/fs/zfs/sys/dsl_scan.h (5) M usr/src/uts/common/fs/zfs/sys/metaslab.h (7) M usr/src/uts/common/fs/zfs/sys/metaslab_impl.h (16) M usr/src/uts/common/fs/zfs/sys/range_tree.h (9) M usr/src/uts/common/fs/zfs/sys/spa.h (14) M usr/src/uts/common/fs/zfs/sys/spa_impl.h (71) M usr/src/uts/common/fs/zfs/sys/space_map.h (13) M usr/src/uts/common/fs/zfs/sys/vdev.h (10) M usr/src/uts/common/fs/zfs/sys/vdev_impl.h (95) A usr/src/uts/common/fs/zfs/sys/vdev_indirect_births.h (80) A usr/src/uts/common/fs/zfs/sys/vdev_indirect_mapping.h (141) A usr/src/uts/common/fs/zfs/sys/vdev_removal.h (93) M usr/src/uts/common/fs/zfs/sys/zfs_debug.h (3) M usr/src/uts/common/fs/zfs/sys/zil.h (2) M usr/src/uts/common/fs/zfs/sys/zio.h (2) M usr/src/uts/common/fs/zfs/sys/zio_priority.h (1) M usr/src/uts/common/fs/zfs/txg.c (2) M usr/src/uts/common/fs/zfs/vdev.c (324) M usr/src/uts/common/fs/zfs/vdev_disk.c (1) M usr/src/uts/common/fs/zfs/vdev_file.c (2) A usr/src/uts/common/fs/zfs/vdev_indirect.c (1035) A usr/src/uts/common/fs/zfs/vdev_indirect_births.c (212) A usr/src/uts/common/fs/zfs/vdev_indirect_mapping.c (594) M usr/src/uts/common/fs/zfs/vdev_label.c (87) M usr/src/uts/common/fs/zfs/vdev_mirror.c (3) M usr/src/uts/common/fs/zfs/vdev_missing.c (2) M usr/src/uts/common/fs/zfs/vdev_queue.c (18) M usr/src/uts/common/fs/zfs/vdev_raidz.c (1) A usr/src/uts/common/fs/zfs/vdev_removal.c (1896) M usr/src/uts/common/fs/zfs/vdev_root.c (3) M usr/src/uts/common/fs/zfs/zcp_get.c (3) M usr/src/uts/common/fs/zfs/zfs_ioctl.c (36) M usr/src/uts/common/fs/zfs/zfs_vnops.c (12) M usr/src/uts/common/fs/zfs/zil.c (2) M usr/src/uts/common/fs/zfs/zio.c (42) M usr/src/uts/common/sys/fs/zfs.h (32) -- Patch Links -- https://github.com/openzfs/openzfs/pull/482.patch https://github.com/openzfs/openzfs/pull/482.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/482 ------------------------------------------ openzfs-developer Archives: https://openzfs.topicbox.com/groups/developer/discussions/Teafc5ffabd7f916f-M7b0f861bb6169a7d87c79137 Powered by Topicbox: https://topicbox.com
