v2: including new implementation of cluster snapshot Current way of doing cluster snapshot is very powerful and has great performance. However it also has some drawbacks: 1. After a new node joins the cluster, all former snapshots are not available to be restored. 2. It is hard to backup cluster snapshot to an other storage system for disaster-tolerance. 3. It is hard to init a new cluster by loading another cluster's snapshot.
New idea is to move "farm" from sheep to collie and save cluster snapshot to a localpath. New cluster snapshot retains all features from "farm", including: 1. object de-duplication 2. incremental store capability In addition, it also provides ability of: 1. export cluster snapshot to other storage device for backup and disaster-tolerance 2. deploy new cluster by restore from one snapshot of other cluster The first 4 patches are used to remove farm from sheep. The others are used to re-implement farm under collie. Kai Zhang (12): sheep: change default store driver from "farm" to "plain" collie: remove snapshot from cluster subcommand script: remove script/simple2farm sheep: remove farm from sheep collie/farm: implement object_rb_tree collie/farm: implement sha1_file collie/farm: implement snap object collie/farm: impelement trunk object sheep: add a flag to let notify_vdi_add set bitmap if needed collie/farm: implement farm collie: implement "collie cluster snapshot" subcommand test: add tests/030 for cluster snapshot collie/Makefile.am | 4 +- collie/cluster.c | 154 ++++++++++++++---------- collie/farm/farm.c | 252 ++++++++++++++++++++++++++++++++++++++ collie/farm/farm.h | 84 +++++++++++++ collie/farm/object_rb_tree.c | 148 +++++++++++++++++++++++ collie/farm/sha1_file.c | 273 ++++++++++++++++++++++++++++++++++++++++++ collie/farm/snap.c | 150 +++++++++++++++++++++++ collie/farm/trunk.c | 75 ++++++++++++ include/sheepdog_proto.h | 7 +- script/simple2farm | 51 -------- sheep/Makefile.am | 10 +- sheep/farm/farm.c | 271 ----------------------------------------- sheep/farm/farm.h | 68 ----------- sheep/farm/sha1_file.c | 273 ------------------------------------------ sheep/farm/snap.c | 152 ----------------------- sheep/farm/trunk.c | 139 --------------------- sheep/ops.c | 65 +---------- sheep/sheep_priv.h | 6 +- sheep/store.c | 1 - sheep/vdi.c | 1 + tests/001.out | 2 +- tests/002.out | 2 +- tests/003.out | 2 +- tests/004.out | 2 +- tests/005.out | 2 +- tests/006.out | 2 +- tests/007.out | 4 +- tests/008.out | 2 +- tests/009.out | 2 +- tests/010.out | 2 +- tests/013.out | 2 +- tests/014.out | 2 +- tests/015.out | 2 +- tests/016.out | 2 +- tests/017.out | 2 +- tests/018.out | 2 +- tests/019.out | 2 +- tests/020.out | 2 +- tests/021.out | 2 +- tests/022.out | 2 +- tests/023.out | 2 +- tests/024.out | 2 +- tests/025.out | 2 +- tests/026.out | 2 +- tests/027.out | 2 +- tests/028.out | 2 +- tests/029.out | 2 +- tests/030 | 77 +++++++++--- tests/030.out | 20 +++- tests/031.out | 2 +- tests/032.out | 2 +- tests/033.out | 2 +- tests/034.out | 2 +- tests/035.out | 2 +- tests/036.out | 2 +- tests/037.out | 2 +- tests/038.out | 2 +- tests/039.out | 2 +- tests/040.out | 2 +- tests/041.out | 2 +- tests/042.out | 2 +- tests/043.out | 2 +- tests/044.out | 2 +- tests/045.out | 2 +- tests/046.out | 2 +- tests/047.out | 2 +- tests/048.out | 2 +- tests/049.out | 2 +- tests/050.out | 2 +- tests/051.out | 2 +- tests/052.out | 2 +- tests/053.out | 2 +- tests/054.out | 2 +- tests/055.out | 2 +- tests/056.out | 2 +- tests/057.out | 2 +- tests/058.out | 2 +- tests/059.out | 2 +- 78 files changed, 1223 insertions(+), 1172 deletions(-) create mode 100644 collie/farm/farm.c create mode 100644 collie/farm/farm.h create mode 100644 collie/farm/object_rb_tree.c create mode 100644 collie/farm/sha1_file.c create mode 100644 collie/farm/snap.c create mode 100644 collie/farm/trunk.c delete mode 100755 script/simple2farm delete mode 100644 sheep/farm/farm.c delete mode 100644 sheep/farm/farm.h delete mode 100644 sheep/farm/sha1_file.c delete mode 100644 sheep/farm/snap.c delete mode 100644 sheep/farm/trunk.c -- sheepdog mailing list [email protected] http://lists.wpkg.org/mailman/listinfo/sheepdog
