Re: [PATCH v5 0/7] [PATCH v4 0/7] Add delta islands support
Sorry, I made a copy paste error in the subject there should not be "[PATCH v4 0/7]" in it. On Thu, Aug 16, 2018 at 8:13 AM, Christian Couder wrote: > This patch series is upstreaming work made by GitHub and available in: [...]
[PATCH v5 0/7] [PATCH v4 0/7] Add delta islands support
This patch series is upstreaming work made by GitHub and available in:
https://github.com/peff/git/commits/jk/delta-islands
The above work has been already described in the following article:
https://githubengineering.com/counting-objects/
The above branch contains only one patch. In this patch series the
patch has been split into 4 patches (1/7, 3/7, 4/7 and 5/7) with their
own commit message, and on top of that 3 new patches (2/7, 6/7 and
7/7) have been added. The new patches implement things that have been
requested on the mailing list.
I kept Peff as the author of the original 4 patches and took the
liberty to add his Signed-off-by to them.
As explained in details in the "Counting Objects" article referenced
above, the goal of the delta island mechanism is for a hosting
provider to make it possible to have the "forks" of a repository share
as much storage as possible while preventing object packs to contain
deltas between different forks.
If deltas between different forks are not prevented, when users clone
or fetch a fork, preparing the pack that should be sent to them can be
very costly CPU wise, as objects from a different fork should not be
sent, which means that a lot of deltas might need to be computed
again (instead of reusing existing deltas).
The following changes have been made since the previous iteration:
* suggested by Ramsay: fix typo in "deltified" in commit message in
patch 1/7
* suggested by Ramsay and Peff: use FLEX_ARRAY when defining 'struct
island_bitmap' in delta-islands.c in patch 1/7
The diff against v4 is:
===
diff --git a/delta-islands.c b/delta-islands.c
index 2ced34d99c..8e5018e406 100644
--- a/delta-islands.c
+++ b/delta-islands.c
@@ -35,7 +35,7 @@ struct remote_island {
struct island_bitmap {
uint32_t refcount;
- uint32_t bits[];
+ uint32_t bits[FLEX_ARRAY];
};
static uint32_t island_bitmap_size;
===
This patch series is also available on GitHub in:
https://github.com/chriscool/git/commits/delta-islands
The previous versions are available there:
V4: https://github.com/chriscool/git/commits/delta-islands32
V3: https://github.com/chriscool/git/commits/delta-islands29
V2: https://github.com/chriscool/git/commits/delta-islands19
V1: https://github.com/chriscool/git/commits/delta-islands6
V4: https://public-inbox.org/git/20180812051151.6425-1-chrisc...@tuxfamily.org/
V3: https://public-inbox.org/git/20180809155532.26151-1-chrisc...@tuxfamily.org/
V2: https://public-inbox.org/git/20180805172525.15278-1-chrisc...@tuxfamily.org/
V1: https://public-inbox.org/git/20180722054836.28935-1-chrisc...@tuxfamily.org/
Christian Couder (3):
pack-objects: refactor code into compute_layer_order()
pack-objects: move tree_depth into 'struct packing_data'
pack-objects: move 'layer' into 'struct packing_data'
Jeff King (4):
Add delta-islands.{c,h}
pack-objects: add delta-islands support
repack: add delta-islands support
t: add t5319-delta-islands.sh
Documentation/config.txt | 19 ++
Documentation/git-pack-objects.txt | 97 ++
Documentation/git-repack.txt | 5 +
Makefile | 1 +
builtin/pack-objects.c | 137 +---
builtin/repack.c | 9 +
delta-islands.c| 502 +
delta-islands.h| 11 +
pack-objects.c | 12 +
pack-objects.h | 39 +++
t/t5319-delta-islands.sh | 143
11 files changed, 932 insertions(+), 43 deletions(-)
create mode 100644 delta-islands.c
create mode 100644 delta-islands.h
create mode 100755 t/t5319-delta-islands.sh
--
2.18.0.673.gcd86e60100
[PATCH v4 0/7] Add delta islands support
This patch series is upstreaming work made by GitHub and available in:
https://github.com/peff/git/commits/jk/delta-islands
The above work has been already described in the following article:
https://githubengineering.com/counting-objects/
The above branch contains only one patch. In this patch series the
patch has been split into 4 patches (1/7, 3/7, 4/7 and 5/7) with their
own commit message, and on top of that 3 new patches (2/7, 6/7 and
7/7) have been added. The new patches implement things that have been
requested on the mailing list.
I kept Peff as the author of the original 4 patches and took the
liberty to add his Signed-off-by to them.
As explained in details in the "Counting Objects" article referenced
above, the goal of the delta island mechanism is for a hosting
provider to make it possible to have the "forks" of a repository share
as much storage as possible while preventing object packs to contain
deltas between different forks.
If deltas between different forks are not prevented, when users clone
or fetch a fork, preparing the pack that should be sent to them can be
very costly CPU wise, as objects from a different fork should not be
sent, which means that a lot of deltas might need to be computed
again (instead of reusing existing deltas).
The following changes have been made since the previous iteration:
* suggested by Ramsay: remove the first patch that made
get_delta_base() in packfile.c non static as it is not necessary
* suggested by Peff: use ALLOC_GROW() in island_config_callback() in
delta-islands.c in patch 1/7
* suggested by SZEDER Gabor: use ALLOC_ARRAY() in
deduplicate_islands() in delta-islands.c in patch 1/7
This patch series is also available on GitHub in:
https://github.com/chriscool/git/commits/delta-islands
The previous versions are available there:
V3: https://github.com/chriscool/git/commits/delta-islands29
V2: https://github.com/chriscool/git/commits/delta-islands19
V1: https://github.com/chriscool/git/commits/delta-islands6
V3: https://public-inbox.org/git/20180809155532.26151-1-chrisc...@tuxfamily.org/
V2: https://public-inbox.org/git/20180805172525.15278-1-chrisc...@tuxfamily.org/
V1: https://public-inbox.org/git/20180722054836.28935-1-chrisc...@tuxfamily.org/
Christian Couder (3):
pack-objects: refactor code into compute_layer_order()
pack-objects: move tree_depth into 'struct packing_data'
pack-objects: move 'layer' into 'struct packing_data'
Jeff King (4):
Add delta-islands.{c,h}
pack-objects: add delta-islands support
repack: add delta-islands support
t: add t5319-delta-islands.sh
Documentation/config.txt | 19 ++
Documentation/git-pack-objects.txt | 97 ++
Documentation/git-repack.txt | 5 +
Makefile | 1 +
builtin/pack-objects.c | 137 +---
builtin/repack.c | 9 +
delta-islands.c| 502 +
delta-islands.h| 11 +
pack-objects.c | 12 +
pack-objects.h | 39 +++
t/t5319-delta-islands.sh | 143
11 files changed, 932 insertions(+), 43 deletions(-)
create mode 100644 delta-islands.c
create mode 100644 delta-islands.h
create mode 100755 t/t5319-delta-islands.sh
--
2.18.0.554.gdb07901040.dirty
