Re: RFC: bitmap storage for precursors and phases

2017-03-07 Thread Pierre-Yves David
On 02/18/2017 04:51 AM, Jun Wu wrote: Excerpts from Stanislau Hlebik's message of 2017-02-17 16:06:33 +: This is implementation of two caches (nonpublic + precursor) using serialized sorted lists and sets

Re: RFC: bitmap storage for precursors and phases

2017-03-07 Thread Pierre-Yves David
On 02/14/2017 02:04 AM, Sean Farley wrote: Jun Wu writes: In general, I think this is a good direction. Some random thoughts: - general purposed I think the bitmap is not always a cache, so it should only have operations like set/unset/readfromdisk/writetodisk.

Re: RFC: bitmap storage for precursors and phases

2017-02-21 Thread Jun Wu
Excerpts from Sean Farley's message of 2017-02-21 15:45:44 -0800: > Augie Fackler writes: > > > On Fri, Feb 17, 2017 at 07:14:12PM -0800, Jun Wu wrote: > >> Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > >> > I think there are multiple topics being

Re: RFC: bitmap storage for precursors and phases

2017-02-21 Thread Sean Farley
Augie Fackler writes: > On Fri, Feb 17, 2017 at 07:14:12PM -0800, Jun Wu wrote: >> Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: >> > I think there are multiple topics being discussed: >> > >> > 1. How to solve the overhead loading hiddenrevs with

Re: RFC: bitmap storage for precursors and phases

2017-02-21 Thread Sean Farley
Jun Wu writes: > Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: >> On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: >> >> > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: >> > > As I said before we will load all

Re: RFC: bitmap storage for precursors and phases

2017-02-20 Thread Stanislau Hlebik
Excerpts from Augie Fackler's message of 2017-02-19 21:06:53 -0500: > On Fri, Feb 17, 2017 at 09:59:48PM +, Stanislau Hlebik wrote: > > Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > > > On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > > > > > > >

Re: RFC: bitmap storage for precursors and phases

2017-02-19 Thread Augie Fackler
On Fri, Feb 17, 2017 at 09:59:48PM +, Stanislau Hlebik wrote: > Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > > On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > > > > > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > > > As

Re: RFC: bitmap storage for precursors and phases

2017-02-19 Thread Augie Fackler
On Fri, Feb 17, 2017 at 07:14:12PM -0800, Jun Wu wrote: > Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > > I think there are multiple topics being discussed: > > > > 1. How to solve the overhead loading hiddenrevs with minimal changes? > > 2. Why is the bitmap format

Re: RFC: bitmap storage for precursors and phases

2017-02-18 Thread Stanislau Hlebik
Excerpts from Jun Wu's message of 2017-02-17 19:14:12 -0800: > Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > > On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > > > > > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > > > As I

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Jun Wu
Excerpts from Stanislau Hlebik's message of 2017-02-17 16:06:33 +: > This is implementation of two caches (nonpublic + precursor) using > serialized sorted lists and sets > https://bitbucket.org/stashlebik/hg/commits/99879579ac2848a2567810b677d8344150a7b319?at=hiddenbitmaps_lists I had a

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Jun Wu
Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > > > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > > As I said before we will load all non-public revs in one set and all > > > >

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Stanislau Hlebik
Excerpts from Bryan O'Sullivan's message of 2017-02-17 13:29:58 -0800: > On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > > > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > > As I said before we will load all non-public revs in one set and all > > > >

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Bryan O'Sullivan
On Fri, Feb 17, 2017 at 10:30 AM, Jun Wu wrote: > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > As I said before we will load all non-public revs in one set and all > > The problem is, loading a Python set from disk is O(size-of-the-set). > > Bitmap's

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Stanislau Hlebik
Excerpts from Jun Wu's message of 2017-02-17 10:30:45 -0800: > Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > > As I said before we will load all non-public revs in one set and all > > The problem is, loading a Python set from disk is O(size-of-the-set). > > Bitmap's

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Jun Wu
Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > As I said before we will load all non-public revs in one set and all The problem is, loading a Python set from disk is O(size-of-the-set). Bitmap's loading cost should be basically 0 (with mmap). I think that's why we want

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Stanislau Hlebik
Excerpts from Stanislau Hlebik's message of 2017-02-17 11:24:34 +: > Excerpts from Jun Wu's message of 2017-02-16 13:42:46 -0800: > > Excerpts from Stanislau Hlebik's message of 2017-02-16 19:39:07 +: > > > Excerpts from Stanislau Hlebik's message of 2017-02-14 09:29:25 +: > > > >

Re: RFC: bitmap storage for precursors and phases

2017-02-17 Thread Stanislau Hlebik
Excerpts from Jun Wu's message of 2017-02-16 13:42:46 -0800: > Excerpts from Stanislau Hlebik's message of 2017-02-16 19:39:07 +: > > Excerpts from Stanislau Hlebik's message of 2017-02-14 09:29:25 +: > > > Excerpts from Sean Farley's message of 2017-02-13 18:30:25 -0800: > > > > Jun Wu

Re: RFC: bitmap storage for precursors and phases

2017-02-16 Thread Jun Wu
Excerpts from Stanislau Hlebik's message of 2017-02-16 19:39:07 +: > Excerpts from Stanislau Hlebik's message of 2017-02-14 09:29:25 +: > > Excerpts from Sean Farley's message of 2017-02-13 18:30:25 -0800: > > > Jun Wu writes: > > > > > > > Excerpts from Sean Farley's

Re: RFC: bitmap storage for precursors and phases

2017-02-16 Thread Stanislau Hlebik
Excerpts from Stanislau Hlebik's message of 2017-02-14 09:29:25 +: > Excerpts from Sean Farley's message of 2017-02-13 18:30:25 -0800: > > Jun Wu writes: > > > > > Excerpts from Sean Farley's message of 2017-02-13 17:04:35 -0800: > > >> I was thinking about a more high-level

Re: RFC: bitmap storage for precursors and phases

2017-02-14 Thread Stanislau Hlebik
Excerpts from Gregory Szorc's message of 2017-02-13 20:28:37 -0700: > > > On Feb 10, 2017, at 10:33, Stanislau Hlebik wrote: > > > > Last year Durham sent a proposal for bitmap storage for computehidden() > >

Re: RFC: bitmap storage for precursors and phases

2017-02-14 Thread Stanislau Hlebik
Excerpts from Sean Farley's message of 2017-02-13 18:30:25 -0800: > Jun Wu writes: > > > Excerpts from Sean Farley's message of 2017-02-13 17:04:35 -0800: > >> I was thinking about a more high-level approach (please feel free to > >> pick apart): > >> > >> r =

Re: RFC: bitmap storage for precursors and phases

2017-02-13 Thread Gregory Szorc
> On Feb 10, 2017, at 10:33, Stanislau Hlebik wrote: > > Last year Durham sent a proposal for bitmap storage for computehidden() > https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-September/087860.html. > > It got a few useful comments, two most important comments:

Re: RFC: bitmap storage for precursors and phases

2017-02-13 Thread Sean Farley
Jun Wu writes: > Excerpts from Sean Farley's message of 2017-02-13 17:04:35 -0800: >> I was thinking about a more high-level approach (please feel free to >> pick apart): >> >> r = repo.filtered("bitmap1") >> r2 = r.filtered("bitmap2") >> >> So that r2 would be an intersection of

Re: RFC: bitmap storage for precursors and phases

2017-02-13 Thread Jun Wu
Excerpts from Sean Farley's message of 2017-02-13 17:04:35 -0800: > I was thinking about a more high-level approach (please feel free to > pick apart): > > r = repo.filtered("bitmap1") > r2 = r.filtered("bitmap2") > > So that r2 would be an intersection of bitmap1 and bitmap2 (haven't > thought

Re: RFC: bitmap storage for precursors and phases

2017-02-13 Thread Sean Farley
Jun Wu writes: > In general, I think this is a good direction. Some random thoughts: > > - general purposed > > I think the bitmap is not always a cache, so it should only have > operations like set/unset/readfromdisk/writetodisk. Practically, I won't > couple cache

Re: RFC: bitmap storage for precursors and phases

2017-02-13 Thread Stanislau Hlebik
Excerpts from Jun Wu's message of 2017-02-10 11:04:24 -0800: > In general, I think this is a good direction. Some random thoughts: > > - general purposed > > I think the bitmap is not always a cache, so it should only have > operations like set/unset/readfromdisk/writetodisk.

Re: RFC: bitmap storage for precursors and phases

2017-02-10 Thread Jun Wu
In general, I think this is a good direction. Some random thoughts: - general purposed I think the bitmap is not always a cache, so it should only have operations like set/unset/readfromdisk/writetodisk. Practically, I won't couple cache invalidation with the bitmap implementation.

RFC: bitmap storage for precursors and phases

2017-02-10 Thread Stanislau Hlebik
Last year Durham sent a proposal for bitmap storage for computehidden() https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-September/087860.html. It got a few useful comments, two most important comments: 1. Use bitmaps for lower-level data structures, for example, bitmap for