Re: RFC: bitmap storage for hidden

2016-09-11 Thread Jun Wu
Excerpts from Pierre-Yves David's message of 2016-09-09 19:25:27 +0200: > I'm still confused, the way we store hidden-ness and the way we compute > it are two different things. The current API definitely allow for other > way to marks things hidden. Sean even successfully implemented >

Re: RFC: bitmap storage for hidden

2016-09-06 Thread Durham Goode
On 9/6/16 10:06 AM, Pierre-Yves David wrote: On 08/31/2016 10:53 PM, Durham Goode wrote: One of the performance costs that affects every command is the computehidden function (which computes what commits are hidden based on a combination of obsmarkers, bookmarks, workingcopy, phases, and

Re: RFC: bitmap storage for hidden

2016-09-06 Thread Pierre-Yves David
On 08/31/2016 10:53 PM, Durham Goode wrote: One of the performance costs that affects every command is the computehidden function (which computes what commits are hidden based on a combination of obsmarkers, bookmarks, workingcopy, phases, and tags). At Facebook this function alone can add

Re: RFC: bitmap storage for hidden

2016-09-05 Thread Durham Goode
I agree that an optimized bitmap can come after the initial implementation. I'm more concerned with getting this rigged into mercurial correctly and with appropriate cache mechanics than I am about shrinking the 100k. On 9/2/16 2:05 PM, Jun Wu wrote: Given the fact that the plain bitmap is

Re: RFC: bitmap storage for hidden

2016-09-02 Thread Durham Goode
On 9/2/16 9:14 AM, Gregory Szorc wrote: On Aug 31, 2016, at 13:53, Durham Goode wrote: One of the performance costs that affects every command is the computehidden function (which computes what commits are hidden based on a combination of obsmarkers, bookmarks,

Re: RFC: bitmap storage for hidden

2016-09-02 Thread Gregory Szorc
> On Aug 31, 2016, at 13:53, Durham Goode wrote: > > One of the performance costs that affects every command is the computehidden > function (which computes what commits are hidden based on a combination of > obsmarkers, bookmarks, workingcopy, phases, and tags). At Facebook

Re: RFC: bitmap storage for hidden

2016-09-02 Thread Bryan O'Sullivan
I agree that it would make a lot of sense to not use a plain bitmap. The current state of the art for compressed high-performance bitmaps is here: http://roaringbitmap.org/ On Fri, Sep 2, 2016 at 8:09 AM, Augie Fackler wrote: > I've considered doing a bitmap index for hidden

Re: RFC: bitmap storage for hidden

2016-09-02 Thread Augie Fackler
> On Aug 31, 2016, at 16:53, Durham Goode wrote: > > One of the performance costs that affects every command is the computehidden > function (which computes what commits are hidden based on a combination of > obsmarkers, bookmarks, workingcopy, phases, and tags). At Facebook

RFC: bitmap storage for hidden

2016-08-31 Thread Durham Goode
One of the performance costs that affects every command is the computehidden function (which computes what commits are hidden based on a combination of obsmarkers, bookmarks, workingcopy, phases, and tags). At Facebook this function alone can add 300ms to every command a user runs (depending