Re: Reimagining notmuch-git/nmbug

2023-04-06 Thread David Bremner
Felipe Contreras writes: > On Tue, Apr 4, 2023 at 12:54 PM David Bremner wrote: >> >> This sounds right. Can we use the detection of missing messages in >> wr_export to reset the appropriate counters? It looks like yes, given >> the call to store_lastmod. [snip] > I would rather go for a

Re: Reimagining notmuch-git/nmbug

2023-04-05 Thread Felipe Contreras
On Tue, Apr 4, 2023 at 12:54 PM David Bremner wrote: > > Felipe Contreras writes: > > > On Mon, Apr 3, 2023 at 6:37 PM David Bremner wrote: > > > Or we could say that after jumping a certain threshold of lastmod we > > delete all the messages and start from scratch, perhaps every 1000 > >

Re: Reimagining notmuch-git/nmbug

2023-04-04 Thread David Bremner
Felipe Contreras writes: > On Mon, Apr 3, 2023 at 6:37 PM David Bremner wrote: > Or we could say that after jumping a certain threshold of lastmod we > delete all the messages and start from scratch, perhaps every 1000 > revisions. > > Or maybe the query could generate a virtual tag if a

Re: Reimagining notmuch-git/nmbug

2023-04-04 Thread David Bremner
Felipe Contreras writes: > > I'm not familiar with git-annex, I would need to see an example of > such merging happening. I was confused, git-annex is using the builtin merge strategy "union", which is not eliminating duplicates or sorting, so probably not applicable here. I still have to try

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread Felipe Contreras
On Mon, Apr 3, 2023 at 5:46 AM David Bremner wrote: > > David Bremner writes: > > > > > I'm intrigued (and indeed I hadn't really thought about the degree to > > which we were re-inventing git-fast-import and friends); however so far > > my experiments did not get far enough to say anything

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread Felipe Contreras
On Mon, Apr 3, 2023 at 6:37 PM David Bremner wrote: > > Felipe Contreras writes: > > > > > That should work to update existing tags, but how are we going to > > detect if a message has disappeared? Or is that not a thing? > > Indeed the same thought had occurred to me not long ago. I remembered

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread David Bremner
Felipe Contreras writes: > > That should work to update existing tags, but how are we going to > detect if a message has disappeared? Or is that not a thing? Indeed the same thought had occurred to me not long ago. I remembered (belately) that I'd been through some similar thought process with

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread Felipe Contreras
On Mon, Apr 3, 2023 at 2:40 PM David Bremner wrote: > > David Bremner writes: > > > Indeed that speeds up the initial clone on this machine from 39 minutes > > (I switched machines) to 30s. I will play with it a bit more, and report > > back. > > It's not a showstopper, but "git pull" takes

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread David Bremner
David Bremner writes: > Indeed that speeds up the initial clone on this machine from 39 minutes > (I switched machines) to 30s. I will play with it a bit more, and report > back. It's not a showstopper, but "git pull" takes about 1/2 the wall time (about 2/3 of the CPU time) of the original

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread David Bremner
Felipe Contreras writes: > By distributing the files in multiple directories like notmuch-git > does using BLAKE2b, the operation is much faster. > > I've pushed the changes, now there's a dependency, but you can just > `gem install blake2b`. > > I'm able to clone the database of the performance

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread Felipe Contreras
On Mon, Apr 3, 2023 at 4:49 AM David Bremner wrote: > Performance-wise the initial clone seems pretty slow. For my 600k > messages I have been waiting a while now. htop tells me that > git-fast-import has about 45 minutes of CPU time at this point. This > machine is not that fast, but for

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread Felipe Contreras
On Mon, Apr 3, 2023 at 4:49 AM David Bremner wrote: > > Felipe Contreras writes: > > > Hi, > > > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > > > Very quickly I realized that most of what it does is something I've > > been working on for at least 10 years:

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread David Bremner
David Bremner writes: > > I'm intrigued (and indeed I hadn't really thought about the degree to > which we were re-inventing git-fast-import and friends); however so far > my experiments did not get far enough to say anything conclusive. > I did manage to finish, about 70 minutes elapsed.

Re: Reimagining notmuch-git/nmbug

2023-04-03 Thread David Bremner
Felipe Contreras writes: > Hi, > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > Very quickly I realized that most of what it does is something I've > been working on for at least 10 years: making git work with other > tools. > > I presume you haven't heard of git

Re: Reimagining notmuch-git/nmbug

2023-03-29 Thread Felipe Contreras
On Wed, Mar 29, 2023 at 3:50 AM Michael J Gruber wrote: > > Am Mi., 29. März 2023 um 10:41 Uhr schrieb Felipe Contreras > : > > > > Hi, > > > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > > > Very quickly I realized that most of what it does is something I've > >

Re: Reimagining notmuch-git/nmbug

2023-03-29 Thread Michael J Gruber
Am Mi., 29. März 2023 um 10:41 Uhr schrieb Felipe Contreras : > > Hi, > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > Very quickly I realized that most of what it does is something I've > been working on for at least 10 years: making git work with other > tools. >

Reimagining notmuch-git/nmbug

2023-03-29 Thread Felipe Contreras
Hi, I noticed you promoted notmuch-git as a user tool to toy around with it. Very quickly I realized that most of what it does is something I've been working on for at least 10 years: making git work with other tools. I presume you haven't heard of git remote-helpers [1], because they do