Re: public-inbox v1.7 update (was: Updating mumi on berlin)

[zimoun]

Hi Kyle,

Thanks for the details and explanations.


On Sun, 08 May 2022 at 00:41, Kyle Meyer  wrote:

> I'd eventually like to be able to follow list with lei via piem's
> interface, but things aren't there yet.

All in all, I would like that the public-inbox format would be indexed
by Notmuch [1] :-) Instead of rely on conversion forth and back.



Cheers,
simon

1: 

Re: public-inbox v1.7 update (was: Updating mumi on berlin)

[Kyle Meyer]

zimoun writes:

> [...]
> Well, using the plain Git repo, it is easy to:
>
>  1. get messages from a list starting at a date;
> using ’git clone --mirror --shallow-since=’
>
>  2. get all the new messages;
> (using ’git pull)
>[...]
> IIUC, ’lei’ avoid this manual dance with the Git repo and do it for me.

It depends.  Yes, you can use it in a way that avoids needing to clone
down repos and index them; in that case you're relying on the remote
instance.

Or you can clone and index the repos yourself and point lei at them.
You can do this with `lei add-external --mirror', but you still have to
update the repos yourself (see example at
).

If you go the clone/index route, however, I'd recommend using grokmirror
to fetch Git repositories from public-inbox instances.  (In
https://issues.guix.gnu.org/43637#1 <878scww903@kyleam.com/>, I
provided an example config that mirrors Guix-related inboxes at
yhetil.org.)

> Well, what I miss is the lei “saved search” query for all messages.
> Other said,
>
> lei q d:..
> lei up
>
> fits #1.  Then, how do I achieve #2?

For remote externals, `lei up' will pull in updates from the remote.
For local externals (things you've cloned down and indexed yourself),
you need to do something to pull in those update (e.g., a grok-pull or a
`git fetch').

> The workflow using “saved search” is not clear for me.  Before investing
> some time, especially when ’lei’ is not packaged in Guix, I would like
> to be sure about how to run «my workflow».

I don't regularly used saved searches myself, so I don't know that I can
be much help, even if I had a clear picture of what you wanted in your
workflow.

Fwiw my setup is:

 1) use nntp via Gnus to skim new messages on mailing lists

 2) if I want to participate in a thread, use
piem-inject-thread-into-maildir to get the messages into my local
mail store and my Notmuch index.

 3) If I want to search for messages, I use lei-q via piem-lei-q.

Using grokmirror and a configured pull.post_update_hook, I fetch
updates for and index a subset yhetil.org inboxes that I want to
have available locally for searching.

(I use piem-inject-thread-into-maildir here too.)

I'd eventually like to be able to follow list with lei via piem's
interface, but things aren't there yet.

Re: public-inbox v1.7 update (was: Updating mumi on berlin)

[zimoun]

Hi Kyle,

Thanks for the answer.


On Thu, 05 May 2022 at 22:24, Kyle Meyer  wrote:
> zimoun writes:

>> How do you query all new ones for a specific list?
>
> Rather than pass --all, you can call `lei up OUTPUT', where OUTPUT is a
> particular saved search generated by `lei q', so you could have a saved
> search that's specific for a list or set of lists:
>
>$ lei ls-external | grep guix
>/home/kyle/inboxes/guix-bugs boost=0
>/home/kyle/inboxes/guix-devel boost=0
>/home/kyle/inboxes/guix-patches boost=0
>/home/kyle/inboxes/guix-science boost=0
>/home/kyle/inboxes/guix-user boost=0
>
>$ lei q -I 'guix-*' -o /tmp/zimoun-on-guix d:20.days.ago.. f:zimoun
>$ lei up /tmp/zimoun-on-guix
>
> See https://public-inbox.org/lei-up.txt and
> https://public-inbox.org/lei-q.txt

For people considering the volume of the lists too much, the “saved
search” is a perfect mechanism.  For instance, it is a nice improvement
to follow guix-patches, IMHO.

However, considering guix-science, for example, the volume is not too
high, and the “saved search” would be: all the messages.

Using the public-inbox Git repo, I just run “guix fetch”, and somehow
the commits in the range master..origin/master are all the new ones.
I can convert them to maildir or whatever.

Well, using the plain Git repo, it is easy to:

 1. get messages from a list starting at a date;
using ’git clone --mirror --shallow-since=’

 2. get all the new messages;
(using ’git pull)

Git becoming the way to transport the information.

However, I have to choose the storage format (Git vs Maildir) to avoid
unnecessary duplication on my poor laptop.  I use Maildir because I can
easily index and search (notmuch) and read (emacs) locally; by locally,
I mean when I am offline.  Therefore, I have to run “git gc” and prune
already imported messages.

IIUC, ’lei’ avoid this manual dance with the Git repo and do it for me.

Well, what I miss is the lei “saved search” query for all messages.
Other said,

lei q d:..
lei up

fits #1.  Then, how do I achieve #2?

The workflow using “saved search” is not clear for me.  Before investing
some time, especially when ’lei’ is not packaged in Guix, I would like
to be sure about how to run «my workflow».


Cheers,
simon

public-inbox v1.7 update (was: Updating mumi on berlin)

[Kyle Meyer]

zimoun writes:

> …why not just disable the specific test or if not possible, turn off the
> test suite ’#:tests? #f’.  Whereas it is not the best, it would allow to
> have ’lei’ while waiting the fix at the Guix build environment level.
>
> WDYT?

public-inbox has a good test suite, so it'd be nice to keep most of it
wired up.  No objections from me about disabling the lei tests (or a
subset of them), though I haven't looked into it, so perhaps it's tricky
to do cleanly.

>> $ guix shell -p path/to/profile -- lei up --all
>
> How do you query all new ones for a specific list?

Rather than pass --all, you can call `lei up OUTPUT', where OUTPUT is a
particular saved search generated by `lei q', so you could have a saved
search that's specific for a list or set of lists:

   $ lei ls-external | grep guix
   /home/kyle/inboxes/guix-bugs boost=0
   /home/kyle/inboxes/guix-devel boost=0
   /home/kyle/inboxes/guix-patches boost=0
   /home/kyle/inboxes/guix-science boost=0
   /home/kyle/inboxes/guix-user boost=0

   $ lei q -I 'guix-*' -o /tmp/zimoun-on-guix d:20.days.ago.. f:zimoun
   $ lei up /tmp/zimoun-on-guix

See https://public-inbox.org/lei-up.txt and
https://public-inbox.org/lei-q.txt

> Kyle, the function ’piem-inject-thread-into-maildir’ [2] is really
> handy.  But I would like this signature instead:
>
> --8<---cut here---start->8---
>   (defun piem-inject-thread-into-maildir (mid  inbox message-only)
> --8<---cut here---end--->8---
> [...]

Without thinking too much about it, that sounds fine to me.  I'll plan
to send a patch to piem's inbox with you cc'd within the next week or
two.