Re: New mailing list archive at https://orgmode/list/

2020-06-19 Thread Eric Wong
Kyle Meyer  wrote:
> Eric Abrahamsen writes:
> 
> > Hey, that works great! It's a bit weird that it still asks for a
> > username and password, I wonder if there's any way to skip that. I've
> > never dealt with anonymous IMAP before -- is there anything in the
> > connection process that explicitly tells us "you don't need to log on"?
> 
> The server advertises AUTH=ANONYMOUS as a capability [*], so Gnus could
> detect that and send "AUTHENTICATE ANONYMOUS", I _think_.

Fwiw, mutt detects AUTH=ANONYMOUS and uses it automatically,
so I think it's reasonable for Gnus and others do the same.

> [*] https://public-inbox.org/meta/20200610070519.18252-...@yhbt.net/



Re: New mailing list archive at https://orgmode/list/

2020-06-08 Thread Eric Wong
Eric Abrahamsen  wrote:
> Kyle Meyer  writes:
> 
> > [ +cc Eric Wong, mostly to say thanks for all the work he puts into
> >   public-inbox, which is the software behind these archives, but also so
> >   that he can correct me if I misrepresent any capabilities of or plans
> >   for public-inbox ]
> 
> Thanks for this response, Kyle (and thanks for public-inbox, Eric)!

you're welcome, both :>

> You wouldn't really use one backend (nnweb) to provide search support
> for another (nntp). nnir can assign different search engines to
> different backends -- what a "search engine" boils down to is a function
> that accepts group search criteria, and returns groups and article
> numbers (and optional relevance scoring) for matching messages. So if
> public-inbox had some sort of an API that accepted a query and returned
> the above information in some sort of easily-digestible format, it
> wouldn't be hard to write a engine for it. Articles referenced in the
> search results would then be retrieved via NNTP, so the article numbers
> would need to correspond.

Fwiw, I've been trying to avoid exposing NNTP article numbers in
the HTTP endpoint in favor of Message-IDs because serial numbers
aren't decentralization-friendly.  Of course, sometimes
Message-IDs get reused, so public-inbox will return all messages
which match a particular Message-ID in those rare cases.

Btw, POST with the "=m" query parameter already allows search
to return a gzipped mboxrd.

And also what I just wrote about about JMAP/GraphQL in the other
message.

A read-only IMAP server is also coming with search support,
and IMAP UIDs will be equivalent to NNTP article numbers.



Re: New mailing list archive at https://orgmode/list/

2020-06-08 Thread Eric Wong
Kyle Meyer  wrote:
> [ +cc Eric Wong, mostly to say thanks for all the work he puts into
>   public-inbox, which is the software behind these archives, but also so
>   that he can correct me if I misrepresent any capabilities of or plans
>   for public-inbox ]
> 
> > Bastien  writes:
> >
> >> with Kyle's help, I've set up a new mailing list archive:
> >>
> >> https://orgmode/list/
> >>
> >> References in https://orgmode.org and https://orgmode.org/list
> >> that pointed to gmane.org are now using this, so many links are
> >> functional again.
> >
> > Cool! I note that there's also NNTP access at news.yhetil.org, in
> > addition to gmane.io. Does yhetil have a search interface, or are there
> > other mechanisms for searching the archives (ideally in Gnus :))?
> 
> The web interface (<https://orgmode.org/list> or
> <https://yhetil.org/orgmode>) is the main mechanism for searching.  You
> don't necessarily have to leave Emacs for that, as public-inbox's pages
> render nicely in EWW.  But of course that's not the Gnus-based search
> you're hoping for.
> 
> I use Gnus to follow some lists via NNTP, a mix of public-inbox archives
> and gmane.io, but I've never really done any fancy searching from it and
> don't use Gnus for my mail.  To try it out, I hit GG to search on a
> gmane.io list, but got an error [^1], so I suppose its search capability
> went away with Gmane's HTTP interface.
> 
> Poking around a bit, I guess nnweb.el would be the main place that
> public-inbox's web search could be integrated into Gnus?  I've been
> (slowly) working on an Emacs package [^2] that adds public-inbox-related
> functionality to different "endpoints" (currently Notmuch, Gnus, EWW,
> Elfeed), and I'd be interested in any ideas for improving the Gnus
> support.

Cool.  I'm not at all familiar with Emacs or Gnus but I"m glad
it works for users of that.

There'll be an alternative search API in addition to what
currently exists over HTTP (and what's brewing with IMAP).
Maybe JMAP and/or GraphQL, or something with JSON which seems to
be favored nowadays.

I'll need to do some research as I'm not familiar with either
JMAP or GraphQL.

> A couple of other notes:
> 
>   * You can get the entire archive locally with a 'git clone', in which
> case you can transform it into a form that can be indexed/searched
> however you prefer (including with public-inbox, running a local
> public-inbox-httpd).  There are some pointers on extracting an
> archive to a Maildir at
> <https://public-inbox.org/meta/20200426060542.GA15896@dcvr/>.
> 
>   * In the message above, Eric W. mentions that he is considering
> working on client tools with mairix-like search results.  That'd
> make the search capabilities available locally, and I'd imagine
> something like that could be nicely integrated with Gnus,
> considering it already has a mairix backend to use as a guide.

Yes, definitely going to work on a mairix workalike this
summer/fall.  To be effective, there first needs to be a global
search that works seamlessly across multiple inboxes.