Konstantin Ryabitsev <[email protected]> wrote: > On Sat, Sep 04, 2021 at 09:36:58PM +0000, Eric Wong wrote: > > Konstantin Ryabitsev <[email protected]> wrote: > > > Yep, that seems to work fine. Question -- I noticed that lei just issues a > > > regular query, retrieves results with curl and then parses the output. Is > > > there a danger of potentially running into issues with parsing the regular > > > HTML output if it changes in the future? > > > > It's actually parsing gzipped mboxrd (&x=m). But you're right > > we could use stronger safeguards in case we see gzipped HTML or > > something else... > > Ooh, okay, I guess I should actually look at the output of the curl call. :) > The questions I have, then: > > 1. this means that each "lei up" call will be increasingly larger and larger, > since when we init the search with rt:, it gets resolved into a datestamp > (e.g. rt:2.weeks.ago becomes rt:1625699031). I'm worried that this will be > increasingly hard on the server side, especially if someone > fires-and-forgets a cronjob that ends up downloading ever-growing mboxes > every 5 minutes.
"rt:2.weeks.ago" stays "rt:2.weeks.ago" in saved searches :> It was one of my primary annoyances when I initially implemented this and commit 2e4e4b0d6f30d9d4612066395ba694c7c7d61e6e solved it. https://public-inbox.org/meta/[email protected]/ ("lei q: --save preserves relative time queries") > 2. is there some sanity limit on the server side that would prevent someone's > overly broad search query from gzipping and downloading gigabytes of mail? Not right now. With public-inbox-httpd, the actual git fetches are handled fairly w.r.t to other requests (and I could deprioritize them further, if needed...). The Xapian query OTOH... -- unsubscribe: one-click, see List-Unsubscribe header archive: https://public-inbox.org/meta/
