Thanks Ian for bringing this up, and Mathieu for your response. I
understand having limited bandwidth only too well.
While I'm learning to hack on Perkeep I'm happy to do it entirely in my
fork of the repo and wait either for the main Perkeep project to kick back
into gear or for a critical mass to coalesce around one fork or another. Of
course I would be grateful for any code review, particularly since it would
help bring me up to speed more quickly, but there's no urgency there.
For now I'm actually more interested to understand some of Perkeep's design
decisions and whether they can be revisited and reconsidered.
For example, the design seems to want claims to modify permanodes, but some
of the implementation seems not to care whether the target of a claim is a
permanode or not. Can that constraint be explicitly removed?
For another example, claims and permanodes seem to need timestamps, but I
quickly ran into cases where a timestamp is not desirable and ended up
using time.Unix(0, 0) as a handy null value. I see that the core Perkeep
code does that in one or two places too. How about making timestamps
optional?
These examples are of interest in part because (as you may have seen) I
have been playing around with importing highly structured static data as
schema blobs. To that end I'd also like to explore design options that
would allow schema blobs (rather than merely claims and permanodes) to
protect one another from garbage collection.
(How might that look? Imagine a "schema+" blob with this structure:
{camliVersion:...,
camliType:..., payload:..., protects:[...]}. Here payload is whatever
structure camliType says it is, and protects is a list of blobrefs
reachable from this blob.)
Cheers,
- Bob
On Tue, Dec 3, 2019 at 1:03 AM Mathieu Lonjaret <mathieu.lonja...@gmail.com>
wrote:
> Hello Ian,
>
> First off, and in case I fail to answer later, I want to say I am
> sorry that I have not committed more to reviewing your work. I know
> how it feels to be in your shoes.
>
> And maybe one advice (and I speak only for myself here), on the short
> term at least, would be to keep on collaborating with people
> interested in the same topic, like you're doing with Bob at the
> moment. I feel that might be the most productive way to go at the
> moment.
>
> Regards,
> Mathieu
>
> On Tue, 3 Dec 2019 at 06:12, Ian Denhardt <i...@zenhack.net> wrote:
> >
> > Hey all,
> >
> > Two separate things I wanted to poke my head in and ask about.
> >
> > First, I've noticed that shortly after the switch to GitHub pull
> > requests, the point of which was to lower the barrier of entry for
> > contributors, patches sortof stopped getting reviewed. I myself have 4
> > of them that have been open since the summer and haven't received any
> > comments[1]. I kinda wandered off after that, but Bob Glickstein's
> > recent work[2] (which has also not seen any review, except for my own
> > comment) got me thinking about Perkeep development again.
> >
> > I know Brad has very little time, and not being paid anymore means
> > Mathieu has limited time to devote as well, and I don't want to demand
> > anything of anyone. I guess I'm just wanting to know what to expect
> > here?
> >
> > ---
> >
> > Second, when I started working on Perkeep I wanted to implement full
> > text search. I still do, and I'm finding that without this my use of
> > Perkeep is much more limited than it might otherwise be, so I'm
> > increasingly itching to get back to this. But I have some uncertainties
> > regarding how to proceed, even assuming the review issue can be
> > solved.
> >
> > The basic issue I'm hitting is: I'm having a really hard time figuring
> > out how to modify the indexer to support this. Attaching the Bleve
> > index itself isn't hard, but there's very little documentation on the
> > format of the index, and after a lot of staring at the implementations
> > of index, corpus, the search handler, and everything that touches those,
> > I still couldn't figure out how to go about integrating significant new
> > functionality like this.
> >
> > I eventually concluded I could get a better sense of how this might work
> > by prototyping a new indexer implementation entirely. I got it to the
> > point where it indexes and does full text search for plain text files
> > and PDFs, but doesn't support any of the existing search predicates. The
> > WIP is here[4]. There are a few lingering design questions I have, but
> > the big thing is: I still don't know how to reconcile this with the main
> > indexer, I don't feel gung ho about suggesting a complete rewrite of the
> > indexer, and I don't feel like I even have enough of a clue about how to
> > extend the indexer to even be able to ask *specific* questions about how
> > it works; I don't know what to ask that could be answered more concisely
> > than someone writing a big overview doc describing at a high level the
> > format of the data in the index, how searches are actually executed,
> > etc. If it came down to it, I could just keep developing the separate
> > indexer implementation, and if Perkeep were abandoned and I was totally
> > on my own I might just go that route, as I've basically given up on
> > understanding how to work with the existing indexer by myself.
> >
> > ...I hate not having a more specific question than "any advice?" but I'm
> > coming up short here and I'd still like to make this happen if I can.
> > Any advice?
> >
> > -Ian
> >
> > [1]: https://github.com/perkeep/perkeep/pulls/zenhack
> > [2]: https://github.com/perkeep/perkeep/pull/1282
> > [3]: https://github.com/perkeep/perkeep/issues/580
> > [4]: https://github.com/zenhack/perkeep/tree/fulltext-index
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Perkeep" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to perkeep+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/perkeep/157534994703.15280.11560776550978742185%40localhost.localdomain
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Perkeep" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to perkeep+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/perkeep/CAHcDtn%3DKeQV_iqEsW2Y3S223qj%2B-9uPUkGu4Cr1Uw8-uUf8THQ%40mail.gmail.com
> .
>
>
--
You received this message because you are subscribed to the Google Groups
"Perkeep" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to perkeep+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/perkeep/CAEf8c4_0WyaMB_tPnnenT6Ez-nRM%3DQaoNJ98GT8Ym%2BLqqqjLcw%40mail.gmail.com.
