There are many reasons why I build everarch but my personal most important ones are:
1) Perkeep has a quite big memory footprint. I think it was probably built for a HDD-era were random hard disk access was super slow. I assume this because the whole index is always in memory. I have a >1TB everarch repository which consumes less than 100MB of RAM. 2) You probably know that 10 years later you might do things differently than today. The properties you write with perkeep into your JSON are pretty much fixed. Changing their meaning can hardly be done without rewriting attributes in probably lots of permanodes. I hope that with the everarch setup I will only need to introduce a new XML namespace in 10 years and then adjust my XSLT transformation to tell how to map the old data into my new way of thinking. We will see if this will work out :D Still I think perkeep is a pretty neat tool. Being able to declare handlers in the server's config in a layered structure is pretty awesome! Of course I would never have written everarch without the many great ideas from perkeep. Am So., 24. Nov. 2024 um 19:55 Uhr schrieb Quinn <[email protected]>: > Cool! I'm reading thru your README, and it sounds a lot like perkeep > and/or restic: > > - store your tiny and GB sized files > - provide a customized index over your data > - make backups complete and fast > - access your files via a virtual file system > - store your data in a format which you still can read in 20 years in > _future programming language_ > - store your data in a way that allows you to easily adopt new data > structures without the need to migrate the existing data > - detect corrupt or externally modified data and restore it with a > partial backup > > Why are you building this instead of just using perkeep (or restic)? Also, > if you are dissatisfied with the claim structure, etc. you could just use > perkeep as a dumb blob store and build your own index using enumeration. > > Really, want i want is this: > > * blob store > * some sort of extensible mechanic for indexing > * code that isn't in a dir called "internal" (i want to use it as a lib > not a server) > * written in go > > I really like perkeep's method of indexing any blob that starts with 0x7b > ({) as an easy way to build meaningful types on top of generic blobs of > content-addressed data. I would like this indexing function to be more > extensible so I can use it to store my own types. > > Also, I get why you chose xml for the claims (xslt and namespaces), but I > think I would prefer to use json anyway 😅 > > Is anyone interested in trying to pick up maintenance of perkeep as a > group? > > Quinn > > On Sunday, November 24, 2024 at 10:32:50 AM UTC-5 markus.peroebner wrote: > >> I'm still using perkeep for storing lots of my personal data. >> >> On the other side I've started to implement an alternative blob store >> with a seperate index called everarch. >> >> You can have a look if you are interested: >> https://evr.ma300k.de/#/title >> >> https://git.ma300k.de/everarch/ >> >> >> Quinn <[email protected]> schrieb am So., 24. Nov. 2024, 15:41: >> >>> Has anyone continued this work? I am very interested in having a general >>> purpose blob storage that I can use for personal projects as a backend/db. >>> Ideally, something I can use as library in go code rather than a server. >>> There really isn't anything that I can find online that fits this, I've >>> spent a lot of time googling stuff like "blob store" or "Content >>> addressable storage" and nothing really comes up. >>> >>> On Friday, April 21, 2023 at 12:39:21 AM UTC-4 [email protected] >>> wrote: >>> >>>> can you incorporate those features into perkeep? fragmentation is >>>> kinda rough. >>>> >>>> Using the blobs as a backend is great. >>>> >>>> On Wed, Apr 19, 2023, 10:15 Lucian D. <[email protected]> wrote: >>>> >>>>> Hey Michael, >>>>> >>>>> In the meantime I realized that PerKeep might not be the best option. >>>>> >>>>> Please see >>>>> https://betterprogramming.pub/the-personal-vault-project-7cdd692b08b0 >>>>> about my vision about the AI-enhanced personal vault that I am working on. >>>>> I have described the system how I see it. >>>>> >>>>> I am curious what you have on your mind. >>>>> >>>>> Some more thoughts on the subject (mine + other peoples) are on the >>>>> Discussion section: https://github.com/dlucian/pvp/discussions >>>>> >>>>> Cheers, >>>>> Lucian. >>>>> >>>>> >>>>> On Sat, Mar 25, 2023 at 2:10 AM Michael Farr <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Lucian, >>>>>> >>>>>> I got an initial system working, but despite its many great features, >>>>>> I couldn't get it to stretch around what I needed. The maintenance >>>>>> <https://github.com/perkeep/perkeep/graphs/code-frequency> doesn't >>>>>> appear to be quite dead yet, but it seems to have slowed down >>>>>> considerably >>>>>> in the past couple of years. >>>>>> >>>>>> The first issue I had was the lack of a system to control the >>>>>> lifecycle of data that I didn't want to keep forever. An example would be >>>>>> to support data that the legal owner only gives me temporary and >>>>>> conditional rights to hold. The second issue was that it couldn't manage >>>>>> data stored on other systems; in general, it just wants to manage files >>>>>> in >>>>>> its own blob storage system and lacks features to link or push external >>>>>> data (it has some good tools for importing data from some sources). >>>>>> >>>>>> I like this idea of wrapping AI into a data management system also. >>>>>> Feel free to contact me directly if you want to discuss some ideas. >>>>>> >>>>>> Mike >>>>>> >>>>>> On Tue, Mar 21, 2023 at 3:20 AM Lucian Daniliuc <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> I've been studying Perkeep for a few weeks now, and even though the >>>>>>> technology and use cases looks solid, I see there's no active community >>>>>>> around it (or I don't know where to loo for it). >>>>>>> >>>>>>> Giving the rise of AI LLMs I am thinking of building a project on >>>>>>> Perkeep and not sure that it is still being developed or maintained. >>>>>>> >>>>>>> So, if you stopped using Perkeep (or just gave up), please reply >>>>>>> with your reason and what are you using instead? >>>>>>> >>>>>>> Cheers, >>>>>>> Lucian. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Perkeep" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/perkeep/e6b06baa-9e44-4df7-8f4b-99841d41b86fn%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/perkeep/e6b06baa-9e44-4df7-8f4b-99841d41b86fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "Perkeep" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/perkeep/eCoO2zm9Plk/unsubscribe. >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/perkeep/CACXS6nQr%2BrxkiZzYBRzgW5bBRNrtm13t%2BU7iCaqKZ9nZtj-cyw%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/perkeep/CACXS6nQr%2BrxkiZzYBRzgW5bBRNrtm13t%2BU7iCaqKZ9nZtj-cyw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "Perkeep" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/perkeep/eCoO2zm9Plk/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/perkeep/CAOXX1pJu9BsAYqM5znP-xr8eMWn%2B7c574hdN1_1p-7Nk1VkxKw%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/perkeep/CAOXX1pJu9BsAYqM5znP-xr8eMWn%2B7c574hdN1_1p-7Nk1VkxKw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Perkeep" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion visit >>> https://groups.google.com/d/msgid/perkeep/9f819edd-5f12-4223-9d99-ebe2cbdcbad3n%40googlegroups.com >>> <https://groups.google.com/d/msgid/perkeep/9f819edd-5f12-4223-9d99-ebe2cbdcbad3n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Perkeep" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/perkeep/6afda2b4-92ec-420e-96fd-322b86f85b33n%40googlegroups.com > <https://groups.google.com/d/msgid/perkeep/6afda2b4-92ec-420e-96fd-322b86f85b33n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/perkeep/CAEfWO%3DGz%3DfiU7Oyy-0wT8TxR2tONbeE%3DZvXb8fucNHAqg4jjLg%40mail.gmail.com.
