Is it possible to only use the indexing on parts of my storage? I.e. let's say I have a copy of my whole Linux FS, but I only want to index the documents + photos? Otherwise, I maybe could run two independent Perkeep stores, one for the whole system backup, and another one for all documents (e.g. just everything in /home). But my other question ( https://groups.google.com/forum/#!topic/perkeep/Z_utKIPe2o0) basically is more related to this.
But it sounds like in any case, a reasonable amount of RAM makes sense for such hardware. On Wed, Jan 2, 2019 at 4:53 PM Tamás Gulácsi <[email protected]> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > IMHO it is a fundamental problem - it should use LevelDB for the index, > not memory. > > But this index is "just" for the UI and searching - I'm using perkeep in > production as > a deduplicating storage with severalt TB of small files without problem, > with a few MBs of RAM usage - but without indexing. > > Tamás > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On 2019. January 2., Wednesday 16:10, 'Albert Zeyer' via Perkeep < > [email protected]> wrote: > > > I want to store several TB of data in the database. > > > > So given that you need 7GB of RAM for about 70GB of blobs, does that > mean I would have to assume maybe 100GB RAM usage for 1TB of blobs? > > > > It seems to me like this does not seem to scale. Is this a fundamental > problem of Perkeep? > > > > Best, > > Albert > > > > On Wednesday, January 2, 2019 at 3:29:38 PM UTC+1, tgulacsi78 wrote: > > > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA256 > > > > > > AFAIK the disks can be slow, but plenty (for a home server) of RAM is > needed, > > > as the metadata index is kept in memory. > > > > > > perkeepd consumes about 7GB of RAM for > > > > > > Jan 02 15:27:48 tequila perkeepd[7308]: 2019/01/02 15:27:48 > index/corpus: stats: 3354.762 MiB mem: 8111182 blobs (71.815 GiB) (7239923 > schema (76206 permanode, 54526 file (53213 image), ...) > > > Jan 02 15:27:48 tequila perkeepd[7308]: 2019/01/02 15:27:48 > index/corpus: scanning CPU usage: 3m13.711292s > > > > > > Tamás > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > On 2019. January 2., Wednesday 12:56, 'Albert Zeyer' via Perkeep < > [email protected]> wrote: > > > > > > > I would like to run some kind of dedicated hardware/server just for > Perkeep, which both acts as a backup, but also as a data provider to e.g. > access my photos or other things. > > > > > > > > Are there suggestions what to run/use? I guess Linux? What file > system? And a hardware RAID? Or software RAID? Or is RAID kind of obsolete > with Perkeep, and I should do the duplication on the Perkeep level? > > > > > > > > I would also like to have this in my home, so it should be quiet > (preferably only passive cooling) and consume low energy. > > > > > > > > Best regards, > > > > Albert > > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "Perkeep" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > For more options, visit https://groups.google.com/d/optout. > -----BEGIN PGP SIGNATURE----- > Version: ProtonMail > Comment: https://protonmail.com > > wsBcBAEBCAAGBQJcLN5KAAoJELj70Bsqr19tqFUH/is0+bUJynLinYFzZfyr > OKGeR14UFJ6rxsYvnQqnuZrh9FKEnWx2jmFkCLHCBq8/Me7SQ1HdVossVSd8 > Q122cFyVErDDAFYY8Ob2gnQQKX1IT3RD7Ybzeb7oaSLbQWUXYY9QWoe9gcb9 > Jo3lgYiR5vvaodAVi64dJirUK0vqU/mcfV7oubKuWS91R1qv3MOD9kcbdZ+M > SrOD/yKF7iMaFS20ah32M28oL2MsC6MtnYbjjCBUZc+r+ooSORbqG9o1MpA0 > O98yo+MCezwauxbUO1bAqtxxWOXvc0ZyaugH868HXCXowd4bL6XzeKmTyRG+ > Ggs0KfHfGhdPCeo53Jc/sUg= > =d100 > -----END PGP SIGNATURE----- > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Perkeep" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/perkeep/IuUE3f12rhw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
