-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 IMHO it is a fundamental problem - it should use LevelDB for the index, not memory.
But this index is "just" for the UI and searching - I'm using perkeep in production as a deduplicating storage with severalt TB of small files without problem, with a few MBs of RAM usage - but without indexing. Tamás ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On 2019. January 2., Wednesday 16:10, 'Albert Zeyer' via Perkeep <[email protected]> wrote: > I want to store several TB of data in the database. > > So given that you need 7GB of RAM for about 70GB of blobs, does that mean I > would have to assume maybe 100GB RAM usage for 1TB of blobs? > > It seems to me like this does not seem to scale. Is this a fundamental > problem of Perkeep? > > Best, > Albert > > On Wednesday, January 2, 2019 at 3:29:38 PM UTC+1, tgulacsi78 wrote: > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA256 > > > > AFAIK the disks can be slow, but plenty (for a home server) of RAM is > > needed, > > as the metadata index is kept in memory. > > > > perkeepd consumes about 7GB of RAM for > > > > Jan 02 15:27:48 tequila perkeepd[7308]: 2019/01/02 15:27:48 index/corpus: > > stats: 3354.762 MiB mem: 8111182 blobs (71.815 GiB) (7239923 schema (76206 > > permanode, 54526 file (53213 image), ...) > > Jan 02 15:27:48 tequila perkeepd[7308]: 2019/01/02 15:27:48 index/corpus: > > scanning CPU usage: 3m13.711292s > > > > Tamás > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On 2019. January 2., Wednesday 12:56, 'Albert Zeyer' via Perkeep > > <[email protected]> wrote: > > > > > I would like to run some kind of dedicated hardware/server just for > > > Perkeep, which both acts as a backup, but also as a data provider to e.g. > > > access my photos or other things. > > > > > > Are there suggestions what to run/use? I guess Linux? What file system? > > > And a hardware RAID? Or software RAID? Or is RAID kind of obsolete with > > > Perkeep, and I should do the duplication on the Perkeep level? > > > > > > I would also like to have this in my home, so it should be quiet > > > (preferably only passive cooling) and consume low energy. > > > > > > Best regards, > > > Albert > > > > > -- > You received this message because you are subscribed to the Google Groups > "Perkeep" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -----BEGIN PGP SIGNATURE----- Version: ProtonMail Comment: https://protonmail.com wsBcBAEBCAAGBQJcLN5KAAoJELj70Bsqr19tqFUH/is0+bUJynLinYFzZfyr OKGeR14UFJ6rxsYvnQqnuZrh9FKEnWx2jmFkCLHCBq8/Me7SQ1HdVossVSd8 Q122cFyVErDDAFYY8Ob2gnQQKX1IT3RD7Ybzeb7oaSLbQWUXYY9QWoe9gcb9 Jo3lgYiR5vvaodAVi64dJirUK0vqU/mcfV7oubKuWS91R1qv3MOD9kcbdZ+M SrOD/yKF7iMaFS20ah32M28oL2MsC6MtnYbjjCBUZc+r+ooSORbqG9o1MpA0 O98yo+MCezwauxbUO1bAqtxxWOXvc0ZyaugH868HXCXowd4bL6XzeKmTyRG+ Ggs0KfHfGhdPCeo53Jc/sUg= =d100 -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
publickey - [email protected] - 0x78A4747D.asc
Description: application/pgp-keys
publickey - [email protected] - 0x78A4747D.asc.sig
Description: PGP signature
