Well... maybe am I a too "old school": I do appreciate lmdb a lot for its clean and simple API, but a parsimonious side of myself is kind of "shocked" demanding 1 TB upfront. After all, my application might not be the only one running on such a system, and there are many technologies using memory mapped files (like Lucene) that might need this addressing space as well. So I have the impression that we might agree to disagree on this point. In any case, thanks again for your support and answers! Very appreciated.
With best regards, Bruno. -----Message d'origine----- De : Howard Chu [mailto:[email protected]] Envoyé : jeudi 14 janvier 2016 10:25 À : Bruno Freudensprung <[email protected]>; [email protected] Objet : Re: LMBD questions (stat & copy) Bruno Freudensprung wrote: > > Hmmm... doing it this way has several drawbacks (if I am understanding the > docs correctly): > - the mutex of the write transactions prevents from writing > simultaneously in my dbis (I am inserting big graphs and sometimes it > takes from seconds to minutes to do so, plus those graphs are really > independent and modified by independent users) > - the doc of mdb_env_set_maxdbs mentions a cost when there are more > than a "moderate number" of dbis > - and, in the end, this adds another "maximum number of" to the > solution > > However don't get me wrong: lmdb has the best seek/get performances I have > seen so far... this is why I really would like to use it (iterating over > graph edges is so fast with lmdb and its zero copy API design). To me it is a > the perfect fit. The last thing I really would like to do is being able to > detect when I am getting close to the map size (thus my question on the > mt_next_pgno field). Even so, on a current x86-64 server you have 48 bits of virtual address space - 256 terabytes worth. Even with hundreds of DBs open you can give them a terabyte each and not have to worry about it. > > Best regards, > > Bruno. > > -----Message d'origine----- > De : Howard Chu [mailto:[email protected]] > Envoyé : mercredi 13 janvier 2016 23:35 > À : Bruno Freudensprung <[email protected]>; > [email protected] > Objet : Re: LMBD questions (stat & copy) > > Bruno Freudensprung wrote: >> Hi Howard, >> >> Thank you very much for your patient answers, and my apologies for my late >> reply. >> The "reserved space" was indeed occupied by the FREE_DBI, thanks again for >> pointing this out for me. >> Since I will have hundreds of envs (of unpredictable size) in my >> application, > > Sounds like a misuse. For hundreds of tables you should be using individual > named DBs within a single env. The point is to use a single env and not have > to worry about hundreds of configuration details. > >> I am a bit reluctant in allocating too much space for all of them, and >> that's why I was trying to find a way emit a warning when room is about to >> lack. In this regard I think I have read something very promising in the >> docs: the mt_next_pgno field of the txn. It looks perfect. Do you think it >> would be safe to expose it in a read-only way? >> In any case I will follow you advice and try to find a safe compromise for >> the map size. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
