[LMDB] some few questions

2014-10-16 Thread Klaus Malorny
Hi, LMDB seems to be quite the right database I am looking for in order to speed up my application. However, I still need to figure some things out. While I could get many answers to my questions by skimming through the archive, the following things are still unclear to me, and it would be qui

Re: [LMDB] some few questions

2014-10-17 Thread Klaus Malorny
Hi Howard, thanks a lot for your answers. No, the data size is limited to 4294967295 bytes. Oh, I missed two "f"s in the constant ;-) - I discovered the return code "MDB_TXN_FULL". By what means are transactions limited? I might consider to use few, but quite large transactions deali

[LMDB] Two API questions

2015-05-04 Thread Klaus Malorny
Hi, during the implementation of my first project using LMDB I got two questions: 1. I wanted to write a small utility function that determines a rough estimation of the free space, using the mdb_env_stat, mdb_env_info and mdb_stat functions, summing up the block counts, multiplying it w

Re: LMDB file size, again

2015-11-16 Thread Klaus Malorny
On 11.11.2015 23:51, Christian Sell wrote: You're being stupid. let's say I am stubborn. As I mentioned, I have the requirement that the user be able to copy the database file away on their USB stick (or send it via email, for that matter). With SQLite, that is no problem. But with LMDB under W

Re: LMDB file size, again

2015-11-16 Thread Klaus Malorny
On 16.11.2015 11:04, Klaus Malorny wrote: On 11.11.2015 23:51, Christian Sell wrote: You're being stupid. let's say I am stubborn. As I mentioned, I have the requirement that the user be able to copy the database file away on their USB stick (or send it via email, for that mat

[LMDB] getting MDB_CORRUPTED when deleting within a DUPSORT database

2017-03-20 Thread Klaus Malorny
Hi, I am using version 0.9.20 on Linux (Ubuntu derivates, uname see [1], [2]). One of the databases is used as an index to another database and thus has been created using the MDB_DUPSORT. Running my software in a test environment, about 33 million entries were generated in this database. In o

Re: [LMDB] getting MDB_CORRUPTED when deleting within a DUPSORT database

2017-03-20 Thread Klaus Malorny
On 3/20/17 2:19 PM, Klaus Malorny wrote: > [...] one addition: I fiddled around to turn mdb_debug on at the right moment and got: entering mdb_cursor_del mdb_page_touch:2426 touched db 8 page 2071338 -> 6529752 mdb_node_del:7376 delete node 0 on leaf page 6529752 mdb_rebalanc

Re: [LMDB] getting MDB_CORRUPTED when deleting within a DUPSORT database

2017-03-21 Thread Klaus Malorny
On 3/20/17 3:32 PM, Howard Chu wrote: Klaus Malorny wrote: Hi, I am using version 0.9.20 on Linux (Ubuntu derivates, uname see [1], [2]). Version 0.9.20 was never officially released. The release was withdrawn due to a corruption bug. I suggest you downgrade to 0.9.19. Hi, after

Re: [LMDB] getting MDB_CORRUPTED when deleting within a DUPSORT database

2017-03-22 Thread Klaus Malorny
On 3/21/17 2:01 PM, Klaus Malorny wrote: On 3/20/17 3:32 PM, Howard Chu wrote: Klaus Malorny wrote: Hi, I am using version 0.9.20 on Linux (Ubuntu derivates, uname see [1], [2]). Version 0.9.20 was never officially released. The release was withdrawn due to a corruption bug. I suggest you

Re: mdb_dbi_open and threads

2017-05-20 Thread Klaus Malorny
On 5/20/17 2:02 PM, Muhammed Muneer wrote: So when the doc says * The database handle will be private to the current transaction until * the transaction is successfully committed. "the handle being private" only refers to the first mdb_dbi_open. Once this transaction is committed, o

Re: mdb_dbi_open and threads

2017-05-22 Thread Klaus Malorny
On 5/21/17 9:43 PM, Muhammed Muneer wrote: Howard Chu wrote "Just follow the recommendation to open all handles at the beginning of the program." But what if I have lots of named databases like maybe 1 or more. Wouldn't this be expensive. I am developing a MongoDB like database (simila

Re: mdb_dbi_open and threads

2017-05-22 Thread Klaus Malorny
On 5/22/17 1:36 PM, Muhammed Muneer wrote: Is there any prospect of implementing mdb_dbi_open or mdb_db_open_immediate to put the dbi into the shared environment without waiting for txn commit. I learned earlier from Howard Chu that it is not a wanted phenomenon in ACID. But just in case, becau

Re: mdb_dbi_open and threads

2017-06-19 Thread Klaus Malorny
On 6/19/17 10:31 AM, Muhammed Muneer wrote: Sorry for the late reply, I was on vacation. Klaus wrote: "I am still unsure what you are trying to achieve. If you are in a read transaction and discover that your database does not exist, what can you do anyway? You cannot create the database at th

Re: mdb fragmentation

2017-08-25 Thread Klaus Malorny
On 8/25/17 2:30 AM, Quanah Gibson-Mount wrote: Hi Geert, If I could, I would delete 8664 from the ITS system entirely as it was filed based on invalid information that was provided to me. It generally should be ignored. When a write operation is performed with LMDB, the freelist is scanned for

Re: mdb fragmentation

2018-01-15 Thread Klaus Malorny
On 03.01.18 00:06, Quanah Gibson-Mount wrote: I wanted to follow up on this, based on doing an examination of Geert's database, and other affected databases.  Geert already has this answer, but it's useful for the general OpenLDAP community. This fragmentation problem is not common.  It depe