MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Hallvard Breien Furuseth
I think MDB v2 should move the variable parts of MDB_meta into the data pages. The datafile header would retain a word with the position of the last *synced* MDB_meta, or of the last meta when MDB_NOSYNC. The lockfile header would hold the position of the *last* MDB_meta. All transactions start f

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Howard Chu
Hallvard Breien Furuseth wrote: I think MDB v2 should move the variable parts of MDB_meta into the data pages. The datafile header would retain a word with the position of the last *synced* MDB_meta, or of the last meta when MDB_NOSYNC. The lockfile header would hold the position of the *last* M

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Hallvard Breien Furuseth
And of course I saw one problem as soon as I hit Send, after mulling over this for hours:( Shouldn't be a problem, though. > All transactions start from the lockfile->metapos commit. Write txns > do not reuse free pages younger than the datafile->metapos commit. Except a system crash may yet lo

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Howard Chu
Hallvard Breien Furuseth wrote: I think MDB v2 should move the variable parts of MDB_meta into the data pages. The datafile header would retain a word with the position of the last *synced* MDB_meta, or of the last meta when MDB_NOSYNC. The lockfile header would hold the position of the *last* M

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Hallvard Breien Furuseth
Howard Chu writes: > I don't see this approach reducing seek overhead. It may be able to reduce > sync overhead, but only if you accept the possibility of delayed syncs > failing. Overall I don't see it as any improvement for ACID compliance. > > What is the real benefit? I thought I said under

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Hallvard Breien Furuseth
Howard Chu writes: >> Setting the latest commit becomes atomic: Just change metapos. >> (Field MDB_txninfo.mti_txnid goes away.) > > The latest commit is already atomic. mti_txnid is updated atomically in the > current code. > >> No sync issues with copying 'MDB_db's from the meta, since the meta

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Howard Chu
Hallvard Breien Furuseth wrote: Howard Chu writes: Setting the latest commit becomes atomic: Just change metapos. (Field MDB_txninfo.mti_txnid goes away.) The latest commit is already atomic. mti_txnid is updated atomically in the current code. No sync issues with copying 'MDB_db's from the

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Hallvard Breien Furuseth
Howard Chu writes: >Hallvard Breien Furuseth wrote: >> Then I'm not sure what's the point of so many speedup options like it >> has now. > > MDB_NOSYNC is perfectly safe on some filesystems like ZFS that guarantee > write > order. Nice. Should tweak the doc a bit, then. Do you mean safe witho

Re: MDB v2: Replace meta pages with "meta position" word

2012-11-11 Thread Hallvard Breien Furuseth
Howard Chu writes: > Using variably positioned meta pages seems like something we would try to cut > down on seek overhead, but looking closer it doesn't appear that it can do > that. Could some commits choose where the next commit's MDB_meta should go? Maybe peel a page off the freelist on beha