[sqlite] Why is a separate journal file needed ?

2016-02-29 Thread Rowan Worth
On 29 February 2016 at 12:13, Igor Tandetnik wrote: > On 2/28/2016 9:19 PM, Rowan Worth wrote: > >> On 27 February 2016 at 00:02, Igor Tandetnik wrote: > > > > I simplified to make the main point stand out: it is not true that the >> >>> database file remains "pristine" while the transaction is

[sqlite] Why is a separate journal file needed ?

2016-02-29 Thread Rowan Worth
On 27 February 2016 at 00:02, Igor Tandetnik wrote: > On 2/26/2016 4:01 AM, Rowan Worth wrote: > >> In principle this is correct, but actually the database *file* is not >> immediately modified in rollback mode. Instead when a page is modified the >> original contents are saved to the rollback

[sqlite] Why is a separate journal file needed ?

2016-02-28 Thread Igor Tandetnik
On 2/28/2016 9:19 PM, Rowan Worth wrote: > On 27 February 2016 at 00:02, Igor Tandetnik wrote: > >> On 2/26/2016 4:01 AM, Rowan Worth wrote: >> >>> In principle this is correct, but actually the database *file* is not >>> immediately modified in rollback mode. Instead when a page is modified the

[sqlite] Why is a separate journal file needed ?

2016-02-26 Thread Rowan Worth
On 24 February 2016 at 23:46, Igor Tandetnik wrote: > On 2/24/2016 10:08 AM, Stephen Chrzanowski wrote: > >> IMO, all that, plus the fact that you have an easy roll back mechanism. >> Anything that needs to be put in the database is external to the pristine >> database. Lock the database with a

[sqlite] Why is a separate journal file needed ?

2016-02-26 Thread Rowan Worth
On 24 February 2016 at 21:49, Richard Hipp wrote: > On 2/24/16, Simon Slavin wrote: > > Why can't the information which SQLite > > stores in a journal file be put in the database file ? > > Doing so would double the size of the database file. Every database > file would contain extra space

[sqlite] Why is a separate journal file needed ?

2016-02-26 Thread Igor Tandetnik
On 2/26/2016 4:01 AM, Rowan Worth wrote: > On 24 February 2016 at 23:46, Igor Tandetnik wrote: > >> On 2/24/2016 10:08 AM, Stephen Chrzanowski wrote: >> >>> IMO, all that, plus the fact that you have an easy roll back mechanism. >>> Anything that needs to be put in the database is external to the

[sqlite] Why is a separate journal file needed ?

2016-02-26 Thread Paul Sanderson
WAL files can be many times bigger than a database - the default WAL checkpoint size is when the WAL grows to > 1000 pages. You can get a DB (for example) with 100 pages and a WAL of 1000 (with multiple different copies of the same page). Paul www.sandersonforensics.com skype: r3scue193 twitter:

[sqlite] Why is a separate journal file needed ?

2016-02-25 Thread Eduardo Morras
On Wed, 24 Feb 2016 12:53:48 + Simon Slavin wrote: > In case you're new to this group, please understand that I'm > experienced with SQLite and I understand how it works and how it uses > journal files. > > Can someone explain to me -- or point at a web page which does -- why > a separate

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread Paul Sanderson
You would need some sort of allocation table for the journal (wal or rollback) and on a rollback commit or a wal checkpoint the allocation would need to be cleared (or maybe some "in use" bit cleared) to show that the pages were now free to be re-used. The additional data stored with rollback

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread R Smith
On 2016/02/24 3:49 PM, Richard Hipp wrote: > On 2/24/16, Simon Slavin wrote: >> Why can't the information which SQLite >> stores in a journal file be put in the database file ? > Doing so would double the size of the database file. Every database > file would contain extra space (normally

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread Paul
Good point! I can assume that the problem would be a sparsity of database file. If you mix normal pages and journal then database will be fragmented. You can't avoid it. Even if you start writing journal at the end of file, hoping that you can later truncate it at commit, there is no way to

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread Simon Slavin
In case you're new to this group, please understand that I'm experienced with SQLite and I understand how it works and how it uses journal files. Can someone explain to me -- or point at a web page which does -- why a separate journal file is needed. Why can't the information which SQLite

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread Igor Tandetnik
On 2/24/2016 10:08 AM, Stephen Chrzanowski wrote: > IMO, all that, plus the fact that you have an easy roll back mechanism. > Anything that needs to be put in the database is external to the pristine > database. Lock the database with a transaction, fill up the journal, the > power goes out, your

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread Stephen Chrzanowski
IMO, all that, plus the fact that you have an easy roll back mechanism. Anything that needs to be put in the database is external to the pristine database. Lock the database with a transaction, fill up the journal, the power goes out, your pristine database isn't touched. Just whatever happens

[sqlite] Why is a separate journal file needed ?

2016-02-24 Thread Richard Hipp
On 2/24/16, Simon Slavin wrote: > Why can't the information which SQLite > stores in a journal file be put in the database file ? Doing so would double the size of the database file. Every database file would contain extra space (normally unused) set aside for the journal. -- D. Richard Hipp