Re: [sqlite] Is there a best practice for breaking up a large update?

2016-10-16 Thread Kevin O'Gorman
There are other writers, but they create new stuff, while this big update wants to consolidate info about existing stuff. There are also pure readers. Some of the docs on WAL logging had caveats that put me off. Not knowing exactly what I was going to be doing with the database, I could not be

Re: [sqlite] Is there a best practice for breaking up a large update?

2016-10-15 Thread Darren Duncan
You didn't say if the other tasks need write access to the database or if it is just read-only. If the others only need read-only, let them access a copy of the database while you make your changes in another copy, then just swap the databases when done. -- Darren Duncan On 2016-10-15 1:21

Re: [sqlite] Is there a best practice for breaking up a large update?

2016-10-15 Thread Igor Tandetnik
On 10/15/2016 4:21 PM, Kevin O'Gorman wrote: So I have some questions: (1) If I do all of my updates to a temporary table, does the database still get locked? Locked against what? What else are you trying to do with the database while the update is in progress? Look at WAL journal mode:

[sqlite] Is there a best practice for breaking up a large update?

2016-10-15 Thread Kevin O'Gorman
I'm new to this, and working in Python's sqlite3. So be patient, and don't expect me to know too much. This is also a personal hobby, so there's nobody else for me to ask. I've got a database of a some tens of millions of positions in a board game. It may be over a billion before I'm done