[sqlite] Bug: Successfully committed transaction rolled back after power failure

2016-02-02 Thread Rowan Worth
Hi Stephen, On 1 February 2016 at 21:45, Stephen Chrzanowski wrote: > > SQLite is nothing more than part of a program run by the OS. It completely > relies on whatever the OS tells it. If the OS tells it that things are OK, > then that is all that can be done. SQLite can't take on the

[sqlite] Bug: Successfully committed transaction rolled back after power failure

2016-02-01 Thread Rowan Worth
On 1 February 2016 at 18:58, Simon Slavin wrote: > > On 1 Feb 2016, at 9:23am, bm.email01 at gmail.com wrote: > > > --- > > No, SQLite does not. On COMMIT it fsyncs the database file and unlinks > the > > journal[1], but does not fsync the directory. > > --- > > > > Since that can cause the last

[sqlite] Customizing the location of the .sqlite_history

2016-02-01 Thread Rowan Worth
sqlite3() { confs='.sqlite_history .sqliterc' for c in $confs; do ln -s ~/.config/$c ~/$c; done (sleep 10; cd; rm $confs)& command sqlite3 "$@" } Alternately, simply create a new user with home directory matching your ~/.config, and su to that user to run sqlite3. (no, neither of

[sqlite] Bug: Successfully committed transaction rolled back after power failure

2016-02-01 Thread Rowan Worth
On 31 January 2016 at 03:56, James K. Lowden wrote: > Surely SQLite does both -- fsync on file and directory -- as part of a > commit. That's not in doubt, is it? > No, SQLite does not. On COMMIT it fsyncs the database file and unlinks the journal[1], but does not fsync the directory. This is

[sqlite] Find out how many times does SQLite hit the disk?

2016-02-01 Thread Rowan Worth
On 31 January 2016 at 15:09, Yannick Duch?ne wrote: > If it's memory mapped, it's less an efficiency issue, > Hm, can you elaborate on this assertion? I don't think I agree. Lets say sqlite wants to access a page in the DB/journal. In the case of normal file access this is a call to

[sqlite] Bug: Successfully committed transaction rolled back after power failure

2016-01-27 Thread Rowan Worth
On 25 January 2016 at 18:26, Meinlschmidt Stefan < Stefan.Meinlschmidt at esolutions.de> wrote: > > In your case it sounds like a controlled shutdown - is there a reason you > > don't do a full disk sync before that? > > Yes, it is a controlled shutdown, so in my case the /* post-commit logic >

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Rowan Worth
On 22 January 2016 at 14:24, Simon Slavin wrote: > > On 22 Jan 2016, at 4:01am, Rowan Worth wrote: > > > To a point I agree, but in reality there's a fixed amount of work > involved > > with each write transaction. I recently profiled an operation involving > &g

[sqlite] Find SQLITE_BUSY reason?

2016-01-22 Thread Rowan Worth
On 22 January 2016 at 06:33, Warren Young wrote: > With SQLite?s lack of row-level locking, your usage pattern should distill > to ?get in, get done, and get out, ASAP.? Many fine-grained queries are > better than heroic multi-statement queries that change the world. > To a point I agree, but

[sqlite] Bug: Successfully committed transaction rolled back after power failure

2016-01-22 Thread Rowan Worth
> Shutting down power right after a successfully committed > transaction rolls back that transaction on next startup. nitpick: This is sqlite behaving as advertised. See https://www.sqlite.org/lockingv3.html section 5.0 step 6, and https://www.sqlite.org/atomiccommit.html section 3.11 which

[sqlite] WAL: difference between IMMEDIATE and DEFERRED transaction

2016-01-18 Thread Rowan Worth
On 15 January 2016 at 22:09, Olivier Vidal wrote: > For the DEFERRED transaction: > > - BEGIN DEFERRED TRANSACTION > - SELECT > - UPDATE > - SELECT > - UPDATE > - INSERT > - SELECT > - COMMIT > > The lock is requested at the first UPDATE (and there have no TIMEOUT?). > The database cannot be

[sqlite] hard links and SQLite

2016-01-12 Thread Rowan Worth
On 12 January 2016 at 13:12, Felipe Gasper wrote: > Same code, just different processes. > > We?ve just noted over the years with race conditions that that ?if it can > fail, it will?, so we try to be as bulletproof as we can. > Good policy :) After you unlink the temp file, I presume the

[sqlite] hard links and SQLite

2016-01-12 Thread Rowan Worth
On 12 January 2016 at 13:26, Scott Hess wrote: > On Mon, Jan 11, 2016 at 9:12 PM, Felipe Gasper > wrote: > > > On 11 Jan 2016 9:06 PM, Rowan Worth wrote: > > > >> * if it returns SQLITE_OK and zero rows, the schema hasn't been created > >> yet > &g

[sqlite] hard links and SQLite

2016-01-12 Thread Rowan Worth
On 12 January 2016 at 03:00, Felipe Gasper wrote: > On 11 Jan 2016 1:45 PM, Scott Hess wrote: > >> >> As far as preventing the other process from using it before the schema >> exists, do "SELECT count(*) FROM sqlite_master", and if the result is 0, >> the schema does not exist. If you create

[sqlite] About vacuum

2016-01-04 Thread Rowan Worth
On 4 January 2016 at 16:59, Simon Slavin wrote: > The VACUUM function takes so long to run that you cannot get faster > overall. > > Depending on which operating system, file system and storage system you > use, it is possible for VACUUM to increase speed of SELECT a little, but > not much [1].

[sqlite] Using colation in Java

2015-12-29 Thread Rowan Worth
On 29 December 2015 at 08:23, Cecil Westerhof wrote: > When working in Python I can use: > con.create_collation("mycollation", collate) > > To change the sort order. How should I do this in Java? > Note there are multiple ways to use sqlite from java, so it would help to specify which

[sqlite] SQLite - Support for VSS writer

2015-12-04 Thread Rowan Worth
On 3 December 2015 at 22:00, Richard Hipp wrote: > On 12/3/15, Murdare, Vijaykumar S (GE Oil & Gas) > wrote: > > Hi, > > > > Also, I would like to know while taking the backup of online databases: > > > > 1) If read operation is in progress, then can I take backup safely > with > > file

[sqlite] method for thousands separator via sed post processing

2015-11-27 Thread Rowan Worth
Hi Bruce, On 27 November 2015 at 10:59, Bruce Hohl wrote: > Thanks to suggestions on this list I explored the sed post processing > avenue and found a solution. There are quite a few sed docs at > http://sed.sourceforge.net For my needs I adapted an example from >

[sqlite] attempt at output with thousands separator via extension

2015-11-23 Thread Rowan Worth
Hi Bruce, I had a go at post-processing the sqlite3 shell's output to apply thousand separators. I don't recommend looking too hard at the sed for the sake of your sanity, but the gist is it repeatedly prepends a comma to trailing groups of three digits, and then repeatedly removes commas which

[sqlite] Simple Math Question

2015-10-26 Thread Rowan Worth
On 23 October 2015 at 23:34, Rousselot, Richard A < Richard.A.Rousselot at centurylink.com> wrote: > Scott, > > I agree with everything you said but... To me if a program/CPU evaluates > something internally, then when it reports the result it should be the > result as it sees it. It shouldn't

[sqlite] Simple Math Question

2015-10-23 Thread Rowan Worth
On 23 October 2015 at 16:08, Dominique Devienne wrote: > Another good good way to think of IEEE I was presented once with, and which > kind of gave me a ah-ah moment, is the fact that numbers with exact > representation fall on the nodes of grid, and there's plenty of "space" in > between the

[sqlite] Compilation fails under Linux (Ubuntu) when FTS5 is enabled

2015-10-19 Thread Rowan Worth
On 18 October 2015 at 04:15, wrote: > I just tried one more time with the -lm switch and this time it worked. > Hmm... > > Problem solved. Thanks. You might find that specifying -lm *after* the .c files works but -lm before the .c files doesn't. The linker processes libraries/object files in

[sqlite] Problem with sqlite3_db_filename

2015-10-09 Thread Rowan Worth
Suspect you are running into more VBA<->C issues. The db path, journal path and wal path are stored sequentially in memory, so if you were to skip the NUL terminators you'd see all three paths. But I'm not exactly sure how that might happen without resulting in a segfault, so I could be missing

[sqlite] Is it legal SQL to select a specific row of a group via a HAVING clause?

2015-09-29 Thread Rowan Worth
Hi guys, Imagine a DB storing a history of currency exchange rate info. CREATE TABLE Prices ( id INTEGER PRIMARY KEY, day INTEGER, currency TEXT, price FLOAT); Assume 'day' increases monotonically and there is at most one price recorded per currency per day - if you want to know

[sqlite] SQLITE_BUSY_SNAPSHOT from BEGIN IMMEDIATE

2015-09-29 Thread Rowan Worth
On 29 September 2015 at 03:47, Florian Weimer wrote: > Relatively rarely, while starting a transaction on a concurrently > modified SQLite database in WAL mode, I get a SQLITE_BUSY_SNAPSHOT > error for just-prepared "BEGIN IMMEDIATE" statement. > BEGIN IMMEDIATE takes a RESERVED lock straight

[sqlite] UPDATE silently failing

2015-09-23 Thread Rowan Worth
On 23 September 2015 at 12:32, Hugues Bruant wrote: > On Wed, Sep 23, 2015 at 12:00 AM, Rowan Worth wrote: > > > Has many possible explanations. > > I can't think of any that is consistent with the Java specification. > Yeah fair enough, SIndex looks watertight. It's st

[sqlite] UPDATE silently failing

2015-09-23 Thread Rowan Worth
> SIndex sidx is just a boxed immutable integer. If it were me I'd be reviewing this assumption very carefully. The sequence of events: 1. SIndex.getInt() returns 0 2. SIndex.toString() returns "1" 3. SIndex.getInt() returns 1 Has many possible explanations. You could also try 'final int sid =

[sqlite] How to know when a sqlite3_stmt has been recompiled ?

2015-09-21 Thread Rowan Worth
On 21 September 2015 at 16:36, Simon Slavin wrote: > > On 21 Sep 2015, at 8:29am, Rowan Worth wrote: > > > 1) Statement is prepared > > 2) Statement is stepped > > 3) Statement is reset > > 4) ... time passes ... > > 5) Statement is stepped > >

[sqlite] How to know when a sqlite3_stmt has been recompiled ?

2015-09-21 Thread Rowan Worth
On 21 September 2015 at 14:38, Simon Slavin wrote: > As expected, ALTER TABLE acts like INSERT: it's a change which requires an > exclusive lock. So just as the documentation says, in a normal journal > mode you can't make a change while the database is locked (which it is > during a SELECT),

[sqlite] [OT] Handling the whole select query from an index

2015-09-18 Thread Rowan Worth
> Outside London, a postcode can cover a far, far wider area specially in rural or sparsely populated areas. I would imagine Australian postcodes to be similar but thats based on a guess rather than actually any knowledge. I'm not sure whether to take this claim of ignorance at face value or

[sqlite] About backup example

2015-09-13 Thread Rowan Worth
On 12 September 2015 at 20:59, David Kazlauskas wrote: > Hi, I see something in the second backup example ( > http://www.sqlite.org/backup.html ) that seems like logical error to me. > > if( rc==SQLITE_OK || rc==SQLITE_BUSY || rc==SQLITE_LOCKED ){ > sqlite3_sleep

[sqlite] BEGINNER - Transactions in shell script

2015-09-06 Thread Rowan Worth
On 6 September 2015 at 18:26, Luuk wrote: > > Suppose i have 'test.sql': > .echo on > DELETE FROM test; > BEGIN; > INSERT INTO test VALUES(1,'test1'); > INSERT INTO test VALUES(3,'test3',3); > INSERT INTO test VALUES(2,'test2'); > COMMIT; > SELECT * FROM test; > > And a database 'test.sqlite'

[sqlite] Performance problems on windows

2015-08-26 Thread Rowan Worth
What about fragmentation of the database itself? Does running VACUUM on the database affect performance? -Rowan On 26 August 2015 at 16:16, Jakub Zakrzewski wrote: > Hi, > > nope. The defragmentation job runs every Wednsday night and the > fragmentation is very low. > > PS: I'm putting

[sqlite] ATTACH DATABASE statement speed

2015-08-17 Thread Rowan Worth
On 17 August 2015 at 14:52, Paolo Bolzoni wrote: > After I attach the INPUT db and I copy the useful lines. > > The input db is about 13GB, so not really large, however the step on > this sql statement (where ? is of course binded to the db name) > > ATTACH DATABASE ? AS indb; > > requires

[sqlite] Using uninitialized value nDummy when calling vdbePmaReaderInit

2015-08-15 Thread Rowan Worth
On 15 August 2015 at 14:35, Carlos Tangerino wrote: > > The variable nDummy is not initialized *(1)* in the function but its > pointer *(2)* is passed to *vdbePmaReaderInit* that increments *(3)* its > value. > nDummy is never used after vdbePmaReaderInit though, so it doesn't really matter that

[sqlite] Problem with SQLite in C++. DB is BUSY (Multithread)

2015-08-11 Thread Rowan Worth
On 11 August 2015 at 06:22, Simon Slavin wrote: > If you did set a timeout then SQLite wais a very short time (a millisecond > ?) before it tries again, then it waits a little longer, then a little > longer still, etc.. It gradually increases the time until the total time > reaches the time you

[sqlite] ATTACH Problem

2015-07-31 Thread Rowan Worth
Chris, you have an extra pair of single quotes in your original code compared to Simon's suggestion. Also in your parameter-based attempt you have a typo when assigning the path (@DNName instead of @DBName). Hope that helps, -Rowan On 31 July 2015 at 15:09, Chris Parsonson wrote: > That's

[sqlite] Interrupting the busy handler

2015-07-25 Thread Rowan Worth
Hiya, Firstly thank you for sqlite, it's proved to be a remarkably solid bit of software. I recently noticed one interesting interaction -- or lack thereof -- between the default busy handler (registered via sqlite3_busy_timeout) and sqlite3_interrupt. Specifically, sqlite3_interrupt appears to

<    1   2   3