Re: [sqlite] Macro expansion of B0 on Linux PPC system

2017-12-19 Thread Rowan Worth
On 16 December 2017 at 11:21, Richard Hipp wrote: > On 12/15/17, Lee, Greg wrote: > > I never got a reply on this issue and someone else tripped up on it: > > > > https://github.com/spack/spack/issues/6698 > > > > Any help or even acknowledgement of this

Re: [sqlite] Macro expansion of B0 on Linux PPC system

2017-12-18 Thread Rowan Worth
On 19 December 2017 at 11:23, Simon Slavin wrote: > > > On 19 Dec 2017, at 2:55am, Richard Hipp wrote: > > > On 12/18/17, Lee, Greg wrote: > >> I am still seeing the problem on Power 8 and others report the problem > >> persists on Power

Re: [sqlite] Atomic DELETE index optimisation?

2017-12-18 Thread Rowan Worth
On 18 December 2017 at 18:03, Dinu <dinumar...@gmail.com> wrote: > Rowan Worth-2 wrote > > I'm not sure what you're worried about? Dropping and recreating identical > > indices within a transaction won't cause a visible structure change to > > concurrent

Re: [sqlite] Atomic DELETE index optimisation?

2017-12-17 Thread Rowan Worth
On 18 December 2017 at 06:59, Dinu wrote: > Richard Hipp-3 wrote > > Can you try this: > > (1) BEGIN TRANSACTION > > (2) DROP all indexes from the table > > (3) Do the DELETE operation > > (4) CREATE all indexes again > > (5) COMMIT > Thanks Richard, as mentioned earlier,

Re: [sqlite] Seasonal syntax

2017-12-14 Thread Rowan Worth
On 14 December 2017 at 01:19, Warren Young wrote: > On Dec 12, 2017, at 10:24 AM, Simon Slavin wrote: > > > > Santa Clause: SELECT name,hobbies,address FROM people WHERE > behaviour=‘nice’ > > I think you mean > > SELECT name,address > CASE

Re: [sqlite] Commit and rollback behaviour during power loss

2017-11-24 Thread Rowan Worth
There is one degenerate case, which has been discussed a few times on this list. With PRAGMA journal_mode=DELETE (the default), the atomic signal that marks a transaction being committed is the deletion of the rollback journal. Deleting a file is a directory level operation, which means there are

Re: [sqlite] Need SQLite datafiles to be locked on OS level (Windows)

2017-11-24 Thread Rowan Worth
On 24 November 2017 at 16:44, Jiří Matějka wrote: > We need to lock our SQLite datafiles at the filesystem level, because our > application frequently crashes with "database is locked" error, which is > probably due to other processes opening our files. At least we

Re: [sqlite] Does journal_mode=DELETE writes uncommitted queries into DB?

2017-11-23 Thread Rowan Worth
On 24 November 2017 at 10:27, J. King wrote: > The rollback journal is used to return the database to its previous state > in the case of a partial write, not complete writes which are interrupted. > As you didn't commit the transaction, no write occurred, never mind a >

Re: [sqlite] Possible bug with strftime('%s') < strftime('%s')

2017-10-13 Thread Rowan Worth
In that case you would be well advised to use a monotonic clock source, rather than a "date-generating" clock. In linux this is the difference between providing CLOCK_MONOTONIC or CLOCK_REALTIME as the first argument to clock_gettime(). But any API you might use to set a trigger for 2 seconds

Re: [sqlite] XOR operator

2017-10-06 Thread Rowan Worth
On 6 October 2017 at 15:42, wrote: > On Fri Oct 06, 2017 at 09:28:08AM +0200, Clemens Ladisch wrote: > > > > For boolean values, "a XOR b" = "a <> b". > > Is the <> operator documented somewhere? I can't find it in either of > these places: > <> is SQL for "not equal to" (shout

Re: [sqlite] Can't find the source to sqldiff - And yes I have looked :)

2017-09-19 Thread Rowan Worth
I don't have sqlite-src-3200100.zip on hand but it seems to be there in tool/sqldiff.c for sqlite-src-3081002 (and in the fossil repo). -Rowan On 19 September 2017 at 15:58, Rob Willett wrote: > Hi, > > I'm trying to find the source to sqldiff and am struggling to

Re: [sqlite] [EXTERNAL] Number of rows in answer set

2017-09-18 Thread Rowan Worth
-- > Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > Im Auftrag von Rowan Worth > Gesendet: Montag, 18. September 2017 11:08 > An: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> > Betreff: Re: [sqlite] [EXTERNAL] Number of rows in answer se

Re: [sqlite] [EXTERNAL] Number of rows in answer set

2017-09-18 Thread Rowan Worth
On 18 September 2017 at 16:52, David Wellman wrote: > To run a select statement the application code has to: > sqlite3_prepare: parse the sql, make sure it's valid, build the plan > (using "nifty heuristics" :-) ) > sqlite3_step - this one runs the sql, builds an

Re: [sqlite] sqlite 3.20.1: Core dump when running query with json functions in subquery

2017-09-04 Thread Rowan Worth
On 4 September 2017 at 15:37, Bart Smissaert wrote: > Trying this out, so I edited the source and indeed now there is no crash. > I get however a message: > > malformed JSON > > This is the dump of that testing table, dataset: > > doc > {"field1": [{"value": "val1"},

Re: [sqlite] Amalgamation compilation with SQLITE_THREADSAFE=0

2017-09-03 Thread Rowan Worth
On 2 September 2017 at 22:19, Simon Slavin wrote: > > And please note that you never /need/ to do any kind of VACUUM in a > production setting. The only situation where it helps is if your database > is significantly shrinking and will not grow again soon, and that’s quite

Re: [sqlite] Fwd: How can I make this faster?

2017-08-24 Thread Rowan Worth
On 24 August 2017 at 13:58, J Decker wrote: > Even though the process will always be single threaded access even if there > are multiple connections, it would be more helpful to have a > locking_mode=NONE. > > > > PRAGMA locking_mode = EXCLUSIVE gives me good speed, at the

Re: [sqlite] what is server-process-edition?

2017-08-21 Thread Rowan Worth
https://sqlite.org/src/artifact/0c6bc6f55191b690 (it was linked recently by Richard Hipp in another thread, to pre-empt questions of "how did you find that" :) ) -Rowan On 22 August 2017 at 10:22, Gelin Yan wrote: > Hi All > >I noticed there is a tag called

Re: [sqlite] PRAGMA table_info could not update schema

2017-08-21 Thread Rowan Worth
On 21 August 2017 at 17:25, Clemens Ladisch wrote: > sanhua.zh wrote: > > I find that `PRAGMA table_info(tableName)` will not check the expired > schema which is modified by other sqlite connections. > > > > Here is the sample code: > > That code is incomplete and buggy.

Re: [sqlite] Fwd: How can I make this faster?

2017-08-14 Thread Rowan Worth
On 14 August 2017 at 17:11, J Decker wrote: > I monitored the process with ProcMon (I'm on windows 7 BTW) > > 12:55:34.7416316 AM node.exe 9012 WriteFile > \Device\HarddiskVolume17\javascript\gun.db\gun.db-wal SUCCESS Offset: > 188,416, Length: 20,480, I/O Flags: Non-cached,

Re: [sqlite] What's the level of B+-Tree ?

2017-08-11 Thread Rowan Worth
Jump to the byte offset specified by the "start of the cell content" header, which comes just after the number of pages (ie. offset 0x0f90 in your pasted example). Cross reference the data at that offset against section "2.1 Record Format" of the Database File Format page. By decoding the record

Re: [sqlite] hex and char functions

2017-08-08 Thread Rowan Worth
On 8 August 2017 at 18:32, x wrote: > Why does it always turn out to be my own fault ☹ > Welcome to programming ;) It's very rare in practice to find an actual bug in a library or language, especially one as widely deployed and tested as sqlite. Glad you're sorted, I

Re: [sqlite] UPDATE database using parameters

2017-07-21 Thread Rowan Worth
On 21 July 2017 at 17:50, Tim Streater wrote: >$sql = "UPDATE Movies SET name = '$newname' where id=$newid"; > > Personally I don't like forcing PHP to scan strings so I tend to use > concatentation, rewriting the last of these as: > >$sql = 'UPDATE Movies SET name

Re: [sqlite] Possible inaccuracy in "Isolation In SQLite" paper

2017-07-19 Thread Rowan Worth
On 18 July 2017 at 21:43, David Raymond wrote: > You can run "begin deferred transaction" then walk away for 3 months > without upsetting anything. If you need the precise timing then why not > just use "begin immediate"? > IMMEDIATE would take a RESERVED lock which is

[sqlite] Suggestion for PRAGMA SYNCHRONOUS=0 documentation

2017-07-14 Thread Rowan Worth
Hi guys, Ran into an interesting case today where one of our programs was writing an sqlite DB to a network file system. Just as it was finishing up there was a brief connectivity issue, and sqlite's call to close() ran into an EIO error. Sqlite3_close() doesn't return an error in this case so

Re: [sqlite] Error message for nonsensical flags.

2017-07-09 Thread Rowan Worth
On 10 July 2017 at 00:22, Dan Ackroyd wrote: > Hi, > > I'm passing on an error report from a downstream library that uses > SQLite - https://bugs.php.net/bug.php?id=74883 - without having > tested that it is an accurate bug report. Apologies in advance if it > is a

Re: [sqlite] Cannot delete a Database file?

2017-07-09 Thread Rowan Worth
On 8 July 2017 at 10:48, Simon Slavin wrote: > On 8 Jul 2017, at 3:33am, domonic wrote: > > > No I just test the state > > of the database once and then test the output in my if statement. > > > > if(sqlite3_close(...)) > > { > > > > } > > else

Re: [sqlite] SQLITE_ERROR ("SQL error or missing database") should be split into two codes: "SQL error" and "missing database"

2017-07-09 Thread Rowan Worth
On 8 July 2017 at 03:28, Yuri wrote: > > There are some other problems in error definitions. For example, what does > SQLITE_FULL mean? How can database be full? Is it really a disk-full > condition? > > #define SQLITE_FULL13 /* Insertion failed because database is >

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

2017-06-28 Thread Rowan Worth
On 27 June 2017 at 18:42, Eric Grange wrote: > So while in theory all the scenarios you describe are interesting, in > practice seeing an utf-8 BOM provides an extremely > high likeliness that a file will indeed be utf-8. Not always, but a memory > chip could also be hit by

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

2017-06-27 Thread Rowan Worth
On 26 June 2017 at 19:03, Eric Grange wrote: > No BOM = you have to fire a whole suite of heuristics or present the user > with choices he/she will not understand. > Requiring heuristics to determine text encoding/codepage exists regardless of whether BOM is used since the

Re: [sqlite] UTF8-BOM not disregarded in CSV import

2017-06-26 Thread Rowan Worth
On 26 June 2017 at 16:55, Scott Robison wrote: > Byte Order Mark isn't perfectly descriptive when used with UTF-8. Neither > is dialing a cell phone. Language evolves. > It's not descriptive in the slightest because UTF-8's byte order is *specified by the encoding*.

Re: [sqlite] UTF8-BOM not disregarded in CSV import

2017-06-26 Thread Rowan Worth
On 26 June 2017 at 15:09, Eric Grange wrote: > Alas, there is no end in sight to the pain for the Unicode decision to not > make the BOM compulsory for UTF-8. > UTF-8 is byte oriented. The very concept of byte order is nonsense in this context as there is no multi-byte

Re: [sqlite] Questions about SQLite Encryption Extension (SEE)

2017-06-11 Thread Rowan Worth
On 9 June 2017 at 22:30, Yuriy M. Kaminskiy wrote: > > Don't know about windows, but on linux no additional "debug privileges" > needed. You can attach debugger (ptrace syscall) to any process running > with under same user. Additional privileges needed only for debugging >

Re: [sqlite] CLI option to NOT autocheckpoint WAL databases

2017-05-29 Thread Rowan Worth
On 29 May 2017 at 14:46, Clemens Ladisch wrote: > Howard Kapustein wrote: > > I'm effectively looking for a -readonly option, or how to achieve that > > net effect > > sqlite3 "file:test.db?mode=ro" > > But if you want to be really sure, set the file permissions to disallow >

Re: [sqlite] SQLiteDiskIOException: disk I/O error (code 1034)

2017-05-11 Thread Rowan Worth
related errors are often still available, but I am very rusty on this > stuff. > > > On 8/05/2017 5:32 PM, Rowan Worth wrote: > >> Preserving the underlying cause is still possible to do portably; >> something >> like sqlite3_os_errmsg() would provide an &

Re: [sqlite] SQLiteDiskIOException: disk I/O error (code 1034)

2017-05-08 Thread Rowan Worth
On 8 May 2017 at 14:35, Clemens Ladisch <clem...@ladisch.de> wrote: > Rowan Worth wrote: > > These days (the past 8 years?) at least there's the > SQLITE_FCNTL_LAST_ERRNO > > parameter to sqlite3_file_control() allowing the underlying cause to be > >

Re: [sqlite] SQLiteDiskIOException: disk I/O error (code 1034)

2017-05-07 Thread Rowan Worth
On 7 May 2017 at 01:39, Dan Kennedy wrote: > The log most likely won't show too much other than the strerror() text. > > It might be a threads-related error I suppose. I wouldn't jump straight to > that though. > > SQLITE_IOERR_FSYNC means just that - an fsync() call

Re: [sqlite] SQLite log - automatic index on sqlite_sq_25FA456860(ID) (284)

2017-05-04 Thread Rowan Worth
On 4 May 2017 at 01:16, Olivier Mascia wrote: > > Le 3 mai 2017 à 18:46, Richard Hipp a écrit : > > > > On 5/3/17, Olivier Mascia wrote: > >> automatic index on sqlite_sq_25FA4563E0(ID) (284) > >> ... > >> > >> I guess they mean

Re: [sqlite] Thread notification for new record in a table.

2017-05-02 Thread Rowan Worth
Is the database being updated by a separate process, or by another thread in the same process? If the former, I don't think you've any choice but to poll the database. But by taking advantage of PRAGMA data_version and a clever schema you can make that polling pretty lightweight. If you really

Re: [sqlite] Issue on Windows 10 app

2017-04-20 Thread Rowan Worth
There's not enough information here to determine what the problem might be. STOWED_EXCEPTION seems to be a microsoft crash dump artifact; I suspect you'll need to somehow extract further exception details from that. I'm not sure how to go about that exactly but here's some links to start you off:

Re: [sqlite] strange behaviour on sqlite shell output…

2017-04-18 Thread Rowan Worth
On 18 April 2017 at 16:28, Hick Gunter wrote: > Richard Hipp wrote: > >I think the OP is referring to a problem that comes up because the field > width and precision of a printf() format are measured in bytes, not > characters, and if the input is multi-byte UTF then it is

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

2017-04-18 Thread Rowan Worth
Hey Brendan, I'm no OSX expert, but from what I've read this afternoon about NSDocument and friends combined with what I know about sqlite I have to say you are completely mad to continue passing sqlite databases to NSDocument, *especially* as you don't define your own sub-class to do any of the

Re: [sqlite] Regression 3.18.0 cannot import Lightroom dump - Expression tree is too large (maximum depth 1000)

2017-04-06 Thread Rowan Worth
This was already reported recently - if you need a workaround in the short term (before sqlite itself is fixed) see CL's answer here: http://stackoverflow.com/q/43145117/11654 -Rowan 2017-04-05 14:10 GMT+08:00 Axel Reinhold : > Dear sqlite-team, > > after update to 3.18.0

Re: [sqlite] Incompatibility into configure.ac

2017-04-05 Thread Rowan Worth
On 5 April 2017 at 21:37, Scott Robison wrote: > bash supports a --posix switch, which makes it more POSIX-compliant. It > also tries to mimic POSIX if invoked as sh. > And while these methods ensure _compatibility_ with POSIX scripts, they don't ensure that all

Re: [sqlite] Incompatibility into configure.ac

2017-04-05 Thread Rowan Worth
On 5 April 2017 at 17:23, Richard Hipp wrote: > On 4/4/17, Jens Alfke wrote: > > > > The issue here seems to be that some scripts in the SQLite source > > distribution are _implicitly_ assuming that the default shell is bash, or > > else that ‘sh’ is an

Re: [sqlite] How does one block a reader connection?

2017-03-26 Thread Rowan Worth
On 26 March 2017 at 14:17, Keith Medcalf wrote: > If you do not specify your own custom busy handler (to display flying ball > bearings, etc, or do your own exponential sleeping, etc) then the default > busy_handler is used. The default busy handler does its own exponential

Re: [sqlite] SQLite 3 locking

2017-03-02 Thread Rowan Worth
On 1 March 2017 at 02:39, Matthew Ceroni wrote: > > So since busy_timeout defaults to 0, all write attempts if a lock can't be > obtained will return SQLITE_BUSY immediately. Where does the PENDING lock > come into play here? I thought the PENDING was meant to be an

Re: [sqlite] Thread safety of serialized mode

2017-02-19 Thread Rowan Worth
On 18 February 2017 at 01:16, James K. Lowden wrote: > It's why I like Go: it's the first language in 30 years to incorporate > concurrency in its design, and finally support a theoretically sound > model. > I like Go too, but this is giving it a bit too much credit.

Re: [sqlite] Bulk Insert in Sqlite3

2017-02-06 Thread Rowan Worth
On 7 February 2017 at 15:11, Simon Slavin wrote: > > On 7 Feb 2017, at 6:56am, Niti Agarwal wrote: > > > Thanks for your reply. The length matters as I am appending 100 rows at a > > time in a sql statement. It is making very fast as compared to

Re: [sqlite] Bulk Insert in Sqlite3

2017-02-06 Thread Rowan Worth
Hi Niti, There's on need to build a giant SQL string; a transaction can span multiple statements. To bind in golang place a ? within your SQL query and provide the values as additional arguments to the Exec/Query function. eg, after using db.Begin() to create a transaction tx, err :=

Re: [sqlite] Why does a query run 50x slower across a network?

2017-02-06 Thread Rowan Worth
I'm in a different environment (linux with database on nfs share) but found the same behaviour. I came to the conclusion that the latency of network file system operations combined with database fragmentation was largely responsible for the reduced performance. SQLite is very seek heavy, unlike a

Re: [sqlite] can't open db when path length extends 512 characters (on linux)

2017-01-31 Thread Rowan Worth
On 31 January 2017 at 17:32, Jan Nijtmans wrote: > 2017-01-30 19:37 GMT+01:00 Dan Kennedy: > > On 01/31/2017 12:48 AM, Nir Paz wrote: > >> Linux doesn't have that limit, my thought is to change the define of > >> MAX_PATHNAME, is there a better option? > > > > I don't

Re: [sqlite] Bitten by lack of isolation between SELECT and UPDATE on the same connection

2017-01-30 Thread Rowan Worth
The iterator pattern has another caveat when applied to sqlite: foreach (row in statement) { if (isMatch(row)) { return true } } return false If the iterator isn't exhausted, how do you know when to dispose the sqlite3_stmt? There are other ways to manage the

Re: [sqlite] Null returned from NOT NULL column thanks to undetected corruption

2017-01-23 Thread Rowan Worth
m guessing this was pre-fossil. I can't find any reference to ticket #3756 in the current system - is that still around or lost forever? -Rowan On 14 January 2017 at 11:38, Rowan Worth <row...@dug.com> wrote: > On 13 January 2017 at 22:59, David Raymond <david.raym...@tomtom.com&

Re: [sqlite] Null returned from NOT NULL column thanks to undetected corruption

2017-01-13 Thread Rowan Worth
On 13 January 2017 at 22:59, David Raymond wrote: > My view is that the general thinking of the program here is simply: "just > don't make things worse." It can't help what pragmas (ie > ignore_check_constraints, writable_schema etc) others may have turned on > for

Re: [sqlite] does integrity check ever modify the db file?

2017-01-13 Thread Rowan Worth
On 14 January 2017 at 03:51, Adam Smith wrote: > Hey all, > can 'pragma integrity_check' ever modify the file? For instance in case > of a journal file laying around (which was journal file of the same schema > db but a bit different data)? > > The following is what

[sqlite] Null returned from NOT NULL column thanks to undetected corruption

2017-01-12 Thread Rowan Worth
Hi guys, Ran into an interesting situation recently where a database was transferred via FTP and the client somehow managed to truncate the file. As a result the last page was only 337 bytes rather than the expected 1024. Surprisingly running a SELECT on the affected table works without sqlite

Re: [sqlite] (dot) output call locks up when used on a named pipe

2017-01-01 Thread Rowan Worth
On 1 January 2017 at 08:55, James K. Lowden wrote: > On Sat, 31 Dec 2016 15:16:19 -0500 > Paul Lambert wrote: > > > I have used the (dot) .output call in conjunction with a both a file > > and name pipe on Linux with Sqlite 3.13 installed. With a

Re: [sqlite] Weird chars inserted

2016-12-19 Thread Rowan Worth
On 20 December 2016 at 08:06, Kevin Youren wrote: > The experiments were conducted by cut-and-paste of the í character from > the email, hence UTF8, "Hence UTF8" is presumptuous. There are many encodings the website could be using to communicate í. The browser may (or

Re: [sqlite] Weird chars inserted

2016-12-18 Thread Rowan Worth
On 19 December 2016 at 08:24, Kevin wrote: > Hi Martin, > > I had a go using a terminal session, with default encoding UTF-8. > > Try using the hex( ) and unicode( ) functions to check what is actually > stored in the sqlite table. > > I put a couple of rows at the end of an

Re: [sqlite] using sqlite extensions within Ruby

2016-12-08 Thread Rowan Worth
On 8 December 2016 at 23:23, Don V Nielsen wrote: > Like you, I like ruby and working with sqlite via sqlite3 gem. So you have > recompiled sqlite3 with the sessions extension. Correct? And this modified > sqlite3 is the nearest sqlite3 available in your execution path.

Re: [sqlite] SQLite as a Shell Script

2016-11-17 Thread Rowan Worth
g that executable flag is indeed not > trivial. > > On Wed, Nov 16, 2016 at 1:17 PM Rowan Worth <row...@dug.com> wrote: > > > Not true. You can go ahead and create a database called "ls", but: > > > > a) it will not be executable, and > > b) the file

Re: [sqlite] SQLite as a Shell Script

2016-11-16 Thread Rowan Worth
way > execute arbitrary code if the sysadmin visits the system. > > Wout. > > On Wed, Nov 16, 2016 at 11:43 AM Rowan Worth <row...@dug.com> wrote: > > > Interesting but doesn't seem overly practical. If you're in a postiion to > > run 'busybox ash /foo/bar' then y

Re: [sqlite] SQLite as a Shell Script

2016-11-16 Thread Rowan Worth
Interesting but doesn't seem overly practical. If you're in a postiion to run 'busybox ash /foo/bar' then you can easily create a shell script through simpler means. Filesystem access via ATTACH DATABASE is worth bringing attention to though, as I think a lot of developers wouldn't expect that.

Re: [sqlite] Select older or not selected records

2016-11-11 Thread Rowan Worth
ORDER BY CASE timestamp WHEN NULL THEN -9223372036854775808 ELSE abs(random())*timestamp END LIMIT 5? Completely untested, and the weighting function (ELSE clause) is almost certainly terrible :P I think the approach is ok, though I remember some recent threads suggesting the

Re: [sqlite] Bus Error on OpenBSD

2016-10-28 Thread Rowan Worth
Hi Mark, A quick google suggests this is a use after free error, as OpenBSD's allocator apparently fills freed memory pages with the pattern 0xdfdfdfdfdf. The stack trace reads like it is crashing while finalizing an sqlite_stmt, as part of some automatic perl destructor logic. Every

Re: [sqlite] Tcl bindings doc update request

2016-10-27 Thread Rowan Worth
On 28 October 2016 at 02:53, Rolf Ade wrote: > Another plea, since I'm already writing: It isn't immediate and without > any doubt clear, how the "timeout" and the "busy" methods play together, > if both are used. I suspect, the timeout, if given, determines, how long > it

Re: [sqlite] Warning automatic index on

2016-10-24 Thread Rowan Worth
On 24 October 2016 at 16:59, Werner Kleiner wrote: > Hello, > > In an error log there is a message like: > SQlite warning (284) automatic index on is_mytable(internalvalue) > > What does this mean? > It means that SQLite's query optimiser has decided the most efficient

Re: [sqlite] How does the pager know whether its memory cache is still valid?

2016-10-24 Thread Rowan Worth
On 24 October 2016 at 15:44, Clemens Ladisch <clem...@ladisch.de> wrote: > Rowan Worth wrote: > > How does sqlite determine that the cached page is out of date? > > http://www.sqlite.org/fileformat2.html#file_change_counter > > > Ultimately the question I'm trying t

[sqlite] How does the pager know whether its memory cache is still valid?

2016-10-24 Thread Rowan Worth
Hi guys, I haven't been able to figure this one out from the docs, nor have I stumbled onto the right section of the source. Say you have two separate processes accessing an sqlite DB. P1 starts a transaction, reads page #5, ends transaction. P2 starts a transaction, modifies page #5, ends

Re: [sqlite] Drop Existing Table Results in 'no such table'

2016-10-21 Thread Rowan Worth
On 21 October 2016 at 14:49, sanhua.zh wrote: > Don't do this. > I don’t think so. It is written on the sqlite.com. So it should be a > tricky way but not a wrong way. > The docs say, regarding the procedure you followed: > The following simpler procedure is appropriate

Re: [sqlite] page_size: bug with PRAGMA or documentation?

2016-09-13 Thread Rowan Worth
Thanks Richard, that's much clearer. Just one thing: "The page_size pragma will only set in the page size if ..." The "in" looks out of place :) -Rowan On 12 September 2016 at 19:43, Richard Hipp <d...@sqlite.org> wrote: > On 9/12/16, Rowan Worth <row

[sqlite] page_size: bug with PRAGMA or documentation?

2016-09-12 Thread Rowan Worth
Hey guys, The docs for PRAGMA page_size say that it is effective if issued "prior to the first CREATE statement". So imagine my surprise when I found page_size to be ineffective without ever issuing a CREATE statement! The sequence goes like so: $ rm /tmp/lol.db; sqlite3 /tmp/lol.db SQLite

Re: [sqlite] schema_version and Vacuum or Backup API

2016-09-07 Thread Rowan Worth
On 6 September 2016 at 20:04, Clemens Ladisch wrote: > Stephen Chrzanowski wrote: > > [...] I'm relying on the results from "pragma > > schema_version". I've noticed that this bumps up every time I run a > vacuum > > or use the backup API against the database. Is this

Re: [sqlite] SqliteDataAdapter fill wrong data when select data from database

2016-07-29 Thread Rowan Worth
On 27 July 2016 at 18:09, Jin ZhiQiao (Joe) wrote: > When I use sqlite ado.net, I met a bug made me confused. > > > My table schema and data is like this. Table name is "a". > > IDTEXT DOUBLE > > 1 AA 1 > 2 BB 1.2 > 3 CC 2 > > When I run

Re: [sqlite] [Question]SQLite cannot remove journal file as soon as UPDATE transaction finishes?

2016-07-10 Thread Rowan Worth
On 11 July 2016 at 12:18, 刘翔 wrote: > Dear SQLite developers, > > Sqlite version: 3.8.8.3 > Linux version: 3.10.31-ltsi > > Question: > We found when updated a sqlite database in emmc disk, it cannot remove > journal file as soon as the transaction finished. > We know

Re: [sqlite] Locking semantics are broken?

2016-06-28 Thread Rowan Worth
On 28 June 2016 at 16:07, dandl wrote: > > Do not use SQLite for concurrent access over a network connection. > Locking > > semantics are broken for most network filesystems, so you will have > > corruption issues that are no fault of SQLite. > > I have seen this comment made

Re: [sqlite] Conversion failure

2016-06-24 Thread Rowan Worth
On 24 June 2016 at 16:13, Simon Slavin wrote: > On 24 Jun 2016, at 5:04am, Igor Korot wrote: > > > But everything should work independently of what code page is being used? > > The SQLite shell tool should work independently of the code page you have >

Re: [sqlite] Conversion failure

2016-06-23 Thread Rowan Worth
The sqlite shell, at least historically, has I think not accounted for text encoding and simply passed whatever it reads from the console into the database. There has been recent changes in this area since your last email on the subject, for sqlite 3.12.0. What version are you using, Igor?

Re: [sqlite] Version issues on Mac after updating

2016-06-02 Thread Rowan Worth
On 31 May 2016 at 23:01, Dominique Devienne wrote: > > I'm not an OSX user, but if it's like Linux, you can run the ldd command on > your executable file, > and it will show you which dynamic libraries it depends on. If you don't > see SQLite, it probably statically linked...

Re: [sqlite] What could be the cause of a zero length database file?

2016-05-26 Thread Rowan Worth
On 25 May 2016 at 19:48, dandl wrote: > This particular program is run as part of a series of > test cases, and the setup for the test involves deleting the database file > and then running the program in a batch file. Since about Windows 7 the > shell does not wait for a file to

[sqlite] Good way for CEIL, or is there a better way

2016-05-10 Thread Rowan Worth
On 10 May 2016 at 08:31, Darren Duncan wrote: > The Ceiling function is not that simple, unless you know that your rank > and outOf are always non-negative numbers. If they might be negative, you > would -1 rather than +1 when the result is negative. -- Darren Duncan > Yeah you can't always

[sqlite] database is locked when using SQLite3 and MPI to generate different databases

2016-05-09 Thread Rowan Worth
On 8 May 2016 at 10:14, just_rookie <925345468 at qq.com> wrote: > Obviously, I did not do incompatible things with a database at the same > time. > You are attempting to drop a table in databases 300_500.db and 600_900.db. Obviously another process must create that table, since your test code

[sqlite] Fastest way to backup/copy database?

2016-05-06 Thread Rowan Worth
On 4 May 2016 at 20:39, Rob Willett wrote: > Clemens, > > We have 8GB of memory which is the most our VPS provider allows. We?d like > 32GB but its not an option for us. Our desktops have more than that, but > the VPS provider is reasonably priced :) > > We hadn?t considered the WAL mode, my

[sqlite] ANALYZE, sqlite_stat1, and query planning

2016-04-29 Thread Rowan Worth
Hi guys, In an attempt to understand a slow query I've had a quick look at the contents of the sqlite_stat1 table. It looks like the stat column contains a series of integers like: ... Is this observation correct? And if so, does sqlite essentially assume that the rows are equally

[sqlite] sqlite3 command line, read-only

2016-04-27 Thread Rowan Worth
On 27 April 2016 at 13:01, Mark Foley wrote: > > Can sqlite handle contention among multiple accessors with read/write if > the > accessors are not on the same host? I.e. is locking intrinsic in sqlite > and any > accessor from any host is able to determine and set a row/table lock? > That

[sqlite] Is it possible that dropping a big table takes very long

2016-04-23 Thread Rowan Worth
On 22 April 2016 at 21:24, Adam Devita wrote: > > That said, why is the dropping of a table dependent on the size of > the table? Does Sqlite have to mark every block of memory it used as > dropped? (This is obvious for high security mode, but otherwise?) In rollback journal mode, every

[sqlite] Is it possible that dropping a big table takes very long

2016-04-22 Thread Rowan Worth
On 22 April 2016 at 16:00, Cecil Westerhof wrote: > What I find very interesting is that the user time and the sys time does > not increase significantly, but the real time does. Does this point to the > problem, or is this to be expected? > It suggests the extra time is spent waiting for I/O

[sqlite] BUG?

2016-04-22 Thread Rowan Worth
On 22 April 2016 at 14:54, Stephan Beal wrote: > On Fri, Apr 22, 2016 at 8:43 AM, Clemens Ladisch > wrote: > > sqlite> select julianday('2000-01-01 00:00:00'); > > ...> select julianday('2000-01-01 00:00:01'); > > ...> select julianday('2000-01-01 00:00:02'); > > 2451544.5 > >

[sqlite] Is it possible that dropping a big table takes very long

2016-04-19 Thread Rowan Worth
On 19 April 2016 at 02:01, Cecil Westerhof wrote: > 2016-04-18 4:04 GMT+02:00 Rowan Worth : > > > On 18 April 2016 at 06:55, Cecil Westerhof > wrote: > > > > > ?I put a strace on it. This was what I got: > > > Process 26455 attached with 20 threads > &g

[sqlite] Is it possible that dropping a big table takes very long

2016-04-18 Thread Rowan Worth
On 18 April 2016 at 06:55, Cecil Westerhof wrote: > ?I put a strace on it. This was what I got: > Process 26455 attached with 20 threads > % time seconds usecs/call callserrors syscall > -- --- --- - - > 99.80 11245.498406

[sqlite] obtaining bracketing rows where rows are missing

2016-04-15 Thread Rowan Worth
sqlite> create table a(a integer primary key); sqlite> SELECT a1.a, a2.a FROM a AS a1, a AS a2 WHERE a2.a > a1.a GROUP BY a1.a HAVING a2.a = min(a2.a) AND a2.a <> a1.a + 1; 3|5 5|8 11|14 IIRC "HAVING x = min(x)" is not portable SQL but it seems to work in sqlite. -Rowan On 15 April 2016

[sqlite] obtainingthe number of rows returned by a query (with a limit)

2016-03-07 Thread Rowan Worth
On 5 March 2016 at 23:52, Paul Sanderson wrote: > That doesn't work for me: > > I am trying to avoid (select col1, col2, ... colx ) from ... > > and want just a row count which is much less resource intensive. > Have you measured this? I think you might be surprised as sqlite doesn't keep track

[sqlite] Why is a separate journal file needed ?

2016-02-29 Thread Rowan Worth
On 29 February 2016 at 12:13, Igor Tandetnik wrote: > On 2/28/2016 9:19 PM, Rowan Worth wrote: > >> On 27 February 2016 at 00:02, Igor Tandetnik wrote: > > > > I simplified to make the main point stand out: it is not true that the >> >>> database file r

[sqlite] Why is a separate journal file needed ?

2016-02-29 Thread Rowan Worth
On 27 February 2016 at 00:02, Igor Tandetnik wrote: > On 2/26/2016 4:01 AM, Rowan Worth wrote: > >> In principle this is correct, but actually the database *file* is not >> immediately modified in rollback mode. Instead when a page is modified the >> original contents a

[sqlite] Fwd: Re: SQLite and Thunderbird

2016-02-29 Thread Rowan Worth
On 26 February 2016 at 22:42, Howard Chu wrote: > Thunderbird has *always* used its own statically built sqlite, just like > all other Mozilla software. > > In fact, it has more than one copy: > https://hg.mozilla.org/mozilla-central/file/918df3a0bc1c/db/sqlite3/src > > >

[sqlite] Why is a separate journal file needed ?

2016-02-26 Thread Rowan Worth
On 24 February 2016 at 23:46, Igor Tandetnik wrote: > On 2/24/2016 10:08 AM, Stephen Chrzanowski wrote: > >> IMO, all that, plus the fact that you have an easy roll back mechanism. >> Anything that needs to be put in the database is external to the pristine >> database. Lock the database with a

[sqlite] Why is a separate journal file needed ?

2016-02-26 Thread Rowan Worth
On 24 February 2016 at 21:49, Richard Hipp wrote: > On 2/24/16, Simon Slavin wrote: > > Why can't the information which SQLite > > stores in a journal file be put in the database file ? > > Doing so would double the size of the database file. Every database > file would contain extra space

[sqlite] Multiple Column index

2016-02-22 Thread Rowan Worth
On 22 February 2016 at 17:07, Michele Pradella wrote: > CREATE INDEX indexAB ON test(DateTime,CarPlate); CREATE INDEX indexA ON test(DateTime); > So if I have a query like this >>> "SELECT * FROM table_name WHERE DateTime>> DateTime>VALUE_MIN" >>> >> > Generally

[sqlite] Andl: choices for generic types: bool, binary, number/decimal, date/datetime, text/varchar

2016-02-12 Thread Rowan Worth
On 10 February 2016 at 11:14, wrote: > >>>Every SQLite database file has a text encoding that applies to the > entire > file: one of utf8, utf16be, or utf16le. The database text encoding is > stored in the header. You can see the encoding for a particular database > using: > > sqlite3

[sqlite] Bug: Successfully committed transaction rolled back after power failure

2016-02-02 Thread Rowan Worth
On 2 February 2016 at 08:22, Stephen Chrzanowski wrote: > On Mon, Feb 1, 2016 at 11:20 AM, Rowan Worth wrote: > As I indicated in the last paragraph of my mail, I'm not in favour of > > fsync-directory-on-commit in the general case. But that's because I worry > > about the

<    1   2   3   >