[sqlite] analysis of a corrupt db

2016-01-14 Thread David Woodhouse
data node. Were there in your dump? That would imply that a data node was lost (its CRC failed, perhaps, and wasn't caught in time to be written out elsewhere). -- David WoodhouseOpen Source Technology Centre David.Woodhouse at intel.com Intel Corporation

[sqlite] Fwd: Usenix paper: Korean researchers invent sqlite WALDIO mode to circumvent redundant journaling by EXT4 on eMMC

2015-07-14 Thread David Woodhouse
On Mon, 2015-07-13 at 21:25 -0400, Donald Griggs wrote: > > -- One mode does require that power not be removed abruptly from the eMMC > controller (but they still claim durability even in the face of a kernel > panic) That's true of *all* modes of operation of most MMC and SSD class devices.

[sqlite] List duplication

2015-03-03 Thread David Woodhouse
On Mon, 2015-03-02 at 21:10 -0600, Mike Owens wrote: > The problem is that this is the very bone of contention in the reply-to > religious war. Religious as in there are strongly-held beliefs on both sides, but only one is really based in logic and common sense? :) > Is it not? I may be wrong,

[sqlite] List duplication

2015-03-02 Thread David Woodhouse
On Mon, 2015-03-02 at 12:45 +0200, R.Smith wrote: > Ok, I've found the source of the list duplications. > > Some emails (Such as the one by J.K. Lowden 2-March-2015 re: Characters > corrupt after importing...) contains a "Reply-To" field in the header > with both list addresses which must have

[sqlite] List duplication

2015-03-02 Thread David Woodhouse
On Sat, 2015-02-28 at 14:10 -0800, Darren Duncan wrote: > > My comment on "whoever" is meant to be plural. > > I think as a general principle that whenever people reply to list messages, > they > look at the headers of the message they are writing and make sure the list > doesn't appear twice

Re: [sqlite] LEFT joins affect query plan for semantically equal queries

2014-11-05 Thread David Woodhouse
On Wed, 2014-11-05 at 16:00 +0200, RSmith wrote: > On 2014/11/05 15:26, David Woodhouse wrote: > > On Wed, 2014-11-05 at 15:13 +0200, RSmith wrote: > > I don't think it's anything to do with the table data being special, > is it? Isn't it generically true that for any LEFT JOIN

Re: [sqlite] LEFT joins affect query plan for semantically equal queries

2014-11-05 Thread David Woodhouse
On Wed, 2014-11-05 at 15:13 +0200, RSmith wrote: > On 2014/11/05 14:13, vita...@yourcmc.ru wrote: > > Hi! > > > > After playing a little with SQLite as a DBMS for Bugzilla, I've discovered > > that LEFT/INNER join affects query plan in a bad way > > even for semantically equal queries: > > > >

Re: [sqlite] x64 vs x32 DLL

2014-11-04 Thread David Woodhouse
> > On 2014/11/04 21:34, jose isaias cabrera wrote: >> Greetings! >> >> I have an application that is written for x32 machines. However, we now >> have a few machines that are x64 and all is well when we are using the >> precompiled x32 DLLs provided by http://www.sqlite.org/download.html. >>

Re: [sqlite] Keeping -wal and -shm files

2014-10-27 Thread David Woodhouse
On Mon, 2014-10-27 at 12:27 -0400, James K. Lowden wrote: > On Mon, 27 Oct 2014 12:33:59 +0100 > Steinar Midtskogen wrote: > > > Is there a way to prevent the -wal and -shm files from being deleted > > after use, so that I can have them always have the right group? Or is

Re: [sqlite] 50% faster than 3.7.17

2014-10-09 Thread David Woodhouse
ptimise the output of the current query planner for them, and still sleep at night :) Thanks. > Unless, of course, you can propose a patch and they turn out to be > simpler than they look. That would be interesting but realistically it's outside my capacity at the moment. I am already *so*

Re: [sqlite] 50% faster than 3.7.17

2014-10-09 Thread David Woodhouse
On Thu, 2014-09-25 at 11:13 +0100, David Woodhouse wrote: > > I suggested a couple of specific optimisations which the query planner > might be able to make, which should hopefully have benefits wider than > just my own use case. Are those not viable? I'm preparing to commit

Re: [sqlite] 50% faster than 3.7.17

2014-09-25 Thread David Woodhouse
On Tue, 2014-09-23 at 17:48 +0100, David Woodhouse wrote: > That looks really promising; thanks for all this work. > > Tristan, you have a comprehensive set of benchmarks for Evolution's > addressbook; is it possible for someone else to run those or would it > take more of your

Re: [sqlite] 50% faster than 3.7.17

2014-09-25 Thread David Woodhouse
On Wed, 2014-09-24 at 19:36 -0600, Keith Medcalf wrote: > > Interesting. From that code you might want to try something like this: > > SELECT uid, vcard, bdata > FROM folder_id > WHERE uid in ( select uid FROM email_list where value like 'p%' >union > select

Re: [sqlite] 50% faster than 3.7.17

2014-09-24 Thread David Woodhouse
On Wed, 2014-09-24 at 06:13 -0600, Keith Medcalf wrote: > > Would it not be more efficient to skip the join altogether since all > you want is the list of uid's, and assuming that you have maintained > the referential integrity of your database mail_list(list_uid) > references main(uid)? > >

Re: [sqlite] 50% faster than 3.7.17

2014-09-24 Thread David Woodhouse
On Fri, 2014-09-19 at 21:14 -0400, Richard Hipp wrote: > The 50% faster number above is not about better query plans. Speaking of better query plans, though... here's a query which takes about 1700ms on my data set, followed by a couple of optimisations which seem like they might be generically

Re: [sqlite] 50% faster than 3.7.17

2014-09-23 Thread David Woodhouse
On Fri, 2014-09-19 at 21:14 -0400, Richard Hipp wrote: > The latest SQLite 3.8.7 alpha version (available on the download page > http://www.sqlite.org/download.html) is 50% faster than the 3.7.17 release > from 16 months ago. That is to say, it does 50% more work using the same > number of CPU

Re: [sqlite] Need advice on using nested selects in JOIN statements as a logical \'AND\'

2014-09-12 Thread David Woodhouse
On Fri, 2014-09-12 at 15:35 -0600, Keith Medcalf wrote: > What happens if you phrase it like this? > >SELECT DISTINCT summary.uid, summary.vcard > FROM folder_id AS summary > LEFT JOIN 'folder_id_email_list' AS email_list >ON email_list.uid = summary.uid > AND

Re: [sqlite] Need advice on using nested selects in JOIN statements as a logical \'AND\'

2014-09-12 Thread David Woodhouse
On Wed, 27 Nov 2013 at 21:21:43 -0800, Igor Tandetnik wrote > Why are you using outer joins when your WHERE clause discards > unmatched records anyway? If you replace LEFT OUTER with INNER, the > end result would be exactly the same. Not for all queries. Consider the query (or (beginswith