[sqlite] Affinity of expression indexes

2018-12-12 Thread Jens Alfke
Consider CREATE INDEX foo_idx ON tbl (myfunction(a)); where ‘myfunction’ is a deterministic C function I’ve registered with the SQLite connection (and ‘a’ is a column of ‘tbl’ of course.) SQLite has no idea what data type(s) ‘myfunction’ returns, and it might well return different data

Re: [sqlite] sqlite_btreeinfo

2018-12-12 Thread Keith Medcalf
I have the extension compiled in. When a database does not exist I get: SQLite version 3.27.0 2018-12-10 01:48:29 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> select * from sqlite_btreeinfo; Error:

Re: [sqlite] ALTER TABLE, modifying columns

2018-12-12 Thread Thomas Kurz
> I never would have allowed the recent > enhancements to ALTER TABLE that broke it. The enhancements made have been way overdue. Personally, I appreciate them very much and they are worth the "trouble". And I hope that the small problem does not prevent you from taking MODIFY COLUMN and DROP

[sqlite] sqlite_btreeinfo

2018-12-12 Thread Wout Mertens
I can't figure out how to get access to the sqlite_btreeinfo vtable that was added in 3.22 :-( The only documentation is the C file and there doesn't seem to be a compile flag for it. I went and downloaded the file from https://sqlite.org/src/artifact/4f0ebf278f46e68e, then compiled it on on

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Scott Perry
On Dec 11, 2018, at 04:01, Daniel Alm wrote: > > Hi, > > For the past half year we’ve been receiving reports from users who had > restored their SQLite-based databases from a Time Machine backup. Afterwards, > they would receive "database disk image is malformed” errors. The app also > backs

Re: [sqlite] ALTER TABLE, modifying columns

2018-12-12 Thread Richard Hipp
On 12/12/18, Thomas Kurz wrote: > This doesn't work either. The error now occurs in the "ALTER TABLE" line, > which is correct as the table "x" being refered to doesn't exist that > moment. Tested with both 3.25.2 and 3.26. Can you please post a script showing us exactly what you are trying to

Re: [sqlite] ALTER TABLE, modifying columns

2018-12-12 Thread Richard Hipp
On 12/12/18, Thomas Kurz wrote: > > Btw, has the "correct vs. incorrect" table that you've cited already been > there before release 3.25? The procedure description is unchanged for many years. I added the "Caution:" section recently, because a lot of people have been having the same problem

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Simon Slavin
On 12 Dec 2018, at 2:59pm, Olivier Mascia wrote: > When TimeMachine makes copies of the files, the database file and -wal file > will be copied at different points in time, albeit they should be copied from > a filesystem snapshot, so should be consistent to each other. But this should > be

Re: [sqlite] How can custom tokenizer tell it's parsing a search string?

2018-12-12 Thread Jens Alfke
Thanks for the reply, Dan! > On Dec 12, 2018, at 7:08 AM, Dan Kennedy wrote: > > Leaving stop words in while parsing queries won't quite work anyway. If your > tokenizer returns "the" when parsing a query, FTS3/4 will search for "the" in > the index. And it won't be there if the tokenizer

Re: [sqlite] ALTER TABLE, modifying columns

2018-12-12 Thread Thomas Kurz
This doesn't work either. The error now occurs in the "ALTER TABLE" line, which is correct as the table "x" being refered to doesn't exist that moment. Tested with both 3.25.2 and 3.26. Btw, has the "correct vs. incorrect" table that you've cited already been there before release 3.25? -

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Rob Willett
Whilst Time Machine does not do snapshots how enterprise storage do snapshots, literally a freeze and recovery point. Time Machine does make backups suitable for booting from. Apple considers Time Machine suitable for home use backups. You can backup with TimeMachine and boot from it. My

Re: [sqlite] Index with calculated value not covering?

2018-12-12 Thread Wout Mertens
So the missing optimization is to use a covering index vs just the index, right? Are there any plans in that direction? Or maybe a way to hint it? And, to replace this functionality, would it be best to add a column and a trigger that calculates the length on insert or update? Or are there better

Re: [sqlite] ALTER TABLE, modifying columns

2018-12-12 Thread Shawn Wagner
You're using a workflow that https://www.sqlite.org/lang_altertable.html explicitly calls out as incorrect and error prone... Try to create a new table, copy data over, drop the original and then rename the new one to see if that fixes the issue. On Wed, Dec 12, 2018, 8:54 AM Thomas Kurz Dear

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Peter da Silva
Apple uses Sqlite in a number of applications, including Apple Mail, so they have to have some kind of accommodation for saving sqlite databases. The Time Machine patent does not describe using file system snapshots: *"An algorithm or other monitoring can be used to detect changes that occur

[sqlite] ALTER TABLE, modifying columns

2018-12-12 Thread Thomas Kurz
Dear all, I don't know whether the behavior is intentional or a bug, so let me describe it (occurs since 3.25): Due to the lack of ALTER TABLE MODIFY COLUMN, I use the following construction: PRAGMA foreign_keys=0 BEGIN TRANSACTION ALTER TABLE x RENAME TO x_old CREATE TABLE IF NOT EXISTS x

Re: [sqlite] [EXTERNAL] Re: SQLITE gives incorrect results for 'NOT IN' query if partial index exists

2018-12-12 Thread Wout Mertens
Well since this thread is very off topic anyway: I think that would be wildly specific spam, I think she genuinely wanted to unsubscribe. Also, my message to Luuk was supposed to be unicast. I even forwarded the mail and typed his address manually but somehow gmail thought it opportune to keep

Re: [sqlite] Index with calculated value not covering?

2018-12-12 Thread Richard Hipp
On 12/12/18, Wout Mertens wrote: > sqlite> CREATE TABLE "history"("v" INTEGER PRIMARY KEY AUTOINCREMENT, > "type" TEXT, "data" JSON); > sqlite> CREATE INDEX "type_size" on history(type, length(data)); > sqlite> explain query plan select type from history group by type; > QUERY PLAN > `--SCAN

Re: [sqlite] How can custom tokenizer tell it's parsing a search string?

2018-12-12 Thread Dan Kennedy
On 12/12/2018 03:37 AM, Jens Alfke wrote: Is there any way for a custom FTS4 tokenizer to know when it’s tokenizing a search string (the argument of a MATCH expression), as opposed to text to be indexed? Here’s my problem: I’ve implemented a custom tokenizer that skips “stop words” (noise

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Keith Medcalf
I know nothing about "Time Machine", but does it copy the entire filesystem in (at least) "crash consistent" state? --- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume. >-Original Message- >From: sqlite-users

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Rob Willett
Also are they using Time Machine on a networked drive? Whilst Time Machine was not supposed to work across networks, people have made it work using 3rd party software. I know because we tried it for a laugh and abandoned it (and Time Machine) as it was wholly unreliable. However I think the

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread R Smith
On 2018/12/12 4:48 PM, Richard Hipp wrote: On 12/11/18, Daniel Alm wrote: Any suggestions on what could be the culprit or what else I could try besides downgrading all the way to SQLite 3.21 would be appreciated. Nothing about SQLite has changed that should make a difference here. Do you

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Olivier Mascia
Dear Daniel, I'm extracting two points out of your report: > Le 11 déc. 2018 à 13:01, Daniel Alm a écrit : > > While our app is running in the background all the time, it is not very > write-heavy (~ one transaction per minute taking just a few milliseconds). > PRAGMA journal_mode = WAL;

[sqlite] Index with calculated value not covering?

2018-12-12 Thread Wout Mertens
sqlite> CREATE TABLE "history"("v" INTEGER PRIMARY KEY AUTOINCREMENT, "type" TEXT, "data" JSON); sqlite> CREATE INDEX "type_size" on history(type, length(data)); sqlite> explain query plan select type from history group by type; QUERY PLAN `--SCAN TABLE history USING COVERING INDEX type_size

Re: [sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Richard Hipp
On 12/11/18, Daniel Alm wrote: > Any suggestions on what could be the culprit or what else I could try > besides downgrading all the way to SQLite 3.21 would be appreciated. > Nothing about SQLite has changed that should make a difference here. Do you know if the corruption is occurring when

Re: [sqlite] [EXTERNAL] Re: SQLITE gives incorrect results for 'NOT IN' query if partial index exists

2018-12-12 Thread Richard Hipp
On 12/12/18, Hick Gunter wrote: > Serves you right for spying on your boyfriend ;P > > Check the link at the bottom of each and every message from the list for the > way to unsubscribe. We don't enjoy the prospect of free floating pieces of > brain on this list ;) I think those two messages from

Re: [sqlite] [EXTERNAL] Re: SQLITE gives incorrect results for 'NOT IN' query if partial index exists

2018-12-12 Thread Hick Gunter
Serves you right for spying on your boyfriend ;P Check the link at the bottom of each and every message from the list for the way to unsubscribe. We don't enjoy the prospect of free floating pieces of brain on this list ;) -Ursprüngliche Nachricht- Von: sqlite-users

Re: [sqlite] SQLITE gives incorrect results for 'NOT IN' query if partial index exists

2018-12-12 Thread Nicole Sexton
I'm very confused as I never sent that e-mail and no longer want to receive these e-mails. I signed up a long time ago to try to figure out what my boyfriend was up to and decrypt his messages. I somehow stubbled upon here joined the mailing list like an idiot. Like I have 0 clue hoe any of

Re: [sqlite] SQLITE gives incorrect results for 'NOT IN' query if partial index exists

2018-12-12 Thread Nicole Sexton
I don't even know what interlingual means > On Dec 11, 2018, at 2:17 PM, Luuk wrote: > > > On 11-12-2018 10:09, Wout Mertens wrote: >> Hi Luuk, >> >> Not sure if you realize this, but your email comes over as very aggressive, >> and if there's one person on this mailing list that doesn't

[sqlite] Mac: Users receive "database disk image is malformed" errors after restoring database from Time Machine backup

2018-12-12 Thread Daniel Alm
Hi, For the past half year we’ve been receiving reports from users who had restored their SQLite-based databases from a Time Machine backup. Afterwards, they would receive "database disk image is malformed” errors. The app also backs up the user’s data “manually” to a ZIP file every week;

Re: [sqlite] Again https://www3.sqlite.org is down

2018-12-12 Thread Richard Hipp
On 12/12/18, Domingo Alvarez Duarte wrote: > Hello Richard ! > > It seems that https://www3.sqlite.org is down for a few days. Fixed now. Thanks for the report. -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list

[sqlite] Again https://www3.sqlite.org is down

2018-12-12 Thread Domingo Alvarez Duarte
Hello Richard ! It seems that https://www3.sqlite.org is down for a few days. Cheers ! ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users