[sqlite] Primary key values can be NULL

2016-04-18 Thread David Raymond
I don't mean to poke a busy thread with a possibly stupid newbie question, but here goes. How is this that much different than say, the SQLITE_DEFAULT_FILE_FORMAT compile option? (Pasting it here then continuing comment below) Text pasted here SQLITE_DEFAULT_FILE_FORMAT=<1 or 4>

[sqlite] Partial index query planner questions

2016-03-03 Thread David Raymond
I've got a question or two on the query planner's use of partial indexes. It seems that the query planner isn't using a partial index unless the fields in the WHERE clause of the query are present as fields in the index, even if the WHERE of the query exactly matches the WHERE of the index.

[sqlite] CTE for a noob

2016-03-09 Thread David Raymond
For my own clarification, the statements quoted way down below aren't exactly equivalent, correct? "For each pair of columns identified by a USING clause, the column from the right-hand dataset is omitted from the joined dataset. This is the only difference between a USING clause and its

[sqlite] SQLite Pronunciation

2016-03-16 Thread David Raymond
SQLite: Crystalline Sulfanyl Query-um Lithide -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp Sent: Wednesday, March 16, 2016 3:54 PM To: SQLite mailing list Subject: Re: [sqlite]

[sqlite] [SPAM] Re: ASK matchinfo performance

2016-03-18 Thread David Raymond
For Python, you just have to replace the sqlite3.dll file in your \Python27\DLLs folder. (Or equivalent folder for other versions) You can either copy in the precompiled one, or compile your own dll with different compilation options etc. The only downside to this is that everyone who's running

Re: [sqlite] newbie has waited days for a DB build to complete. what's up with this.

2016-08-05 Thread David Raymond
There's a depreciated pragma, PRAGMA temp_store_directory = 'directory-name'; that apears to still work ok for now at least. http://www.sqlite.org/pragma.html#pragma_temp_store_directory I've used it for the same reasons you've both mentioned about space, though for me it vacuuming a huge db,

Re: [sqlite] "DISTINCT" makes a query take 37 times as long

2017-02-03 Thread David Raymond
Un-analyzed here's what I'm getting while looking at the db: With distinct: sqlite> explain query plan select distinct ppos from move join pos on mto = pnum where pcensus = 18 and pmin < pmax; selectid|order|from|detail 0|0|1|SCAN TABLE pos USING INDEX pipos 0|1|0|SEARCH TABLE move USING

Re: [sqlite] Does SQLite use field definitions?

2017-02-02 Thread David Raymond
An integer primary key is the only field that absolutely must be the type declared. It basically says to use the internal rowid as a visible field. Currently it does start at 1 and any newly inserted record where the id is not explicitly specified will get an id of 1 more than whatever's the

Re: [sqlite] Segfault when inserting ascii text using python-Django with Sqlite3

2017-01-21 Thread David Raymond
Quick note: the mailing list doesn't accept attachments. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of SASSOULAS Pierre 250112 Sent: Friday, January 20, 2017 11:25 AM To: sqlite-users@mailinglists.sqlite.org Subject: [sqlite]

Re: [sqlite] text/numeric comparison confusion

2017-01-26 Thread David Raymond
Other thing to point out is that constants/(expressions that aren't coming from a field in a table) have "no affinity", so explicitly typing '25' is a no affinity, not a text affinity, and 25 is no affinity, not integer affinity. (Section 4.2 in the page linked by DRH) From section 4.3 linked

Re: [sqlite] application function value in sql where

2017-01-26 Thread David Raymond
Did you mean to order by name in both the inside and outside? Or did you mean "order by id desc" for the inner part? If so you can always do that on the outside, which as mentioned is the only one that counts in the end. select row_number(name) as id, name from example order by name asc, id

Re: [sqlite] Intercepting execution of sqlite3 command to see final query

2017-02-13 Thread David Raymond
In the page for expressions (http://www.sqlite.org/lang_expr.html) check out the Parameters section and the REGEXP operator section: "The REGEXP operator is a special syntax for the regexp() user function. No regexp() user function is defined by default and so use of the REGEXP operator will

Re: [sqlite] bug: fields from external (being updated) table cannot be used in "order by" clause of a subselect

2017-02-14 Thread David Raymond
in "order by" clause of a subselect On 14 Feb 2017, at 3:55pm, David Raymond <david.raym...@tomtom.com> wrote: > It's not. It sets up an alias to a new view of test. In the where clause the > i.whereField is referring to that new view of the whole table, whereas the > tes

Re: [sqlite] bug: fields from external (being updated) table cannot be used in "order by" clause of a subselect

2017-02-14 Thread David Raymond
It's not. It sets up an alias to a new view of test. In the where clause the i.whereField is referring to that new view of the whole table, whereas the test.whereField is referring to the field in the current record of test that's being updated. So if the table is things, and whereField is the

Re: [sqlite] Partially "hidden" row, very bizarre state

2017-02-13 Thread David Raymond
I vaguely recall a while ago someone finding a small bug when "primary key unique" was used. Since those are redundant you should get rid of the "unique" in the id field creation as "primary key" takes care of that. (It "shouldn't" make a difference, but don't tempt fate) Other things: run a

Re: [sqlite] Seems that '-cmd .timer on' does not work in non interactive mode

2017-02-15 Thread David Raymond
I'm probably not the best to answer this since I don't know C, but the timing results are printed out in their own special thing. The BEGIN_TIMER and END_TIMER functions only get used in the runOneSqlLine function, and inside the END_TIMER function is where the results actually get printed

Re: [sqlite] Non-unique columns in unique index

2017-03-01 Thread David Raymond
The extra space would just be the few bytes to store the text, which is nothing compared to the "extra" index size. And since it's "infrequent update" then it wouldn't be too bad for the extra time. The normal way would have an extra index to update, the trigger way would be less efficient. On

Re: [sqlite] sqlite importing csv

2016-08-25 Thread David Raymond
Various notes -At the prompt, running "sqlite3 DXCC" is starting the sqlite3 program, and telling it to open the file named "DXCC" as its database. If it's not given a full path then it opens it in the directory the command prompt is in when you run it. If the file you give it doesn't exist,

Re: [sqlite] schema_version and Vacuum or Backup API

2016-09-08 Thread David Raymond
Random question from when I just loaded up that URL: Is it supposed to have a futuristic SQLite version? SQLite Version: 2016-08-22 20:10:01 [7839519349] (3.15.0) -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp

Re: [sqlite] schema_version and Vacuum or Backup API

2016-09-08 Thread David Raymond
to see here... -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp Sent: Thursday, September 08, 2016 10:47 AM To: SQLite mailing list Subject: Re: [sqlite] schema_version and Vacuum or Backup API On 9/8/16, David Raymond

Re: [sqlite] SQLite 3.15.0 scheduled for 2016-10-14

2016-09-26 Thread David Raymond
database. sqlite> create table tbl (a int, b text, c real); sqlite> insert into tbl values (1, 'one', 1.0), (2, 'two', 2.0), (3, 'three', 3.0); sqlite> .testcase a sqlite> select * from tbl where a = 4; sqlite> .check * Error: cannot read 'testcase-out.txt' D:\Temp> -O

Re: [sqlite] converting unix10 and unix13 dates in the same column

2016-09-29 Thread David Raymond
When you have a base expression in the CASE, then it compares each of the WHEN values to that base value. So in your situation there you have it written like if dt = (unix10and13.dt < 100) then... if dt = (unix10and13.dt > 100) then... I think if you get rid of the dt

Re: [sqlite] Complicated join

2016-09-15 Thread David Raymond
Can it be done in SQL? Yes. In any sort of pretty or efficient manner? Ehhh, maybe? I came up with something that seems to work with the small sample cases that I came up with to try and cover your requirements there, but it's got a couple levels of CTE's with long "where not exists..."

Re: [sqlite] Complicated join

2016-09-19 Thread David Raymond
Something that works, but is ugly so I hesitate to post it. Again, working with the results of a simpler query outside of SQL would be preferred. Just because you make ASCII-art of the Mandelbrot set using SQL doesn't mean it's the best choice. (Would that now be UTF-8-art? Doesn't have quite

Re: [sqlite] SQLite 3.15.0 scheduled for 2016-10-14

2016-09-23 Thread David Raymond
-Add the ".testcase" and ".check" dot-commands. Is there documention on what these are? I don't see anything on them in the linked CLI page draft. https://www.sqlite.org/draft/cli.html -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf

Re: [sqlite] Consistency, rollback and such

2016-09-20 Thread David Raymond
Your understanding of the isolation_level parameter there is correct. It only effects how the transactions are created. If you're going to be doing a few selects before you update then doing the "begin someSortOf transaction;" is what you want. After having some trouble getting savepoints to

Re: [sqlite] Query time execution difference between my application and SQLiteBrowser

2016-09-07 Thread David Raymond
Just a reminder that the automatic_index pragma does not create permanent indexes, it's for places where SQLite feels that "hey, if I take the time to create this temporary index then it'll save me more than that time during the rest of this one query." Those indexes get wiped at the end of the

Re: [sqlite] Error File is Encrypted or is not a database

2016-08-18 Thread David Raymond
1) One could download a hex editor and review the beginning of the file and compare to https://www.sqlite.org/fileformat2.html . If some other program has over-written the header, you should be able to observe that, hopefully identifying a program with a problem. 2) Back up you hard drive. Run

Re: [sqlite] Locking databases - Possibly (probably?) a dumb question

2016-08-26 Thread David Raymond
Pragmas are listed out here: http://www.sqlite.org/pragma.html Many do not persist after closing your connection, including synchronous and foreign_keys that you mentioned. In fact, I'd say that the number that do persist is pretty small, and those that do usually require a vacuum or such

Re: [sqlite] Minor pragma doc update request

2016-10-27 Thread David Raymond
Yup, thank you very much. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp Sent: Thursday, October 27, 2016 11:21 AM To: SQLite mailing list Subject: Re: [sqlite] Minor pragma doc update request On 10/27/16, David

[sqlite] WAL pragma question

2016-10-27 Thread David Raymond
I'm playing around with WAL mode here for the first time, along with some of the pragmas, and I'm getting some weird results. I was hoping someone could let me know if I'm missing something, or if yes, it is indeed weird. For starters, I'm looking at the journal_size_limit pragma.

Re: [sqlite] .mode column .width .separator

2016-10-14 Thread David Raymond
If you're using the CLI exclusively you can go into shell.c and get rid of the double spaces. In the copy I'm looking at that's bundled with the 3.14.2 amalgamation it looks like there're 5 lines to alter and you'll be good. They're in static int shell_callback( ... case MODE_Column: { ...

Re: [sqlite] Troubles matching variable as type TIMESTAMP

2016-10-21 Thread David Raymond
"Python 3.19"? SQLite doesn't have a set datetime record format. It's up to you to standardize the input. There're some built-in functions to help out, but you have to format it yourself. If you're doing them as standardized strings, ('2016-10-21 15:40:14') then when you're retrieving them

Re: [sqlite] Drop Existing Table Results in 'no such table'

2016-10-21 Thread David Raymond
I believe "analyze sqlite_master;" will force a re-parse without requiring a new connection. Try it and let us know. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of sanhua.zh Sent: Friday, October 21, 2016 2:49 AM To: SQLite

Re: [sqlite] Drop Existing Table Results in 'no such table'

2016-10-21 Thread David Raymond
Using the writable_schema pragma "only" makes sqlite_master treated like any other table for queries, it doesn't introduce extra logic to check for needed re-parsing or other controlled stuff. So running an UPDATE query on sqlite_master becomes just like any UPDATE on any normal table. It's

Re: [sqlite] pragma integrity_check doesn't check constraints without indices

2016-10-17 Thread David Raymond
Reads about right. I'm writing the below step-by-step thing as much to test my own knowledge of what's going on as to help, so knowledgeable folks please correct me where I mess up. The short version is: don't mess with writable schema unless you have no other choice. (Which you probably do)

[sqlite] Table name syntax

2016-11-21 Thread David Raymond
Basic syntax question on qualified table names in a select. I've got 2 attached databases, say db1 and db2, and I try to run... insert into main.foo select db1.foo.* from db1.foo left outer join db2.bar on db1.foo.pk = db2.bar.pk where db2.bar.pk is null; and I get "Error: near "*": syntax

Re: [sqlite] Table name syntax

2016-11-22 Thread David Raymond
Makes sense, thanks. For one-offs and things like the "Copying from one table to another" thread that aren't going to be part of a regular running program it's easy enough to use the alias version. -Original Message- From: sqlite-users

Re: [sqlite] CREATE TABLE fails

2016-11-22 Thread David Raymond
It's needed. The arrow coming out of [column-def] (visually) goes past [table-constraint] first, with the option to loop down to a comma on its way to a [table-constraint] -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Niall

Re: [sqlite] Changing ID's to UUID

2016-11-16 Thread David Raymond
https://xkcd.com/221/ "The generation of random numbers is too important to be left to chance." -I forget who A UUID is basically a 16 byte integer that people agree to type out in a standard way. All the properties of UUID's that you've mentioned below depend on the random number generator

[sqlite] Hung query question

2016-11-18 Thread David Raymond
I've got a query that I've tried a few times here that seems like it's hanging up on something and I'm wondering if it's just some brain dead thing I'm forgetting or doing wrong. I've got a database with a bunch of records, and am trying to populate a table in another database with coordinate

Re: [sqlite] Autoincrement sequence not updated by UPDATE

2016-11-01 Thread David Raymond
sqlite> create table tbl1 ...> ( ...> id integer primary key autoincrement, ...> someOtherfield, ...> yetAnotherField ...> ); sqlite> create trigger trg_imInChargeAndSayNoAutoincrementUpdates ...> before update of id on tbl1 ...> begin ...> select raise(abort, 'Bad

[sqlite] Minor pragma doc update request

2016-10-27 Thread David Raymond
Minor documentation update request for the pragma page's entry for incremental_vacuum: http://www.sqlite.org/pragma.html#pragma_incremental_vacuum Currently: "PRAGMA schema.incremental_vacuum(N); The incremental_vacuum pragma causes up to N pages to be removed from the freelist. The database

Re: [sqlite] Bug in latest sqlite Release vacuum crashes?

2016-10-26 Thread David Raymond
I've gotten this with this version and with some previous ones (Windows 7). Closest I was ever able to guess as to what was going on was that Windows was memory mapping the file even though I had the CLI compiled with SQLITE_DEFAULT_MMAP_SIZE=0, and SQLITE_MAX_MMAP_SIZE=0. (First should be

Re: [sqlite] WAL pragma question

2016-10-28 Thread David Raymond
bject: Re: [sqlite] WAL pragma question On 10/28/2016 03:16 AM, David Raymond wrote: > I'm playing around with WAL mode here for the first time, along with some of > the pragmas, and I'm getting some weird results. I was hoping someone could > let me know if I'm missing s

[sqlite] Website broken link

2016-10-31 Thread David Raymond
Going through the documentation at http://www.sqlite.org/arch.html In the Parser section there's a link for Lemon: http://www.sqlite.org/lemon.html which is coming up as page not found. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Backward compatibility of indexes with "WHERE function()"

2016-10-11 Thread David Raymond
http://www.gocomics.com/pearlsbeforeswine/2016/06/22 -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of R Smith Sent: Sunday, October 09, 2016 4:33 AM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] Backward

Re: [sqlite] Inserting a new record (anywhere) into a table of ordered records that have an integer auto-increment primary key

2016-10-17 Thread David Raymond
My thinking on this is use a trigger or two. So if you have fields Date and Tiebreaker (as mentioned in the other replies) ... Date text, Tiebreaker int, ... Have it so if you insert it with a given Tiebreaker value it increments things after that, or if you give it null it puts it at the end

Re: [sqlite] A possible double bug?

2016-10-17 Thread David Raymond
Discussions on floating point aside, I'm likewise getting results that are equal when trying it. So I'm curious as to the original poster's SQLite version, platform, language they're coding in, etc. When you run "select foo, typeof(foo) from test;" are you getting two results of (62.027393,

[sqlite] Minor typo in example

2016-10-14 Thread David Raymond
In the Row Values page http://www.sqlite.org/rowvalue.html, section 2.2 there’s an extra semicolon in the code box, turning it into 2 statements. UPDATE tab3 SET (a,b,c) = (SELECT x,y,z FROM tab4 WHERE tab4.w=tab3.d); WHERE tab3.e BETWEEN 55 AND 66;

Re: [sqlite] Problem with CASE in WHERE clause

2016-12-06 Thread David Raymond
http://www.sqlite.org/datatype3.html section 3+ When you declare a field as type "bool" it gets assigned NUMERIC type affinity. "A column with NUMERIC affinity may contain values using all five storage classes. When text data is inserted into a NUMERIC column, the storage class of the text is

Re: [sqlite] About a potential error in sqlite3.c, could you please check it?

2016-12-09 Thread David Raymond
I believe DRH answered this on Tuesday: "The two lines at https://www.sqlite.org/src/artifact/4e4aea7c?ln=7314-7316 guaranteed that the NULL pointer deference at https://www.sqlite.org/src/artifact/4e4aea7c?ln=7328 never happens. -- D. Richard Hipp d...@sqlite.org" -Original Message-

Re: [sqlite] Foreign key child index question

2016-12-13 Thread David Raymond
Hurray for intelligent logic! Also seems to work for composite foreign keys. Thank you all for checking more into this. And thank you to the developers for already having this in there. I think this would be worthy of having a couple sentences written up for the "Foreign Key Support" page.

Re: [sqlite] Foreign key child index question

2016-12-13 Thread David Raymond
] On Behalf Of Dominique Devienne Sent: Tuesday, December 13, 2016 2:50 PM To: SQLite mailing list Subject: Re: [sqlite] Foreign key child index question On Tue, Dec 13, 2016 at 8:07 PM, David Raymond <david.raym...@tomtom.com> wrote: > sqlite> create table parentTable >...> ( >

[sqlite] Foreign key child index question

2016-12-07 Thread David Raymond
http://www.sqlite.org/foreignkeys.html#fk_indexes Question on making indexes for the child fields of foreign keys. I have a child table with a number of foreign keys on fields which the majority of the time are null. I've currently got indexes on the child fields for the purposes of speeding

Re: [sqlite] Advice on using dates and hours for bin counts, time-zone agnostic

2017-01-10 Thread David Raymond
I'm not quite sure on that first update statement, seems like you're not using it right. Otherwise though I think I'd recommend storing things as text "-MM-DD HH:MM:SS" as that's the input type for strftime, which is what would be used for the queries. It also doesn't include the time

Re: [sqlite] Null returned from NOT NULL column thanks to undetected corruption

2017-01-13 Thread David Raymond
My view is that the general thinking of the program here is simply: "just don't make things worse." It can't help what pragmas (ie ignore_check_constraints, writable_schema etc) others may have turned on for their connections, or what sort of junk was there when it arrived. In its head it's

Re: [sqlite] Why this query plan?

2017-01-12 Thread David Raymond
...@mailinglists.sqlite.org] On Behalf Of Richard Hipp Sent: Thursday, January 12, 2017 3:35 PM To: SQLite mailing list Subject: Re: [sqlite] Why this query plan? On 1/12/17, David Raymond <david.raym...@tomtom.com> wrote: > > In the same vane I assume DRH's random ordering would be only ra

Re: [sqlite] Why this query plan?

2017-01-12 Thread David Raymond
of SQLite Database Subject: Re: [sqlite] Why this query plan? How about a pragma_sort_asc_on_rowid_or_primary_integer_key_for_unordered_selects ? RBS On 12 Jan 2017 20:35, "Richard Hipp" <d...@sqlite.org> wrote: > On 1/12/17, David Raymond <david.raym...@tomtom.com> wrote:

Re: [sqlite] Foreign key error...

2017-01-10 Thread David Raymond
On the foreign key page (http://www.sqlite.org/foreignkeys.html) at the very end of section 3 is has: CREATE TABLE artist( artistidINTEGER PRIMARY KEY, artistname TEXT ); CREATE TABLE track( trackid INTEGER, trackname TEXT, trackartist INTEGER REFERENCES artist ); CREATE

Re: [sqlite] Why this query plan?

2017-01-12 Thread David Raymond
Back to the original question: In this case, since the main table is a normal rowid table then the interior pages of the B tree that stores it are only going to contain the rowid part of the table's records, and you have to go all the way down to the leaves to get the rest of each record. In

Re: [sqlite] SQLite does not use opportunity to optimise COUNT() queries using partial indexes

2016-12-01 Thread David Raymond
Gonna take a stab and answering this. http://www.sqlite.org/opcode.html The explain output for select count() from foo; uses the "Count" opcode. The description for that is "Store the number of entries (an integer value) in the table or index opened by cursor P1 in register P2" So that is

Re: [sqlite] Datatype for prices (1,500)

2016-11-30 Thread David Raymond
Storing as either an integer or real will let you do math on the returned value. If you're looking for a pretty representation of the number, then that should be done in your own application. Alternatively though, you can explicitly query for the formatted text version of your number using the

Re: [sqlite] Quest for "scratch table" implementation in SQLite.

2017-01-05 Thread David Raymond
are always for each row, then there would be no such thing as a deferred "temporary fake reference" -Umm, other limitations that my melted end-of-the-day brain can't think of right at the moment. -Original Message----- From: David Raymond Sent: Thursday, January 05, 20

Re: [sqlite] Quest for "scratch table" implementation in SQLite.

2017-01-05 Thread David Raymond
Probably not the most elegant solution, but with enough triggers you can simulate close approximations to foreign keys. create table permTable (pk integer primary key, t text); create temp table tempTable (pk int /*references permTable on update cascade on delete set null*/); create temp

Re: [sqlite] Foreign key integrity checking.

2017-01-06 Thread David Raymond
http://www.sqlite.org/foreignkeys.html My version of the very basic basics anyway: -Foreign key enforcement is turned on or off with a pragma and is a connection-level setting, not a property of the database itself. -The default enforcement is off unless a compile option is used. -So unless you

Re: [sqlite] SQLite announce

2017-01-03 Thread David Raymond
Makes sense, thank you. -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Richard Hipp Sent: Tuesday, January 03, 2017 10:50 AM To: SQLite mailing list Subject: Re: [sqlite] SQLite announce On 1/3/17, David Raymond <david.r

[sqlite] SQLite announce

2017-01-03 Thread David Raymond
Quick question, did something go out to the sqlite-announce mailing list for the 3.16.0 release? Just noticed that it got released yesterday and I'm not seeing a notification in my inbox or in any of my junk or spam folders. I confirmed I'm on the list and a password reminder request email got

Re: [sqlite] SQlite shell tool

2017-01-03 Thread David Raymond
Unfortunately I don't think there is a way to get the times into redirected output automatically. It looks like the callback that handles the redirected output runs completely before the endTimer function runs, which does a straight up printf right there in the endTimer function with no

[sqlite] Opcodes missing from documentation

2016-12-23 Thread David Raymond
I'm looking at some explain output and am looking for info at http://www.sqlite.org/opcode.html but am not seeing a couple of the opcodes listed there. Specifically I'm looking for "SorterInsert", "SorterSort" and "SorterNext". I'm assuming they're gonna be like the similarly named entries,

Re: [sqlite] Possible bug with union and join.

2016-12-22 Thread David Raymond
Problem appears to be coming from an automatic index. sqlite> select * from data left join id_map using (id); --EQP-- 2,0,0,SCAN TABLE map_integer --EQP-- 3,0,0,SCAN TABLE map_text --EQP-- 1,0,0,COMPOUND SUBQUERIES 2 AND 3 (UNION ALL) --EQP-- 0,0,0,SCAN TABLE data --EQP-- 0,1,1,SEARCH SUBQUERY 1

Re: [sqlite] change ORDER BY slowly

2017-03-27 Thread David Raymond
The main issue there I believe is that the order by/limit for the first two is on the outermost table, so it can order by/limit that right away. In the last query the order by/limit is on a table in the middle, so it can't order or filter on it right away, and needs the temp tree there. I saw

Re: [sqlite] BUG?: operation unexpectedly applied to both main and temp

2017-03-30 Thread David Raymond
Close. It rests on the backs of 4 elephants, who in turn stand on the back of the Great A'Tuin "Of course, temp itself is attached to a elephant standing on a stack of turtles. Mark" ___ sqlite-users mailing list

Re: [sqlite] RIGHT JOIN! still not supported?

2017-03-22 Thread David Raymond
Just to be sure, have you actually checked your right join syntax on a system that supports it? Because I don't think what you have written there will actually achieve what you think it will. There is no three-way join operator that will perform a left and a right join . No matter how you word

Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread David Raymond
Remember to make the blob field the very last field in your table schema, and avoid "select * from", otherwise you can create performance problems. The data for a record is stored in the same order as the fields are defined, and overflow pages are a linked list. So if you have your 100 MB blob

Re: [sqlite] Yet Another Why Doesn't Sqlite Use My Index question ...

2017-03-17 Thread David Raymond
The data for each record is stored in the same order as the fields are defined. When reading a record's information, SQLite starts at the first field, and stops at the last field needed to complete the query. The situation where this makes the most difference is where there's a reasonably sized

Re: [sqlite] Yet Another Why Doesn't Sqlite Use My Index question ...

2017-03-20 Thread David Raymond
"... Our query has moved from 32 mins to 54 secs. We're quite happy with that performance increase." I have to admit that the hearty chuckle which that statement produced from me hurt my sore throat. Totally worth it though. ___ sqlite-users mailing

Re: [sqlite] [sqlite-dev] SELECT this set ELSE that set

2017-04-11 Thread David Raymond
Maybe: with FooOrBar as ( select case exists(select 1 from X where x = foo) when 1 then foo else bar end as FooOrBar), select * from A inner join B on A.a = B.a inner join C on B.b = C.b inner join X on C.c = X.c where X.x in FooOrBar; ? -Original Message- From: sqlite-users

Re: [sqlite] Testing sqlite db to see if exists & ready

2017-04-06 Thread David Raymond
Before opening the connection you could do something along the lines of if not os.path.isfile(fi) or not os.access(fi, os.W_OK): print "File isn't there or isn't writable" return 1 with open(fi, "r") as f: if f.read(16) != "SQLite format 3\x00": print "Magic header isn't

[sqlite] CLI .lint question

2017-04-05 Thread David Raymond
Looks like the .lint command has been in since 3.16.0, but this is my first time noticing it and trying it out. I ran the .lint fkey-indexes on one of my larger databases where I had thought I had indexed all the foreign keys ok, and it spat out a giant swath of text, basically for every

Re: [sqlite] Ambiguity in 'pragma page_size' docs

2017-04-12 Thread David Raymond
I think there're two questions going on here. (Though I'm probably adding on to the original question) First is similar to a recent thread and is: When I open a connection, what's the best way to ensure I've connected to an existing file, and am not about to create a brand new one by trying to

Re: [sqlite] Which pragmas are persistent?

2017-04-12 Thread David Raymond
I recommend taking a look at http://www.sqlite.org/fileformat2.html and look at 1.2 The Database Header. Since everything's stored in the file, the permanent pragmas are going to be ones which change one of the values in there. Permanent: page_size (change requires vacuum) auto_vacuum (change

Re: [sqlite] Filter results based on contents of another single field table

2017-04-19 Thread David Raymond
I don't think like works with a subquery as its righthand operand. Or at least not the way you're expecting it to. It's probably only using the very first result of the subquery for all the comparisons. If you're looking for an exact match then what Simon suggested is the way to go. If you're

Re: [sqlite] Why isn't my time formatting working?

2017-03-08 Thread David Raymond
Correct. The ISO strings are the de-facto standard since that's what all the date and time functions take in. http://www.sqlite.org/lang_datefunc.html "The strftime() routine returns the date formatted according to the format string specified as the first argument." It's there so you can store

Re: [sqlite] Why isn't my time formatting working?

2017-03-08 Thread David Raymond
strtime is for formatting a standard format into whatever format you give it. You're thinking of strtime which would parse a given string based on your format. SQLite includes the format function, but not the parse one. -Original Message- From: sqlite-users

Re: [sqlite] Testing sqlite db to see if exists & ready

2017-04-06 Thread David Raymond
ts & ready On Thu, Apr 6, 2017 at 2:07 PM, David Raymond <david.raym...@tomtom.com> wrote: > Before opening the connection you could do something along the lines of > > if not os.path.isfile(fi) or not os.access(fi, os.W_OK): > print "File isn't there or isn't writab

Re: [sqlite] Another Inconsistency in .lint fkeys-index

2017-04-06 Thread David Raymond
That fixed the issue I was seeing the first time around, thanks. Still getting some weirdness where it looks like the results are highly dependent on the contents of sqlite_stat1. I've been trying to construct a simplified version to share but am having trouble reproducing it on a smaller

Re: [sqlite] Performances and Foreign keys

2017-04-24 Thread David Raymond
Another situational space/speed helper I found for when you have a sparsely populated foreign key field is that the child key index can be a partial index and still work. create table child (id primary key, parentID references parent, stuff);--parentID is nullable I had cases where parentID

Re: [sqlite] calculated-value indexes are not covering?

2017-08-09 Thread David Raymond
There's the issue of whether SQLite takes the value from the index, or recalculates it from the table data. So for a "covering index" you would need to index all the inputs to the function, for example sqlite> create index lc on t (length(s), s); sqlite> explain query plan select distinct

Re: [sqlite] Slow query, with correlated sub-sub-query

2017-07-07 Thread David Raymond
Also of note is that when you make an index on ab (size), your original query, unchanged, becomes about 3 times faster than my modification. I'm not sure on what it looks at to decide if a temporary autoindex is worth it, but in this case it would have been. When you move back to your more

Re: [sqlite] Version 3.20.0 coming soon...

2017-07-13 Thread David Raymond
Is there a page for the LSM1 extension? Minor typos: https://sqlite.org/draft/releaselog/3_20_0.html "Evaluation WHERE clause constraints" -> "Evaluate WHERE clause constraints" https://sqlite.org/draft/stmt.html "The enable the STMT extension" -> "To enable the STMT extension" "in default

Re: [sqlite] Example/recipe for truncating fp numbers

2017-07-11 Thread David Raymond
Not to be the new guy here, but would someone be so good as to explain why no one else is panicking that the modulo operator is horrifically broken? In http://www.sqlite.org/lang_expr.html it just says: "The operator % outputs the value of its left operand modulo its right operand." There's

Re: [sqlite] VALUES clause quirk or bug?

2017-07-10 Thread David Raymond
(New changes in testing look good, so sorry if this is reopening this) If you need column names with a VALUES table, why not just kick the VALUES to the front in a CTE where you can name the fields? Then you don't need a temp table or temp view that you need to remember to drop, and since you

Re: [sqlite] Example/recipe for truncating fp numbers

2017-07-12 Thread David Raymond
alf Of Keith Medcalf Sent: Tuesday, July 11, 2017 9:38 AM To: SQLite mailing list Subject: Re: [sqlite] Example/recipe for truncating fp numbers On Tuesday, 11 July, 2017 07:24, David Raymond <david.raym...@tomtom.com> said: > Not to be the new guy here, but would someone be so good as

Re: [sqlite] 3.18.0 SELECT..WHERE x & ? != 0; doesn't use an index

2017-07-14 Thread David Raymond
You can browse what gets looked at for indexes and optimizations here: http://www.sqlite.org/optoverview.html So I agree there's an "opportunity for optimization", but yes it is known. "The only way to get that information from an index that includes all values of y (even and odd) is to scan."

Re: [sqlite] Possible inaccuracy in "Isolation In SQLite" paper

2017-07-18 Thread David Raymond
I think the documentation's good. I think you're missing the whole point of a deferred transaction: that it doesn't start a "transaction" until it needs to. You can run "begin deferred transaction" then walk away for 3 months without upsetting anything. If you need the precise timing then why

Re: [sqlite] Slow query, with correlated sub-sub-query

2017-07-07 Thread David Raymond
I acknowledge you said you weren't so much interested in an alternative solution, but... How about something like select a, min(size) as minSize, recCount from (select a, size, count(*) as recCount from a inner join ab using (a) group by a, size) group by a; The inner one will group by

Re: [sqlite] Version 3.20.0 release candidate 2

2017-07-25 Thread David Raymond
Union extension page has weird numbering on it for each sentence: https://sqlite.org/draft/unionvtab.html For the query in making a union vtab, is there a shorthand for "smallest/largest possible rowid" for columns 3 and 4 to say "just look at everything"? Definitely looks like an extension I

Re: [sqlite] High memory usage for in-memory db with nested transactions

2017-07-25 Thread David Raymond
Would you elaborate a bit more on that? To my untrained-in-C eyes it looks like there's the outer transaction, then a savepoint1, then a loop of (savepoint2, update the only record, release savepoint2). Is savepoint2 there not actually getting released each time? Wouldn't the outer transaction

Re: [sqlite] High memory usage for in-memory db with nested transactions

2017-07-25 Thread David Raymond
with nested transactions On 07/26/2017 12:58 AM, David Raymond wrote: > Would you elaborate a bit more on that? To my untrained-in-C eyes it looks > like there's the outer transaction, then a savepoint1, then a loop of > (savepoint2, update the only record, release savepoint2). Is s

  1   2   3   4   5   >