Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Keith Medcalf
You need a UNIQUE index on the PARENT KEY because when you operate on a child, you need to be able to look up the PARENT. If there is no index on the PARENT KEY then you have to do a table scan. A table scan of a billion parent records many take quite some time. Similarly, you need an index

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Graham Holden
Hello Yuri, Friday, August 03, 2018, 12:06:14 AM, Yuri wrote: > On 8/2/18 3:46 PM, Keith Medcalf wrote: >> You are required to have a UNIQUE index on the PARENT KEYS in a foreign key >> relationship. >> >> Depending on whether the PARENT:CHILD is 1:1 or 1:N you need either >> a UNIQUE (1:1) or

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Yuri
On 8/2/18 3:46 PM, Keith Medcalf wrote: You are required to have a UNIQUE index on the PARENT KEYS in a foreign key relationship. Depending on whether the PARENT:CHILD is 1:1 or 1:N you need either a UNIQUE (1:1) or regular index on the child key. Why is index on PARENT KEY in a foreign

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Keith Medcalf
It is. If you create the missing index then your "testcase" does not demonstrate any slowdown. After this line: doSql(db, "create table a (id int PRIMARY KEY, name text, aid int not null, FOREIGN KEY(aid) REFERENCES a(id))"); insert this line: doSql(db, "create index aid on a (aid)");

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Yuri
On 8/2/18 3:17 PM, Keith Medcalf wrote: .lint fkey-indexes and it will tell you what indexes you forgot to create that cause the issue you are seeing. But this problem isn't about a missing index. Yuri ___ sqlite-users mailing list

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Keith Medcalf
Many versions ago a CLI command (that is, the sqlite3 Command Line Interface) was created so that folks would stop complaining about referential integrity enforcement being slow when they did not create the indexes that were necessary to enforce referential integrity (because failing to have

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Keith Medcalf
Well, ok, but it is not really creating an index drawing data from multiple tables. The table definition part "interleaves" data with the same value for the same column into a "cluster" of pages, and the creation of the index on the cluster is an index of the "common column data". Creating

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Yuri
On 8/2/18 7:02 AM, David Raymond wrote: So for your test script there, try inserting a record with the violation ID a little bit later and see if it suddenly speeds up again. In my Python version of your script it does indeed speed back up again once the outstanding violation is fixed. The

Re: [sqlite] Using CTE with date comparison

2018-08-02 Thread R Smith
On 2018/08/02 10:29 PM, Csányi Pál wrote: Hi, I just want to know why the following SQLite query does not work as I expected? WITH RECURSIVE dates(dateD) AS ( VALUES('2015-10-03') UNION ALL SELECT date(dateD, '+1 year') FROM dates WHERE date(dateD) <= date('2016-11-01') ) SELECT max(dateD),

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Jean-Luc Hainaut
On 02/08/2018 20:50, Keith Medcalf wrote: In no DBMS known can you index data sourced from multiple tables in the same index -- this applies to "Relational" databases and all other database models (such as pure hierarchical, network, network extended, etc.) In all DBMS systems the contents of

[sqlite] Using CTE with date comparison

2018-08-02 Thread Csányi Pál
Hi, I just want to know why the following SQLite query does not work as I expected? WITH RECURSIVE dates(dateD) AS ( VALUES('2015-10-03') UNION ALL SELECT date(dateD, '+1 year') FROM dates WHERE date(dateD) <= date('2016-11-01') ) SELECT max(dateD), count(dateD) FROM dates; 2017-10-03|3 which is

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Simon Slavin
On 2 Aug 2018, at 7:44pm, John R. Sowden wrote: > another point that I did not make clear. The accounting programs are not > associated with the technical programs, different people, different security > access. The tech databases and programs are in portable computers that go > out in the

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Keith Medcalf
>SQLite3 stores the table (.DBF) and all the indexes associated with >that table in a single file called a database. You can also have >multiple tables in one database (rather than one table per file) and >all the indexes associates with all those tables are stored in the >same database file.

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Keith Medcalf
You misunderstand how dBase databases work. An index is created on a table (.DBF file) and stored in an index file (.NDX). You can have multiple indexes associated with a single .DBF file (which means multiple .NDX files). FoxPro has a non-standard index format that permits the multiple

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Igor Korot
Hi, On Thu, Aug 2, 2018 at 1:44 PM, John R. Sowden wrote: > another point that I did not make clear. The accounting programs are not > associated with the technical programs, different people, different security > access. The tech databases and programs are in portable computers that go > out

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread John R. Sowden
another point that I did not make clear.  The accounting programs are not associated with the technical programs, different people, different security access.  The tech databases and programs are in portable computers that go out in the field, but not the accounting, etc.  There indexes would

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread David Raymond
The "index" is on what's actually in the "table". So the index on each database's tables are always up to date. What you're saying is that each satellite database should have a complete, up to date copy of the customer ID "table". The only way to do that would be to have a centrally accessible

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Igor Korot
Hi, On Thu, Aug 2, 2018 at 1:27 PM, John R. Sowden wrote: > I made a mistake. I should have said table, not database. My concern is if > I have 4 databases each with tables associated with a particular use, like > accounting, technical, etc., which may reside on different computers, how do > I

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread John R. Sowden
I made a mistake.  I should have said table, not database.  My concern is if I have 4 databases each with tables associated with a particular use, like accounting, technical, etc., which may reside on different computers, how do I keep the index in each database file current.  I assume that I

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread Simon Slavin
On 2 Aug 2018, at 6:11pm, John R. Sowden wrote: > I do not want these databases to all reside in one sqlite file. How do I > index each database on this customer account number when each database and > associated index are in separate files? Is this what seems to be referred to > as an

Re: [sqlite] Common index for multiple databases

2018-08-02 Thread David Raymond
Each SQLite file will be its own database with its own tables and indexes. So for each database you just define an index on the appropriate tables. I'm not sure why you want multiple files, but it's fine. You can always have one master db from which you ATTACH the others as needed. You just

[sqlite] Common index for multiple databases

2018-08-02 Thread John R. Sowden
I have been reviewing sqlite for a couple of years, but still use foxpro.  I have a question regarding an index issue. Currently I have several types of databases (in foxpro, one per file) that all point to an index of a common field, a customer account number.  The databases are for

Re: [sqlite] Add Column with "If Not Exists"

2018-08-02 Thread Richard Hipp
On 8/1/18, Charles Leifer wrote: > You can simply use: > > PRAGMA table_info('my_table') > > To get a list of columns, which you can check against and then > conditionally add your column. From C-code, you can use the sqlite_table_column_metadata() interface [1] to quickly check for the

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread David Raymond
The way I it happens with deferred foreign keys is this (Gut feeling from observations. Experts please correct me if I'm way off): Keep track with an integer, let's call it "netBroken" While netBroken = 0 then when you insert a row you don't have to check if anything references it, only if it

Re: [sqlite] In tea/configure.ac why isn't --with-system-sqlite default on linux?

2018-08-02 Thread Peter Da Silva
This is also a problem on FreeBSD, I just haven’t noticed it because the ports guys are on top of keeping their sqlite3 build up to date, and they do the smart thing and build --with-system-sqlite. If you’re building sqlite-autoconf-3xx/tea, then why would the “system sqlite” ever be

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Dominique Devienne
On Thu, Aug 2, 2018 at 10:34 AM Keith Medcalf wrote: > > Yes. Look at the CREATE TABLE for table A (completely ignore table B as > it serves no purpose whatsoever) > Rah, silly me... I assumed A.aid referenced B.id. Why have a B table at all then. > Also, if you create an index on the

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Keith Medcalf
This is the line that create the violation: sprintf(s, "insert into a values(%d, 'The name field for %d', %d)", NROWS+i, i, NROWS+i+VIOLATION); if VIOLATION is 0 then you are inserting a record with id = NROWS+i and aid = NROWS+i. Since the referential constraint is that aid

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Keith Medcalf
Yes. Look at the CREATE TABLE for table A (completely ignore table B as it serves no purpose whatsoever) If you change the database to write to a database on disk so you can examine it after (or modify doSql so that it outputs the SQL so that you can read it) everything will become much

Re: [sqlite] [EXTERNAL] Re: Is pragma index_list without supplied table name valid SQL?

2018-08-02 Thread Bart Smissaert
I didn't say I knew how you to do it in SQLitespeed, I said you your use-case was very likely the same as mine. I did notice the new keyword API and will move to 3.24 so I can use that. Will look further into using explain to solve this problem, but guess the output of explain may change in future

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Dominique Devienne
On Thu, Aug 2, 2018 at 9:35 AM Keith Medcalf wrote: > You observe no violation when VIOLATION is 0 because there is no > referential integrity violation to report ... > Really Keith? Parent IDs are in range [0, NROWS) Child/FK IDs inserted are in range [NROWS, 2*NROWS) How's that not an FK

[sqlite] Best approach for applying DB migrations to an existing SQLite DB

2018-08-02 Thread Phani Rahul Sivalenka
Our application is an ASP.NET MVC application with an N-Tier architecture running on Mono in Linux. Interaction with SQLite DB is done through ADO.Net using System.Data.SQLite provider. Can you suggest best approach for implementing and applying migrations to the DB?

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Dominique Devienne
On Thu, Aug 2, 2018 at 12:57 AM Yuri wrote: > The attached testcase injects the foreign key violation into a long > transaction. This makes the remainder of the transaction much slower, > even though the foreign key is deferred, and should only be checked in > the end of the transaction. >

Re: [sqlite] Violated failed foreign key constraint delays the rest of transaction ; Some foreign key violations don't trigger the error at all

2018-08-02 Thread Keith Medcalf
You observe no violation when VIOLATION is 0 because there is no referential integrity violation to report ... However, you are correct that when inserting data the as shown in your code (where there is a referential integrity violation) the insertion is much slower after the violation