[sqlite] SQLite3 database slowdown at 1GB

2016-09-26 Thread Domingo Alvarez Duarte
Hello Richard ! I'm trying to create a sqlite3 database with some pubic data (the sqlite3 dump gziped is 9184MB) that took a very long time to build (2 days tested on linux and os x). I was trying to vacuum it to see if I could get better performance but it take too long so I did a ".dump" (2

[sqlite] In-Memory database PRAGMA read_uncommitted

2016-04-24 Thread Domingo Alvarez Duarte
Hello ! I posted this some time ago, it's a program to test and tune sqlite with concurrency: https://gist.github.com/mingodad/79225c88f8dce0f174f5 Maybe it can be util to you and if you ! Cheers !

[sqlite] The LIKE optimization breaks user defined like functions

2016-04-16 Thread Domingo Alvarez Duarte
Hello ! Today I decided to see why my applications stop using index when using my user defined "like" function and it seems that during the creation of the "LIKE" optimization sqlite3 made some special settings to the builtin "like" function but didn't exposed it to third party developers. I

[sqlite] Patch that add table alias capability to delete/updatestatements.

2016-04-15 Thread Domingo Alvarez Duarte
Hello ! Here is a patch that add table alias capability to delete/update statements. https://github.com/mingodad/sqlite/commit/817636a48e5c92f7b747f3ddf487cbda21ec 8fa5 Cheers ! ?

[sqlite] Patch that add table alias capability to delete/update statements.

2016-04-14 Thread Domingo Alvarez Duarte
Hello ! Here is a patch that add table alias capability to delete/update statements. https://github.com/mingodad/sqlite/commit/817636a48e5c92f7b747f3ddf487cbda21ec 8fa5 Cheers !

[sqlite] Table alias not accepted with "delete/update" ?

2016-04-14 Thread Domingo Alvarez Duarte
PostgreSQL do accept table? alias on both delete/update.

[sqlite] Table alias not accepted with "delete/update" ?

2016-04-14 Thread Domingo Alvarez Duarte
ave implemented proprietary extenstions which permit the assignment >of table alias names in the UPDATE clause. > > >>-Original Message- >> From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users- >> bounces at mailinglists.sqlite.org] On Beh

[sqlite] Why pragma case_sensitive_like do not return theactualsetting ?

2016-04-14 Thread Domingo Alvarez Duarte
Re: [sqlite] Why pragma case_sensitive_like do not return >theactualsetting ? > > On 4/14/16, Domingo Alvarez Duarte wrote: > >>Also I'm noticing that when the "pragma case_sensitive_like" is executed >> there is two internal functions >> "sqlit

[sqlite] Table alias not accepted with "delete/update" ?

2016-04-14 Thread Domingo Alvarez Duarte
Hello ! Today I'm trying to execute this statememt and it works: SELECT * FROM __tables_metadata a WHERE NOT EXISTS(SELECT 1 FROM sqlite_master where tbl_name=a.name); Then I tried this one and it fails: DELETE FROM __tables_metadata a WHERE NOT EXISTS(SELECT 1 FROM sqlite_master where

[sqlite] Why pragma case_sensitive_like do not return theactualsetting ?

2016-04-14 Thread Domingo Alvarez Duarte
It seems that we have a mess here. How would we overload/replace the "like" function with things like this done internally by sqlite without public documentaion ? Also I do not see a setting of SQLITE_DETERMINISTIC isn't like deterministic ? Cheers ! /* ** Set the LIKEOPT flag on the 2-

[sqlite] Why pragma case_sensitive_like do not return theactualsetting ?

2016-04-14 Thread Domingo Alvarez Duarte
See bellow the setLikeOptFlag internal function, if we have installed custom "like" functions what would be the return value of "sqlite3FindFunction" be ? It seems that it needs to add the collation also to be able to find the correct function to set the flags. /* ** Set the LIKEOPT flag on th

[sqlite] Why pragma case_sensitive_like do not return the actualsetting ?

2016-04-14 Thread Domingo Alvarez Duarte
Also I'm noticing that when the "pragma case_sensitive_like" is executed there is two internal functions "sqlite3RegisterLikeFunctions/setLikeOptFlag" been called to swap the built-in LIKE and GLOB functions. What this mean for user installed collates/functions overloading ? I have some custo

[sqlite] Why pragma case_sensitive_like do not return the actual setting ?

2016-04-14 Thread Domingo Alvarez Duarte
Hello ! I'm trying to find why in my application I'm getting a case sensitive like behavior by default and when trying to see what is the actual setting using "pragma case_sensitive_like;" without a boolean parameter I was expecting to get back the actual active setting but nothing comes back.

[sqlite] regression in 3.12.0 vs. 3.11.0, column type information in PRAGMA missing

2016-04-06 Thread Domingo Alvarez Duarte
+1 for INFORMATION SCHEMA ! > Wed Apr 06 2016 04:18:10 PM CEST from "Michael Schlenker" > Subject: Re: [sqlite] regression in 3.12.0 vs. >3.11.0,column type information in PRAGMA missing > > > btw. would be nice to have an INFORMATION SCHEMA style view for this > info instead of a pragma. > >

[sqlite] FOREIGN KEY constraint failed

2016-04-05 Thread Domingo Alvarez Duarte
onstraint failed > > On 2016/04/04 10:43 PM, Domingo Alvarez Duarte wrote: > >>Thanks for reply ! >> >> I already sent a proposal to Richard to add a pragma "PRAGMA DEBUG_MODE" >>and >> when set throw any kind of error to stderr/sqlite3_(hook

[sqlite] How to order by absolute value ?

2016-04-05 Thread Domingo Alvarez Duarte
uld be good. I'm guessing in your first example the ORDER BY only applies >to the last SELECT. > > Graham. > > > Sent from Samsung Mobile > > Original message > From: Domingo Alvarez Duarte > Date: 05/04/2016 09:47 (GMT+00:00) > T

[sqlite] How to order by absolute value ?

2016-04-05 Thread Domingo Alvarez Duarte
Thank you for reply ! Yes it works ! > Tue Apr 05 2016 11:12:03 AM CEST from "R Smith" >Subject: Re: [sqlite] How to order by absolute value ? > > On 2016/04/05 10:47 AM, Domingo Alvarez Duarte wrote: > >>Hello ! >> >> I want to achieve this:

[sqlite] How to order by absolute value ?

2016-04-05 Thread Domingo Alvarez Duarte
Hello ! I want to achieve this: select 1 as num union select 3 as num union select -2 as num order by abs(num) -- result [1, -2, 3] But it doesn't work this works but doesn't show what I want: select abs(num) from ( select 1 as num union select 3 as num union select -2 as num ) order

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
o flag the error, it doesn't throw a dice to do it. Cheers ! > Mon Apr 04 2016 10:35:35 PM CEST from "Petite Abeille" > Subject: Re: [sqlite] FOREIGN KEY constraint >failed > > >>On Apr 4, 2016, at 6:14 PM, Richard Hipp wrote: >> >> On 4/4/16,

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
. Cheers ! > Mon Apr 04 2016 10:17:00 PM CEST from "Tim Streater" > Subject: Re: [sqlite] FOREIGN KEY constraint failed > > On 04 Apr 2016 at 17:36, Domingo Alvarez Duarte > wrote: > > >>Fair enough ! >> >> But even then could it have le

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
to determine what happened, not > the SQLite dev team. > > My two cents. > > > On Mon, Apr 4, 2016 at 1:49 PM, Domingo Alvarez Duarte < > sqlite-mail at dev.dadbiz.es> wrote: > > >>Thank you for reply ! >> >> Good point, but for sure ther

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
can be run without problem. ? > Mon Apr 04 2016 07:49:57 PM CEST from "Domingo Alvarez Duarte" > Subject: Re: [sqlite] FOREIGN KEY constraint >failed > > Thank you for reply ! > > Good point, but for sure there is great minds here that can come with a > solutio

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
be done lazy so if no violations occur no hash table is ever created. The actual situation is better than nothing but it's still frustrating ! Cheers ! > Mon Apr 04 2016 07:39:19 PM CEST from "Clemens Ladisch" > Subject: Re: [sqlite] FOREIGN KEY constraint failed

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
Fair enough ! But even then could it have let's say a place for record the last foreign key violation ? Only one place will be better than nothing, it can be overwritten every time a foreign key is found and at least we could have a message like this: "FOREIGN KEY constraint failed (x : la

[sqlite] FOREIGN KEY constraint failed

2016-04-04 Thread Domingo Alvarez Duarte
Hello ! I know this was discussed here before several times, but it seems that still we have poor solution for this on sqlite: FOREIGN KEY constraint failed I have some databases with lots of foreign keys and I sometimes I need to change the structure of some tables and I get this message,

[sqlite] Difference between sqlite3 executable and application

2016-04-04 Thread Domingo Alvarez Duarte
code>? AND post_code Mon Apr 04 2016 12:39:35 PM CEST from "Simon Slavin" > Subject: Re: [sqlite] Difference between sqlite3 >executable and application > > On 4 Apr 2016, at 10:47am, Domingo Alvarez Duarte > wrote: > > >>It would be a gross mistake

[sqlite] Difference between sqlite3 executable and application

2016-04-04 Thread Domingo Alvarez Duarte
It would be a gross mistake if the databases weren't the same. It's an existing and populated database. ? > Mon Apr 04 2016 11:31:45 AM CEST from "Simon Slavin" > Subject: Re: [sqlite] Difference between sqlite3 >executable and application > > On 4 Ap

[sqlite] Difference between sqlite3 executable and application

2016-04-04 Thread Domingo Alvarez Duarte
ot;Simon Slavin" > Subject: Re: [sqlite] Difference between sqlite3 >executable and application > > On 4 Apr 2016, at 9:53am, Domingo Alvarez Duarte > wrote: > > >>What compile time options if any can cause this ? >> > I recommend you compare the re

[sqlite] Difference between sqlite3 executable and application

2016-04-04 Thread Domingo Alvarez Duarte
fy sqlite3 using indexes and my application not using then. Cheers ! > Mon Apr 04 2016 10:59:17 AM CEST from "Simon Slavin" > Subject: Re: [sqlite] Difference between sqlite3 >executable and application > > On 4 Apr 2016, at 9:53am, Domingo Alvarez Duarte > wrote: >

[sqlite] FTS5 problem ? more than 3,000 queries !

2016-04-04 Thread Domingo Alvarez Duarte
On 04/04/2016 05:43 AM, Domingo Alvarez Duarte wrote: > >>Hello ! >> >> I'm testing fts5 on a database with data from hacker news (around 13M >> records) and it took more than 2 hours to index, I have this application >>to >> log all queries executed for

[sqlite] Difference between sqlite3 executable and application

2016-04-04 Thread Domingo Alvarez Duarte
Hello ! I have an application that uses sqlite3 and in one database I get different responses between the sqlite3 executable and my application that uses the same sqlite3.c: The query: explain query plan select * from companies where post_code like 'rg%'; My application: 0??? 0??? 0?

[sqlite] FTS5 problem ? more than 3,000 queries !

2016-04-04 Thread Domingo Alvarez Duarte
Hello ! I'm testing fts5 on a database with data from hacker news (around 13M records) and it took more than 2 hours to index, I have this application to log all queries executed for debugging and although the search isn't slow it's making a lot of queries. Is this normal ? More than 3,000 qu

[sqlite] Understanding table and database locking mechanism inshared cache mode and normal mode

2016-03-31 Thread Domingo Alvarez Duarte
Hello ! You can check the behavior of sqlite3 in a multi thread environment with this program: https://gist.github.com/mingodad/79225c88f8dce0f174f5 I did it to test how fast sqlite3 can be on a system (cpu/memory/threads) and with it you can from the command line change the following para

[sqlite] FTS5 Prefix Indexes

2016-03-28 Thread Domingo Alvarez Duarte
Hello ! Is it possible to expose commands to create/remove prefix indexes ? Something like (or any other friendly way to do it without reindex the whole database): INSERT INTO ft(ft) VALUES('remove prefix 2 3'); INSERT INTO ft(ft) VALUES('remove prefix 2'); INSERT INTO ft(ft) VALUES('creat

[sqlite] Help needed for COPY Command.

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! Have you ever heard about user friendly "alias/shortcut" ? Cheers ! > Fri Mar 25 2016 10:48:52 PM CET from "James K. Lowden" > Subject: Re: [sqlite] Help needed for COPY >Command. > > On Fri, 25 Mar 2016 17:18:16 +0100 > "Domingo Al

[sqlite] Testing sqlite wal performance OS X memory/disk

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! Comparing the three systems tested: raspibian raspberry pi3: 53us linux celeron: 12us os x i5: 7.7us One thing that I was expecting is that when using all core/threads (and because basically all data in in memory/cache) that I'll have around 100% cpu load but I don't, so ther

[sqlite] Testing sqlite wal performance OS X memory/disk

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! Here is the same tests on a raspberry pi3: raspberry pi3 memory database 1 thread: Update rate: total=214229, last_total=194712, rate=19517, avg1=19517, avg2=19475 Update busy rate: total=0, last_total=0, rate=0, avg1=0, avg2=0 Update failed rate: total=0, last_total=0, rate=0, avg

[sqlite] Testing sqlite wal performance OS X memory/disk

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! I modified a bit the program to somehow test sqlite/system performance at https://gist.github.com/mingodad/79225c88f8dce0f174f5 , now we can also specify a memory database and doing so on a mac mini i5 2/4 cores surprisingly memory database when shared with 4 threads is slower than di

[sqlite] Help needed for COPY Command.

2016-03-25 Thread Domingo Alvarez Duarte
> >>On 3/25/16, Simon Slavin wrote: >> >> >>>There is no COPY command in SQLite. >>> >> There *was* a COPY command, way back in SQLite 2.x. It was omitted >> from SQLite3, circa 2004. >> > Wow. > > On 25 Mar 2016, at 12:22

[sqlite] Help needed for COPY Command.

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! I also asked the same thing a few days ago and received the same answer. But discovered that it can be somehow emulated by "import tablename" using csv. Would be a good idea to have the complementary of ".import" -> ".export". Cheers ! Cheers ! ?

[sqlite] Testing sqlite wal performance, and surfacing a problem

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! Again the problem was my fault, I was not checking the return code of setting "PRAGMA synchronous=0" and for some threads it was SQLITE_BUSY so they started in synchronous mode and that slowdown everything ! Sorry by the noise ! The updated program to test somehow sqlite performanc

[sqlite] Testing sqlite wal performance, and surfacing a problem

2016-03-25 Thread Domingo Alvarez Duarte
Hello ! After fight a bit with sqlite3 "wal" mode I wrote a small multi thread program to test it's performance https://gist.github.com/mingodad/79225c88f8dce0f174f5 , doing so and testing on linux and os x I dicovered that os x threads are around 50% slower than linux. Sqlite3 seems to have

[sqlite] SQLite with wall enabled what's wrong C demo

2016-03-24 Thread Domingo Alvarez Duarte
d why not update the same record instead of adding a new one in such situation ? I'll try with a bigger record to see what happens. Cheers ! ? > Thu Mar 24 2016 11:55:11 AM CET from "Domingo Alvarez Duarte" > Subject: [sqlite] SQLite with wall enabled >w

[sqlite] SQLite with wall enabled what's wrong C demo

2016-03-24 Thread Domingo Alvarez Duarte
abled >what's wrong C demo > > On Thu, Mar 24, 2016 at 11:55 AM, Domingo Alvarez Duarte < > sqlite-mail at dev.dadbiz.es> wrote: > > >>There is something wrong with the program or with sqlite3 ? >> >> >> >> >> rc

[sqlite] SQLite with wall enabled what's wrong C demo

2016-03-24 Thread Domingo Alvarez Duarte
Hello ! I'm sending here a C program that demonstrates the problem of sqlite3 in wall mode, in this simple example of a simulated session management for a web server when executing the wall log file will grow and grow till eat all our disk. ? There is something wrong with the program

[sqlite] SQLite with wall enabled what's wrong

2016-03-24 Thread Domingo Alvarez Duarte
ith wall enabled >what's >wrong > > On 23 Mar 2016, at 11:18pm, Domingo Alvarez Duarte > wrote: > > >>I'm using prepared statements and "sqite3_step == SQLITE_ROW" undeer >> "stmt.next_row()". >> > Which programming lang

[sqlite] SQLite with wall enabled what's wrong

2016-03-24 Thread Domingo Alvarez Duarte
Hello ! Thanks for reply ! I'm using prepared statements and "sqite3_step == SQLITE_ROW" undeer "stmt.next_row()". Cheers ! > Thu Mar 24 2016 12:13:22 AM CET from "Simon Slavin" > Subject: Re: [sqlite] SQLite with wall enabled >what's >wrong > > How are you executing your SQLite call

[sqlite] SQLite with wall enabled what's wrong

2016-03-24 Thread Domingo Alvarez Duarte
Hello again ! After all my tests I conclude that the actuall implementation of sqlite "wall" mode needs active manual management to not grow unbounded and with multiuser/long running proesses sharing the same database sqlite will eat all our disk space, because the chance of a process get the

[sqlite] SQLite with wall enabled what's wrong

2016-03-23 Thread Domingo Alvarez Duarte
#x27;s total size -rw-r--r-- 1 ? 562626560 Mar 23 22:26 hacker-news-items-diff.db //550MB db with all users and users_submitted ? > Wed Mar 23 2016 11:10:56 PM CET from "Domingo Alvarez Duarte" > Subject: [sqlite] SQLite with wall enabled >what's wrong > >

[sqlite] SQLite with wall enabled what's wrong

2016-03-23 Thread Domingo Alvarez Duarte
Hello ! Can someone give any insight here ? For me the main problem is the wall size that becomes stable at 387MB (+- 8% of the database, but around 15 times the size of the tables it's manipulating). It seems to me that something is not working as advertised. I'm testing sqlite3 with

[sqlite] SQLite RBU the missing piece !

2016-03-23 Thread Domingo Alvarez Duarte
> On 23 Mar 2016, at 6:45pm, Domingo Alvarez Duarte > wrote: > > >>The sqlite rbu extension concept is interesting but it's missing the >> generation of the diff files on the fly, we need something like >> sqlite3_trace, sqlite3_update_hook or better yet a prag

[sqlite] SQLite RBU the missing piece !

2016-03-23 Thread Domingo Alvarez Duarte
Hello ! The sqlite rbu extension concept is interesting but it's missing the generation of the diff files on the fly, we need something like sqlite3_trace, sqlite3_update_hook or better yet a pragma: PRAGMA generate_rbu=ON; And with that all insert/update/delete operation would generate th

[sqlite] dump only data, change schema, reload

2016-03-23 Thread Domingo Alvarez Duarte
Hello ! I had this problem before and asked to add this option to sqlite but somehow it was not added so now I'm submitting here a patch that adds this functionality to shell.c updated to today repository. Please consider add this to sqlite ! Cheers ! ? --- /third-party/sqlite3/src/s

[sqlite] FTS5 "constraint failed"

2016-03-22 Thread Domingo Alvarez Duarte
22 2016 06:23:53 PM CET from "Dan Kennedy" > Subject: Re: [sqlite] FTS5 "constraint failed" > > On 03/23/2016 12:06 AM, Domingo Alvarez Duarte wrote: > >>Hello ! >> >> After seeing several times work/commits on fts5 I decided to try it on a &g

[sqlite] FTS5 "constraint failed"

2016-03-22 Thread Domingo Alvarez Duarte
? -ledit -ldl -lpthread -lm -Wl,-rpath -Wl,/usr/local/lib ? > Tue Mar 22 2016 06:06:55 PM CET from "Domingo Alvarez Duarte" > Subject: [sqlite] FTS5 "constraint failed" > > Hello ! > > After seeing several times work/commits on fts5 I decided to try it on a

[sqlite] FTS5 "constraint failed"

2016-03-22 Thread Domingo Alvarez Duarte
Hello ! After seeing several times work/commits on fts5 I decided to try it on a table shown bellow, and when trying to populate it I get this error message: sqlite> INSERT INTO fts_idx_items(fts_idx_items) VALUES('rebuild'); Error: constraint failed The table has 12,000,000 records and it

[sqlite] SQLite big tables create index in parallel

2016-03-19 Thread Domingo Alvarez Duarte
Mar 19 2016 09:56:44 PM CET from "Simon Slavin" > Subject: Re: [sqlite] SQLite big tables create index >in parallel > > On 19 Mar 2016, at 8:54pm, Domingo Alvarez Duarte > wrote: > > >>When you say [The majority of the time in 'CREATE INDEX' is spent

[sqlite] SQLite big tables create index in parallel

2016-03-19 Thread Domingo Alvarez Duarte
in" > Subject: Re: [sqlite] SQLite big tables create index >in parallel > > On 19 Mar 2016, at 11:03am, Domingo Alvarez Duarte > wrote: > > >>Would be nice if sqlite provide a way to create indexes in parallel, I mean >> when working with big tables and creat

[sqlite] SQLite big tables create index in parallel

2016-03-19 Thread Domingo Alvarez Duarte
Hello ! Would be nice if sqlite provide a way to create indexes in parallel, I mean when working with big tables and creating several indexes the time spent scanning the whole database/table is considerable and it's the same for each "create index", we could have a big time/cpu/disk seek saving

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
_dml("PRAGMA wal_checkpoint(FULL);"); ??? ??? stmt_find.bind(1, id); ??? ??? count = 0; ??? } ??? } ??? stmt_find.finalize(); ??? stmt_update.finalize(); } Cheers ! > Fri Mar 18 2016 03:50:06 PM CET from "Domingo Alvarez Duarte" > Subject: Re: [sqlite] Wall checkpoint no

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
Hello ! So why I was having the problem in the scenario described on the original post ? Cheers ! > Fri Mar 18 2016 04:12:41 PM CET from "Richard Hipp" >Subject: Re: [sqlite] Wall checkpoint not running > > On 3/18/16, Domingo Alvarez Duarte wrote: > &g

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
qlite] Wall checkpoint not running > > On 3/18/16, Domingo Alvarez Duarte wrote: > > >>I have a database with more than 12,000,000 rows in one table and around >>6GB >> in size, it's in wall mode and when I run this program the wall file grows >> unbou

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
to achieve it ? (no redundant checkpoints beween several applications) Cheers ! * * > Fri Mar 18 2016 03:50:30 PM CET from "Richard Hipp" >Subject: Re: [sqlite] Wall checkpoint not running > > On 3/18/16, Domingo Alvarez Duarte wrote: > > >>I have a da

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
u know how to do that ? (avoid redundant checkpoints between several applications ?) Cheers ! > Fri Mar 18 2016 03:16:24 PM CET from "Simon Slavin" > Subject: Re: [sqlite] Wall checkpoint not running > > On 18 Mar 2016, at 2:00pm, Domingo Alvarez Duarte > wrote:

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
Hello ! After writing my problem and thinking about it I found that my unique reader probably is the one blocking the checkpoint, so I modified it and now wall file do not grows unbound. But how to coordinate several applications to not been doing redundant checkpoints ? Maybe we need a pr

[sqlite] Wall checkpoint not running

2016-03-18 Thread Domingo Alvarez Duarte
Hello ! I have a database with more than 12,000,000 rows in one table and around 6GB in size, it's in wall mode and when I run this program the wall file grows unbound, it doesn't seem that the checkpoint is doing any work. I read the documentation and as I understood the automatic checkpoint

[sqlite] sqlite3_progress_handler do not seem to be called byvacuum

2016-03-17 Thread Domingo Alvarez Duarte
ect: Re: [sqlite] sqlite3_progress_handler do not seem to be called >byvacuum > > On 3/17/16, Domingo Alvarez Duarte wrote: > >>Hello ! >> >> Hello I have an application that uses sqlite3_progress_handler to update a >> gui application and when we do a "vacu

[sqlite] sqlite3_progress_handler do not seem to be called byvacuum

2016-03-17 Thread Domingo Alvarez Duarte
heers ! > Thu Mar 17 2016 08:51:11 PM CET from "Richard Hipp" >Subject: Re: [sqlite] sqlite3_progress_handler do not seem to be called >byvacuum > > On 3/17/16, Domingo Alvarez Duarte wrote: > >>Hello ! >> >> Hello I have an application th

[sqlite] sqlite3_progress_handler do not seem to be called by vacuum

2016-03-17 Thread Domingo Alvarez Duarte
Hello ! Hello I have an application that uses sqlite3_progress_handler to update a gui application and when we do a "vacuum" on a big database the gui get frozen, maybe vacuum should use different parameters to call sqlite3_progress_handler I'm using actually: db.progress_handler(1000, sqlite

[sqlite] Sqlite do not use index collate nocase

2016-03-17 Thread Domingo Alvarez Duarte
a lot cheaper to scan using the index and then decide if the record is valid by comparing it case sensitive afterwards. Cheers ! > Thu Mar 17 2016 07:24:46 PM CET from "Clemens Ladisch" > Subject: Re: [sqlite] Sqlite do not use index collate >nocase > > Domingo Alva

[sqlite] Sqlite do not use index collate nocase

2016-03-17 Thread Domingo Alvarez Duarte
Hello ! In one database I created an index with collate nocase but it seems that sqlite do not recognize it as a candidate index for a select. Bellow is the schema, the real database has more than a million rows. If I alter the table to include the collation or remove the collation from the i

[sqlite] How does your sqlite script binding handle aggregate UDFs?

2016-03-05 Thread Domingo Alvarez Duarte
Hello ! There is an user pointer that you pass and you can get it back using? https://www.sqlite.org/c3ref/user_data.html ! Cheers ! > Sat Mar 05 2016 9:22:23 pm CET CET from "Stephan Beal" > Subject: [sqlite] How does your sqlite script >binding handle aggregate UDFs? > > Hi, all, > >

[sqlite] Changing the default page_size in 3.12.0

2016-03-05 Thread Domingo Alvarez Duarte
Sorry I mean resurrect it ? >Is it too hard to survive it ? > > >>SQLite version 2 supported COPY That was dropped when we moved to SQLite >> 3. >> > > > Also could the commands that take time to complete have an option to show the completion stats ? Cheers !

[sqlite] Changing the default page_size in 3.12.0

2016-03-05 Thread Domingo Alvarez Duarte
Is it too hard to survive it ? > > SQLite version 2 supported COPY That was dropped when we moved to SQLite >3. > > > >

[sqlite] Changing the default page_size in 3.12.0

2016-03-05 Thread Domingo Alvarez Duarte
Hello ! I have a question, is it enough to vacuum a database to update to the new page size ? Or need a complete backup restore ? Also I have a database not too big 6GB but only 12GB free space on that disk, if I try a dump/restore with the actual sqlite3 maybe I'll run out of disk space,

[sqlite] How this /*A-overwrites-S*/ should work ?

2016-02-24 Thread Domingo Alvarez Duarte
Thanks for the answer ! > Wed Feb 24 2016 6:38:46 pm CET CET from "Richard Hipp" > > > The /*A-overwrites-S*/ comment is processed by Lemon, and Lemon has no > knowledge of #ifdefs. > > > I can understand that but it will confuse other people like it did to me ! Cheers !

[sqlite] How this /*A-overwrites-S*/ should work ?

2016-02-24 Thread Domingo Alvarez Duarte
I solved it setting a variable before " /*A-overwrites-S*/", but I's still confusing it's usage inside a wrapped "#ifdef" that supposed only will be executed conditionally. Cheers ! oneselect(A) ::= SELECT|XSELECT(S) distinct(D) selcollist(W) from(X) where_opt(Y) groupby_opt(

[sqlite] How this /*A-overwrites-S*/ should work ?

2016-02-24 Thread Domingo Alvarez Duarte
Here is the full changes I did in parser.y: ? %endif SQLITE_OMIT_COMPOUND_SELECT oneselect(A) ::= SELECT|XSELECT(S) distinct(D) selcollist(W) from(X) where_opt(Y) groupby_opt(P) having_opt(Q) orderby_opt(Z) limit_opt(L). { #if SELECTTRACE_ENABLED ? Token s = S; /*A-overwrites

[sqlite] How this /*A-overwrites-S*/ should work ?

2016-02-24 Thread Domingo Alvarez Duarte
Hello ! I have a modification on the parser.y for my own purposes and it was been parsed till the introduction of " /*A-overwrites-S*/" and it seems that something is wrong on parser.y see bellow, after the SELECT specification the " /*A-overwrites-S*/" is wrapped by "#if SELECTTRACE_ENABLED" th

[sqlite] Fossil sqlite clone problem

2016-02-09 Thread Domingo Alvarez Duarte
Thank you again for the answer ! Now I understand the problem, maybe fossil could give an extra bit of info when it's invoked like "fossil update" maybe something like: You are now at "branch_name" and it's behind "trunk" by X commits. ? > Tue Feb 09 2016 6:46:13 pm CET CET from "Richar

[sqlite] Fossil sqlite clone problem

2016-02-09 Thread Domingo Alvarez Duarte
Thanks for the answer ! Although it seems a bit weird this behavior of get stuck on one specific branch. Cheers ! > OTOH, if that tag is checked out, which your output indicates is the case, > > >>then 'update' will apply no changes because that's the last commit in that >> branch. You

[sqlite] Fossil sqlite clone problem

2016-02-09 Thread Domingo Alvarez Duarte
? > OTOH, if that tag is checked out, which your output indicates is the case, > then 'update' will apply no changes because that's the last commit in that > branch. You'll need to 'fossil update trunk' once to get back to the trunk. > > Thanks "fossil update trunk" solved the problem !

[sqlite] Fossil sqlite clone problem

2016-02-09 Thread Domingo Alvarez Duarte
Can you try doing this one from the sqlite tree: > > fossil pull --verily > > in the past that's helped people reporting problems about a repo silently > failing to pull past a certain version. > > ? fossil pull --verilyPull from http://www.sqlite.org/src Round-trips: 2?? Artifacts sen

[sqlite] Fossil sqlite clone problem

2016-02-09 Thread Domingo Alvarez Duarte
Hello ! I frequently follow the updates of fossil and sqlite but I noticed that my sqlite clone stop updating the source tree at this commit : fossil status repository:?? /sqlite3/../sqlite.fossil local-root:?? /sqlite3/ config-db:?? /.fossil checkout: ec653cbcae74ecab4ec9df05

[sqlite] Wish list: allow developers use the power of sqliteparser

2016-01-18 Thread Domingo Alvarez Duarte
Is this something crazy to ask as a developer ? I think that even for the author something like this would make several tasks easier. Ideally I would like to feed the parser with an sql string, get it's syntax tree, maybe do some rewrite and feed it execute it, this possibility can open the d

[sqlite] Wish list: allow developers use the power ofsqliteparser

2016-01-18 Thread Domingo Alvarez Duarte
answer for the original question ? Anyone used it ? What's it's output like ? Cheers ! ? > Mon Jan 18 2016 3:36:05 pm CET CET from "Simon Slavin" > Subject: Re: [sqlite] Wish list: allow developers >use >the power ofsqliteparser > > On 18 Jan 2

[sqlite] Wish list: allow developers use the power of sqliteparser

2016-01-18 Thread Domingo Alvarez Duarte
your freedom and can provide a way to reuse the parser power ? Cheers ! ? > Mon Jan 18 2016 2:15:04 pm CET CET from "Richard Hipp" >Subject: Re: [sqlite] Wish list: allow developers use the power of >sqliteparser > > On 1/18/16, Domingo Alvarez Duarte wrote: &g

[sqlite] Wish list: allow developers use the power ofsqliteparser

2016-01-18 Thread Domingo Alvarez Duarte
st: allow developers >use >the power ofsqliteparser > > On 18 Jan 2016, at 11:51am, Domingo Alvarez Duarte > wrote: > > >>The idea is to have a way to get a syntax tree of a sql string through the >> sqlite parser: synatx_tree sqlite3_sql_syntax_tree(const char *zS

[sqlite] Wish list: allow developers use the power of sqliteparser

2016-01-18 Thread Domingo Alvarez Duarte
On the right panel there is a live syntax tree of the sql on the left panel, we can edit the sql on the left panel and the syntax tree on the right panel is updated. The idea is to have a way to get a syntax tree of a sql string through the sqlite parser: synatx_tree sqlite3_sql_syntax_tree(cons

[sqlite] Wish list: allow developers use the power of sqlite parser

2016-01-18 Thread Domingo Alvarez Duarte
Hello Richard ! Can we have a sqlite function that given a sql string return something like this https://codeschool.github.io/sqlite-parser/demo/ => Syntax Tree ? syntax_tree sqlite3_sql_syntax_tree(const char *zSql); Possible return types for sqlite3_sql_syntax_tree: - json structur

[sqlite] SQLite3 with support for _Decimal64 on github

2016-01-17 Thread Domingo Alvarez Duarte
Hello ! I finally got time to publish my changes to sqlite3 to allow the use of _Decimal64/double/int64 as floating point unit by defining SQLITE_USE_DECIMAL, almost all tests pass with exception of direct binary manipulation of doubles (_Decimal64 have different binary representation) and "with

[sqlite] Bug in openDatabase function with patch

2016-01-16 Thread Domingo Alvarez Duarte
Hello ! When testing one application that uses sqlite3 with "-fsanitize=address" I was getting an error: ==1310==ERROR: AddressSanitizer: heap-use-after-free on address And after study the code I found that the problem is in the function openDatabase in src/main.c , it only happens when SQ

[sqlite] Work toward making the schema parsing logic simplier

2016-01-15 Thread Domingo Alvarez Duarte
Hello Richard ! Now that you are refactoring on the schema parsing could be nice if somehow sqlite3 expose the schema/sql parser for developers. A kind of sax style parser, with callbacks given option to do some rewrite/extend syntax ? (ideas welcome) An interesting project can be seem her

[sqlite] Setting SQLITE_OMIT_FLOATING_POINT has surprisingundocumented consequences

2016-01-15 Thread Domingo Alvarez Duarte
Hello ! When we declare SQLITE_OMIT_FLOATING_POINT what really happens is that this is activated (mainly #define double sqlite_int64): /* ** If compiling for a processor that lacks floating point support, ** substitute integer for floating-point */ #ifdef SQLITE_OMIT_FLOATING_POINT # define dou

[sqlite] whish list for 2016

2016-01-12 Thread Domingo Alvarez Duarte
I think this is a very good feature to have, thumbs up ! ? > Tue Jan 12 2016 9:50:33 pm CET CET from "James K. Lowden" > Subject: Re: [sqlite] whish list for 2016 > > On Fri, 8 Jan 2016 08:28:29 +0100 > Dominique Devienne wrote: > > >> >>>One way to do that would be to honor a special

[sqlite] Misspelling bug

2016-01-02 Thread Domingo Alvarez Duarte
Hello ! There is a misspelling bug on vdbe.c:3405 a missing "S" at the end of "SQLITE_ENABLE_CURSOR_HINT" ??? #ifdef SQLITE_ENABLE_CURSOR_HINT ? testcase( pOp->p2 & OPFLAG_SEEKEQ ); ??? #endif Cheers !

[sqlite] Do you mean utf8_fprintf on this commit ?

2015-12-31 Thread Domingo Alvarez Duarte
Hello mistachkin ! On this commit https://www.sqlite.org/src/info/8cf01fff232d15d3 do you mean "utf8_fprintf" instead of "utf8_printf" ? To be consistent with the underling wrapped function ! Cheers !

[sqlite] Magic number in sqlite source code

2015-12-31 Thread Domingo Alvarez Duarte
Hello Duncan ! I saw a very good point on your suggestion ! I'll use it when writing/refactoring code. Thanks a lot ! ? > Wed Dec 30 2015 11:51:54 pm CET CET from "Darren Duncan" > Subject: Re: [sqlite] Magic number in sqlite >source code > > On 2015-12-30 12:51 PM, Richard Hipp wr

[sqlite] Magic number in sqlite source code

2015-12-31 Thread Domingo Alvarez Duarte
Hello Richard ! Sorry again for the mess with format and missing parameter on the maybe more friendly function "ptrmapPutOvflPtr" ! I decided to really look at the code now (before I looked at the diffs and saw so many magic numbers "4"). I know that what I'm saying here not always

[sqlite] Magic number in sqlite source code

2015-12-31 Thread Domingo Alvarez Duarte
Hello Richard ! !!! Last message got messed format, sorry !!! I decided to really look at the code now (before I looked at the diffs and saw so many magic numbers "4"). I know that what I'm saying here not always I follow for n reasons but thanks for giving me a chance to take a moment and ref

<    1   2   3   >