Re: [sqlite] sqlite 3.31.1 crashes in SVN on OpenBSD/sparc64

2020-03-10 Thread Dominique Pellé
Stefan Sperling wrote: > > Does valgrind give any clues? > > Valgrind does not run on the OpenBSD/sparc64 platform, unfortunately. Would the address sanitizer (i.e. gcc -fsanitize=address, or clang -fsanitize=address) work on OpenBSD/sparc64? Regards Dominique

Re: [sqlite] Performance Issue on Large Table

2020-02-23 Thread Dominique Pellé
Chip Beaulieu wrote: > I also recently vacuumed the file. The issue really showed up after the FTS5 > was > setup on the table. I suspect it’s got something to do with the triggers more > than > the indexes. I am definitely open to suggestions. I’ve not been able to find > much > info on

[sqlite] Remarks/questions about SQLITE_OMIT_LOOKASIDE & SQLITE_OMIT_TWOSIZE_LOOKASIDE

2020-02-06 Thread Dominique Pellé
Hi Here are a few remarks and questions about SQLITE_OMIT_LOOKASIDE and SQLITE_OMIT_TWOSIZE_LOOKASIDE: 1) SQLite has SQLITE_OMIT_LOOKASIDE to disable lookaside memory completely at compilation time. However, I see that some of the lookaside code is not disabled by SQLITE_OMIT_LOOKASIDE. For

Re: [sqlite] CVE's opened on 3.30.1 status

2019-12-25 Thread Dominique Pellé
Raitses, Alex wrote: > Thanks a lot for the prompt response, > As far as I found in Fossil repository fixes for all CVE's , > excepting erroneously submitted CVE-2019-19646, were merged to Fossil. > Can you please estimate next official release of SQLite including these fixes? According to

Re: [sqlite] Heap Out of Bound Read in Sqlite

2019-12-24 Thread Dominique Pellé
Yongheng Chen wrote: > This seems a problem of gcc(Ubuntu 5.5.0-12ubuntu5~16.04). When > I use this specific version to compile sqlite, the problem can be repro. Unlikely to be a compiler issue. I can reproduce the bug with valgrind with SQLite shell built with: * gcc (Ubuntu

Re: [sqlite] Heap Out of Bound Read in Sqlite

2019-12-24 Thread Dominique Pellé
On Tue, Dec 24, 2019 at 5:48 PM Richard Hipp wrote: > > On 12/24/19, Yongheng Chen wrote: > > > > When we run it with sqlite compiled with asan, we got a heap overflow crash. > > > > The bug exists in the latest development code. > > Unable to repro. Tried tip of trunk and release, using gcc

Re: [sqlite] Performance vs. memory trade-off question

2019-12-14 Thread Dominique Pellé
Richard Hipp wrote: > On 12/14/19, Dominique Pellé wrote: > > > > I'm curious about what kind of change reduces memory per > > connection. > > > > The branch is here: > https://www.sqlite.org/src/timeline?r=mini-lookaside-take-2 > > I'll try to ad

Re: [sqlite] Performance vs. memory trade-off question

2019-12-14 Thread Dominique Pellé
Richard Hipp wrote: > A new feature on a branch has the following disadvantages: > > (1) It uses about 0.25% more CPU cycles. (Each release of SQLite is > normally about 0.5% faster, so enabling this feature is sort of like > going back by one-half of a release cycle.) > > (2) The code space

Re: [sqlite] https://www.sqlite.org/draft/gencol.html Typo

2019-10-29 Thread Dominique Pellé
Keith Medcalf wrote: > At the end of the second paragraph of section 2.1: > > Only VIRTUAL tables can be added using ALTER TABLE. > > should be > > Only VIRTUAL columns can be added using ALTER TABLE. Above typos is already corrected, but here are 2 other typos in the same page:

Re: [sqlite] Why sqlite3 running at different time can have very different speed?

2019-10-19 Thread Dominique Pellé
Peng Yu wrote: I have never seen such a dramatic difference in non-sqlite3 > operations. For example, cat has some difference but is ~13%. > > $ time cat file.sqa > /dev/null > > real0m7.282s > user0m0.067s > sys0m2.371s > $ time cat file.sqa > /dev/null > > real0m6.316s > user

Re: [sqlite] [patch] avoid dynamic alloc in vdbeSorterSort(...)

2019-10-09 Thread Dominique Pellé
Keith Medcalf wrote: > > On Monday, 7 October, 2019 14:58, Dominique Pellé < > dominique.pe...@gmail.com> wrote: > > >Here the allocated size is fixed (always 64 pointers), so alloca does > >not seem needed. > > >I wonder how many other functions could av

Re: [sqlite] [patch] avoid dynamic alloc in vdbeSorterSort(...)

2019-10-07 Thread Dominique Pellé
; What about sqlite3StackAllocZero and SQLITE_USE_ALLOCA? > > pon., 7 paź 2019 o 20:51 Dominique Pellé > napisał(a): > > > Hi > > > > Below is a patch which avoids a dynamic > > allocation in vdbeSorterSort(...), using a local > > stack array instead (fas

[sqlite] [patch] avoid dynamic alloc in vdbeSorterSort(...)

2019-10-07 Thread Dominique Pellé
Hi Below is a patch which avoids a dynamic allocation in vdbeSorterSort(...), using a local stack array instead (faster and smaller code). I assume that a local array of 64 pointers is small enough to be in the stack. Is this worth merging? $ fossil diff src/vdbesort.c Index: src/vdbesort.c

Re: [sqlite] SQLITE and the memory

2019-09-08 Thread Dominique Pellé
Philippe RIO <51...@protonmail.ch> wrote: > Hello, > I have an application composed of 256 databases. Each database occupied 42Mb > for more than 950 000 records into. > > [samedi 7 septembre 2019 13:11:45] : Number of passwords stored : 244 152 > 645 > [samedi 7 septembre 2019 13:19:28] :

[sqlite] demoRead() function in test_demovfs.c not consistent with documentation

2019-06-15 Thread Dominique Pellé
Hi SQLite page https://sqlite.org/c3ref/io_methods.html says at the bottom: === BEGIN QUOTE === If xRead() returns SQLITE_IOERR_SHORT_READ it must also fill in the unread portions of the buffer with zeros. A VFS that fails to zero-fill short reads might seem to work. However, failure to

Re: [sqlite] Should SQLite distinguish between +0.0 and -0.0 on output?

2019-06-12 Thread Dominique Pellé
James K. Lowden wrote: > On Wed, 12 Jun 2019 09:35:13 -0400 > Richard Hipp wrote: > > > Question: Should SQLite be enhanced to show -0.0 as "-0.0"? > > No. > > 1. Prior art. I can't think of a single programming language that > displays -0.0 without jumping through hoops. I disagree. I

Re: [sqlite] Stack allocation upper bound with SQLITE_USE_ALLOCA

2019-04-03 Thread Dominique Pellé
Jonathan Brandmeyer wrote: > What is the upper bound for stack consumption under the > SQLITE_USE_ALLOCA compile-time option? I see that there are a number > of configurable size limits available as compile-time and/or run-time > options. Which ones affect the maximum alloca? > > Thanks, >

Re: [sqlite] FTS5 Transaction Leads to OOB Read

2019-03-18 Thread Dominique Pellé
Chu wrote: > The code: > > ``` > CREATE VIRTUAL TABLE t1 USING fts5(content); > > BEGIN; > INSERT INTO t1 (content) VALUES(''); > SELECT * FROM > t1('*'); > END; > ``` > > As you

Re: [sqlite] Was there an announcement of 3.27?

2019-02-08 Thread Dominique Pellé
David Raymond wrote: > SQLite version 3.27.1 is now available on the SQLite website: > > https://sqlite.org/ > https://sqlite.org/download.html > https://sqlite.org/releaselog/3_27_1.html Release notes https://sqlite.org/releaselog/3_27_1.html say: === BEGIN QUOTE === Added the

[sqlite] Missing SQLITE_OMIT_* in https://www.sqlite.org/compile.html

2018-10-15 Thread Dominique Pellé
Hi I noticed that the recently added options SQLITE_OMIT_WINDOWFUNC and SQLITE_OMIT_UPSERT are not described at: https://www.sqlite.org/compile.html. I suppose that they should be described in that page. Perhaps there are other SQLITE_OMIT_* options missing in the page. I did not check all

Re: [sqlite] Bug Report: corrupted double-linked list

2018-10-10 Thread Dominique Pellé
Daniel Espinosa wrote: > Thanks for the response. > > Until now I just used GDB, cause is easy on Meson. I will try with > Valgrind, but I will be slow until I find a way to reproduce the problem. > > Just take the desition to report this bug, because just the issue trace the > fault deeply in

Re: [sqlite] LIMIT with OFFSET is not working in android platform

2018-05-21 Thread Dominique Pellé
On Mon, May 21, 2018 at 6:04 PM Richard Hipp wrote: > > LOCAL_CFLAGS += -O0 > > > > LOCAL_CPPFLAGS += -O0 > Looks like you have found a bug in clang. I recommend updating to the > latest version of the compiler that you can get your hands on, and see > if that doesn't fix the

Re: [sqlite] Lots of enhancements coming to version 3.24.0 - please test

2018-05-09 Thread Dominique Pellé
Richard Hipp wrote: > There are a lot of important changes in the SQLite code base since the > previous release. See > https://www.sqlite.org/draft/releaselog/current.html for the latest > summary. Some corrections in the draft doc: In

[sqlite] SQLite performance graph not updated + 3 typos

2018-04-09 Thread Dominique Pellé
Hi The SQLite performance graph at https://www.sqlite.org/cpu.html has not been updated since SQLite-3.21 (currently release being SQLite-3.23). Was it forgotten, or is the graph updated only once in a while? Also here are 3 typos found at https://www.sqlite.org/cpu.html in the same sentence:

Re: [sqlite] printf() problem padding multi-byte UTF-8 code points

2018-02-17 Thread Dominique Pellé
Richard Hipp wrote: > On 2/17/18, Ralf Junker wrote: >> Example SQL: >> >> select >>length(printf ('%4s', 'abc')), >>length(printf ('%4s', 'äöü')), >>length(printf ('%-4s', 'abc')), >>length(printf ('%-4s', 'äöü')) >> >> Output is 4, 3, 4, 3.

Re: [sqlite] custom rank function with FTS3/FTS4: potential crash.

2017-10-06 Thread Dominique Pellé
Dan Kennedy wrote: On 10/06/2017 11:58 PM, Richard Hipp wrote: > >> On 10/6/17, Jens Alfke wrote: >> >>> Thanks for posting this — my adaptation of that sample code in my project >>> had the same bug. >>> >>> I’m not sure how the SQL in your

Re: [sqlite] Seeking advice on improving batch sql update performance on zipvfs

2017-09-08 Thread Dominique Pellé
Yue Wu wrote: > Hello, > As mentioned in the subject, our goal is to improve performance regarding > to batch sql updates. > The update sql as follow, > >> UPDATE ITEM SET FIELD4 =? WHERE DS=? > > We run 100,000 updates in a single transaction. The zipvfs version takes >

Re: [sqlite] Version 3.20.0 coming soon...

2017-07-13 Thread Dominique Pellé
Richard Hipp wrote: > The 3.20.0 release of SQLite is expected in about a week. Please > report any issues that you have with the beta as soon as possible. > > Code is available in the "prerelease snapshot" at > https://sqlite.org/download.html > > An overview of changes is at

Re: [sqlite] Tiny typo in doc.

2017-05-22 Thread Dominique Pellé
Richard Hipp wrote: > On 5/22/17, Donald Griggs wrote: >> http://www.sqlite.org/releaselog/3_19_0.html >> >> "containing two ore more" >> > > Thank you. And since "ore" is a real word, the spelling checker > didn't catch this. :-\ Another typo not caught by the

Re: [sqlite] json_valid() crash on Windows - was: json() number value parsing

2017-04-11 Thread Dominique Pellé
Ralf Junker wrote: > SQLite on Windows crashes when running this test: > > n_structure_10_opening_arrays.json > > The crash results from a stack overflow because json_valid() is implemented > using a recursive parser. All versions of json1.c up to current trunk are >

Re: [sqlite] json() number value parsing

2017-04-10 Thread Dominique Pellé
Dominique Devienne wrote: > On Sun, Apr 9, 2017 at 10:34 AM, Olivier Mascia wrote: > >> > Le 9 avr. 2017 à 03:08, Jens Alfke a écrit : >> > >> >> On Apr 7, 2017, at 5:26 PM, Rolf Ade wrote: >> >> ./sqlite3 >> >>

Re: [sqlite] Compressed schema in memory?

2017-03-17 Thread Dominique Pellé
On Thu, Mar 16, 2017 at 11:19 PM, Bob Friesenhahn wrote: > On Thu, 16 Mar 2017, Richard Hipp wrote: >> >> >> Your 664K is a conservative estimate. On my (64-bit linux) desktop, >> I'm showing 1.58MB of heap space used to store the schema. (Hint: >> bring up the

Re: [sqlite] Why is this so much more efficient?

2017-02-16 Thread Dominique Pellé
Cecil Westerhof wrote: > I have a table vmstat that I use to store vmstat info. ;-) > At the moment it has more as 661 thousand records. > > In principle the values of usertime, systemtime, idletime, waittime and > stolentime should add up to 100. I just wanted to check

Re: [sqlite] Smallest reasonable cache size

2017-02-16 Thread Dominique Pellé
Kim Gräsman wrote: > Hi all, > > In my battles with lots of connections competing over precious cache > memory, I've considered giving some non-critical connections zero > cache using `PRAGMA cache_size=0`. > > Is this a reasonable thing to do? If zero is too extreme, what

Re: [sqlite] SQLite 3.16.0 enters testing

2016-12-29 Thread Dominique Pellé
Bob Friesenhahn wrote: > On Thu, 29 Dec 2016, Darko Volaric wrote: > >> What are you basing that theory on? > > > Perf is claimed to provide very good results but they are real results based > on real measurements. Due to this, the measured results are very

Re: [sqlite] SQLite 3.16.0 enters testing

2016-12-29 Thread Dominique Pellé
Richard Hipp wrote: > On 12/29/16, Bob Friesenhahn wrote: >> Is there a way to know how well cachegrind CPU >> cycles map to real-world CPU usage? > > Not that I know of. If you have any suggestions, please speak up. The 'stat' command of Linux

Re: [sqlite] Any performance penalty for SELECTing more columns? (C API)

2016-10-15 Thread Dominique Pellé
Jens Alfke wrote: > In a simple SELECT query of a single table, using the C API, > is there any difference in performance for requesting more or > fewer columns of the table in the result? Or is the performance > penalty only incurred when actually reading the column values?

Re: [sqlite] transaction during reading on read-only database

2016-10-06 Thread Dominique Pellé
Andrii Motsok wrote: > Hi, > > > My understanding is that any reading from database automatically > starts read transaction. The question: why does sqlite need to do it on > read-only database? > > > Regards, > > Andrii You may open a database as read-only, but

Re: [sqlite] Bug in CREATE INDEX

2016-08-07 Thread Dominique Pellé
Kevin O'Gorman wrote: > CREATE INDEX has two problems: > 1) poor default location of temporary storage. > 2) gets wedged on very large indexes. > > I'm using the sqlite that came with Xubuntu 14.04, I think it's version > 3.8.2. SQLite-3.8.2 is old (Dec 2013). It's better to download and compile

Re: [sqlite] Typographical error in sqlite.org/howtocorrupt.html

2016-07-22 Thread Dominique Pellé
Hi I see another typo in the same page: https://www.sqlite.org/howtocorrupt.html === BEGIN QUOTE === 6.2 Failures of mmap() on QNX There exists some subtle problem with mmap() on QNX such that making a second mmap() call against *the a* single file descriptor can cause the memory [...] === END

Re: [sqlite] Fastest way to add many simple rows to a table?

2016-05-26 Thread Dominique Pellé
Richard Hipp wrote: > On 5/26/16, Eric Grange wrote: >> >> I am looking for the fastest way to insert many rows to a simple table. > > (1) Create a prepared statement: "INSERT INTO xyzzy VALUES(?1,?2,...)" > (2) Run "BEGIN" > (3) Loop over each row you want

[sqlite] Use of __builtin_expect in SQLite

2016-02-07 Thread Dominique Pellé
Hi I see that SQLite has many small optimizations being checked-in. Wouldn't it help to use the following macros and use unlikely(...) for error paths: #ifdef __GNUC__ #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else #define likely(x)

[sqlite] Find out how many times does SQLite hit the disk?

2016-01-31 Thread Dominique Pellé
dpb wrote: > Dear SQLite Community, > > On windows, is there a way to find out how many times does my SQLite DB hit > the disk? > > This will help me in deciding if moving to In-memory SQLite will improve my > application performance. I am done with adding indexes to my tables in > SQLite DB. >

[sqlite] why is searching for a range of rowids in an FTS table a slow operation?

2016-01-24 Thread Dominique Pellé
skywind mailing lists wrote: > Hi, > > according to the documentation this is a slow query for FTS tables: > > SELECT * FROM mail WHERE rowid BETWEEN 15 AND 20; > > while > > SELECT * FROM mail WHERE rowid=15; > > is fast. > As far as I know both queries are fast on normal tables. > Where is the

[sqlite] In the case of ZIPVFS

2015-12-23 Thread Dominique Pellé
??? <2004wqg2008 at 163.com> wrote: > HI,all > SQLite retrieves the compressed records is slower than the uncompressed > records about 30%. > How can improve the problem? and anyone any suggustion? > > best regards > wqg What is your SQLite page size? If you access small

[sqlite] about attach database

2015-12-16 Thread Dominique Pellé
Scott Robison wrote: > On Tue, Dec 15, 2015 at 11:19 PM, Dominique Pell? gmail.com >> wrote: > >> ??? <2004wqg2008 at 163.com> wrote: >> >> > >> > After testing the Sqlite3_open and ATTACH DATABASE, >> > I found that the attach database is slower than sqlite3_open. >> >

[sqlite] about attach database

2015-12-16 Thread Dominique Pellé
??? <2004wqg2008 at 163.com> wrote: > > After testing the Sqlite3_open and ATTACH DATABASE, > I found that the attach database is slower than sqlite3_open. > there is attachment after the mail which includ the speed > information ( millisecond ). Your attachment

[sqlite] Database corruption using zipvfs on android 64 bits (x86_64)

2015-11-06 Thread Dominique Pellé
Alexandre Mainville wrote: > Hi, > > I am experiencing database corruption using zipvfs on android 64 > bits (x86_64). > The same program executing on 32 bits runs without problems. > > The program creates a new database on every run and does a series of > inserts and updates (always the same

[sqlite] Documentation change suggestion: formatchng.html

2015-10-30 Thread Dominique Pellé
Richard Hipp wrote: > On 10/29/15, Simon Slavin wrote: > > > > On 29 Oct 2015, at 6:58pm, Richard Hipp wrote: > > > >> That page was really old and mostly obsolete - at least the header > >> text was. I've rewritten it. Please press "Reload" > > > > Good. Will it be updated for everything

[sqlite] version 3.9.0 doc errors

2015-10-09 Thread Dominique Pellé
R.Smith wrote: > More draft doc errors: > https://www.sqlite.org/draft/versionnumbers.html An additional typo near the end of above page: === BEGIN QUOTE === Again, the *destinction* (-> distinction) between "major" and "minor" is subjective. === END QUOTE === Regards Dominique

[sqlite] Unable to find the file 'memleak.txt'

2015-07-17 Thread Dominique Pellé
Richard Hipp wrote: > On 7/17/15, Sairam Gaddam wrote: >> I ran the test files which are present in SQLite for my custom build and I >> got some memory leaks. >> I got a message like, >> >> Writing unfreed memory log to "./memleak.txt" >> >> But I couldn't able to locate that file. >> Can

[sqlite] SQLite version 3.8.11 coming soon...

2015-07-15 Thread Dominique Pellé
On Wed, Jul 15, 2015 at 8:43 PM, Richard Hipp wrote: > On 7/15/15, T?r?k Edwin wrote: >> On 07/15/2015 08:34 PM, T?r?k Edwin wrote: >> >> In fact such detection could be added to fuzzcheck.c too (I didn't know that >> you can detect presence of address sanitizer at

[sqlite] Segfault during FTS index creation from huge data

2015-04-30 Thread Dominique Pellé
Dan Kennedy wrote: > On 04/30/2015 07:41 AM, Scott Robison wrote: >> >> >> I wrote up some test code today that definitely forces a SQLITE_NOMEM >> condition, and there are no leaks (though there are a couple really large >> outstanding allocations to force an error without needing to insert >>

[sqlite] Segfault during FTS index creation from huge data

2015-04-29 Thread Dominique Pellé
Simon Slavin wrote: > On 29 Apr 2015, at 9:39am, Scott Robison wrote: > >> On windows, malloc returns null if the allocation fails. Sqlite detects >> this and returns an error. >> >> On linux, malloc may return a non null yet invalid pointer and only fail >> when the memory is accessed because

[sqlite] Segfault during FTS index creation from huge data

2015-04-03 Thread Dominique Pellé
Artem wrote: ...snip... > SQLite is functioning for about 4 hours, after which Segmentation Fault error > occurs inevitably. ...snip... You did not provide any stack dump, so I don't think that SQLite developers can do anything to help you. Anyway, it's more likely that the bug is in your

[sqlite] Questions about SQLITE_CONFIG_SCRATCH and SQLITE_CONFIG_PAGECACHE

2015-01-29 Thread Dominique Pellé
Hi 1) Question about SQLITE_CONFIG_SCRATCH In SQLite documentation about SQLITE_CONFIG_SCRATCH, I read: === BEGIN QUOTE https://sqlite.org/c3ref/c_config_getmalloc.html === SQLite will never require a scratch buffer that is more than 6 times the database page size. If SQLite needs needs

Re: [sqlite] Logging and timing queries

2014-07-31 Thread Dominique Pellé
Ketil Froyn wrote: > Hi, > > In debugging and inspecting applications using sqlite, I find it would be > very useful to log some or all queries sent to the database, ideally also > with timing information on how long it took to gather and send the > response, so that it can be

Re: [sqlite] Problem with many connections

2014-07-02 Thread Dominique Pellé
Grzegorz Sikorski wrote: > Hi Hick, > > You were right I was not finalizing statements and this prevented close to > complete. When I modified my code to finalize all statements before close it > works fine. This implies that you have not checked your application with

Re: [sqlite] Problem with many connections

2014-07-02 Thread Dominique Pellé
Eduardo Morras wrote: > On Wed, 02 Jul 2014 12:02:27 +0100 > Grzegorz Sikorski wrote: > >> Hi, >> >> I am not sure if my previous email had reached the list, so I just >> repeat it: >> >> I am developing sharding database using SQLite3 for embedded >>

Re: [sqlite] Building multiple indexes in one scan over a table

2014-06-10 Thread Dominique Pellé
Fabian Giesen wrote: > Simon Slavin wrote: ... >> Also, store your time as a numeric value (Unix epoch or Julian day) rather >> than a string. Numbers are faster to handle than strings. INTEGERs are >> faster to handle than REALs. > > > :-) > > The time fields are 64-bit ints. The exact meaning

Re: [sqlite] Documentation typo

2014-04-25 Thread Dominique Pellé
Drago, William wrote: > In section 2 of http://sqlite.org/foreignkeys.html the word > "separately" appears twice in the following sentence: > > Foreign key constraints are disabled by default (for backwards > compatibility), so must be enabled separately for each > database connection

[sqlite] SQLITE_ENABLE_SQLLOG and sqlite3_init_sqllog

2014-01-10 Thread Dominique Pellé
Hi I wanted to use the SQLite error logging documented here... http://www.sqlite.org/errlog.html But I found that the implementation in SQLite is odd: when defining SQLITE_ENABLE_SQLLOG, SQLite calls a function sqlite3_init_sqllog() inside sqlite3_initialize(). That function

Re: [sqlite] SQlite Memory Leakage with 65536 Page Size

2013-10-28 Thread Dominique Pellé
Raheel Gupta wrote: > Sir, is there any way to not allow malloc to hold memory ? I mean shouldnt > free(), be freeing the memory ? free() frees memory in the heap, making it available to other malloc again. But it does not necessarily shrink the amount of memory used by the

Re: [sqlite] SQlite Memory Leakage with 65536 Page Size

2013-10-27 Thread Dominique Pellé
Raheel Gupta wrote: > Hi, > > Yes, I tried Valgrind and it shows no leaks. > But why would this happen with 64K pages ? In 1024 Sqlite is able to > release all the memory. > >>> It might also be that your memory allocator is holding onto freed memory > rather than releasing it back to the OS. > >

Re: [sqlite] Segmentation Fault With Trigger

2013-10-15 Thread Dominique Pellé
techi eth wrote: You should compile your code & sqlite3.c with -g -O0 when sending a stack trace. Most likely, the bug is in your program. Since you're on Linux, try running with valgrind memcheck tool to find bugs (access to free memory, uninitialized memory, etc.) If you compile with

[sqlite] Missing optimization with id BETWEEN xxx AND xxx?

2013-05-15 Thread Dominique Pellé
Hi The SQL script at http://dominique.pelle.free.fr/query-between.sql shows 2 SELECT queries: * Select query #1 takes 20.2 sec (slow!) * Select query #2 takes 0.004 sec (fast!) Yet the 2 queries are equivalent. Here is how to download and run the script:

Re: [sqlite] How to achieve fastest possible write performance for a strange and limited case

2013-03-29 Thread Dominique Pellé
Pavel Ivanov wrote: > On Fri, Mar 29, 2013 at 12:17 PM, Jeff Archer > wrote: >> I have previously made an apparently bad assumption about this so now I >> would like to go back to the beginning of the problem and ask the most >> basic

Re: [sqlite] random, infrequent disk I/O errors

2013-02-10 Thread Dominique Pellé
Greg Janée wrote: > Hello, I'm running a web service that uses SQLite that throws a disk I/O > exception every once in a while, meaning once every few weeks. ...snip... > Any ideas? Unfortunately, the (standard) Python SQLite wrapper I'm using > doesn't provide access to any more information (if

Re: [sqlite] populating a table as quickly as possible

2013-02-04 Thread Dominique Pellé
Paul Sanderson wrote: > I want to populate a large table (millions of rows) as quickly as possible, > > The data set will not be operated on until the table is fully populated and > if the operation fails I will be starting again. the operation is a one off > and the table will not be added to at

Re: [sqlite] populating a table as quickly as possible

2013-02-03 Thread Dominique Pellé
Simon Slavin wrote: > > On 3 Feb 2013, at 6:34pm, Paul Sanderson wrote: > >> I want to populate a large table (millions of rows) as quickly as possible, >> >> The data set will not be operated on until the table is fully populated and >> if

[sqlite] FTS SQLite "optimize" command creates very large BLOBs in *_segment table.

2013-01-31 Thread Dominique Pellé
Hi I have a database using SQLite-3.7.14 with a FTS4 virtual table (Free Text Search). The FTS table contains several millions of small documents. The FTS DB is created on a server (where creating time does not matter) and then used on an embedded device as a read-only database for FTS queries

Re: [sqlite] integrity_check "out of memory"

2013-01-22 Thread Dominique Pellé
Max Vlasov wrote: > Hi, > > I found with the web search that this error (out of memory for PRAGMA > integrity_check) appeared in the wild and there were some fixes related to > it. > > Today I decided to do the check for a base that lives with its own > encrypted vfs and with an sqlite version

Re: [sqlite] How do you load a ".csv" and skip the first line?

2012-09-02 Thread Dominique Pellé
joe.fis...@tanguaylab.com wrote: > Is there some way to load a ".csv" file into a SQLite database table when > the first row (record) of the file contains headers? > > This does not work? > -- > .separator , > .import

Re: [sqlite] VACUUM fails once in a while with status=14 (unable to open database file)

2012-08-21 Thread Dominique Pellé
Richard Hipp <d...@sqlite.org> wrote: > On Tue, Aug 21, 2012 at 5:30 AM, Dominique Pellé > <dominique.pe...@gmail.com>wrote: > >> Hi >> >> Once in a while, I see an error when doing a "VACUUM" operation. >> >> sqlite3_exec3(...) r

[sqlite] VACUUM fails once in a while with status=14 (unable to open database file)

2012-08-21 Thread Dominique Pellé
Hi Once in a while, I see an error when doing a "VACUUM" operation. sqlite3_exec3(...) returns status=14 (unable to open database file). I suppose that it fails to open a temporary database when doing the VACUUM operation, but I don't see why. sqlite3_extended_errcode(db) also returns

Re: [sqlite] How to know what terms were created using FTS

2012-08-17 Thread Dominique Pellé
Mohit Sindhwani wrote: > Hi Ralf, > > > > On 17/8/2012 3:50 PM, Ralf Junker wrote: >> >> On 17.08.2012 09:30, Mohit Sindhwani wrote: >> >>> We're using FTS4 and it works well for many things. One of the things >>> that we'd like to do is to see what terms are being created by

Re: [sqlite] VERY weird rounding error

2012-06-16 Thread Dominique Pellé
Etienne wrote: > Hi Simon, > > Once again, I really do not care of the accuracy. > > I KNOW 0.1 CAN NOT BE STORED EXACTLY IN A REAL VARIABLE. > > I just try to figure out why the SAME CODE (sqlite3 library) with the SAME > INPUT gives DIFFERENT RESULTS (no matter how

Re: [sqlite] (shell dependent) library rounding error

2012-06-15 Thread Dominique Pellé
Etienne wrote: > R:\>sqlite NUL "select 0.1;" > 0.1 > > > JSDB (www.jsdb.org) output: > > js>var db = new SQLite(); > js>db.exec("select 0.1", function(r){writeln(r)}); > 0.1=0.11 > true > >> You are not

Re: [sqlite] NaN in, 0.0 out?

2012-03-27 Thread Dominique Pellé
Francis J. Monari, Esquire wrote: > All, > > How are +infinity and -infinity handled? > > Frank. Don't forget signed zero, signaling and quiet NaN. Here is a short c++ programs to show how +infinity, -infinity zero, -zero, quiet NaN, signaling NaN are internally

Re: [sqlite] Bulk Insert

2011-08-13 Thread Dominique Pellé
Stephan Beal wrote: > On Fri, Aug 12, 2011 at 1:51 PM, Sumit Gupta wrote: > >> I was try to to Read a Binary file in to the Sqlite Database using .NET. >> All >> NET to do bulk upload. I use that Dataset method to upload about 70,000 >> records in to SQL server and it

Re: [sqlite] Best solution to import big DBF file?

2011-06-27 Thread Dominique Pellé
Gilles Ganault wrote: > Hello > > I have a 87MB file in DBF format that I'd like to import into SQLite. > > What is the best tool for this, free or affordable? > > Thank you. Spatialite (extension of SQLite) implements VirtualDbf and VirtualShp. To create a virtual

Re: [sqlite] Big difference in performance between Python and gcc

2011-06-02 Thread Dominique Pellé
Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 06/01/2011 12:25 PM, Dominique Pellé wrote: >> [Various optimisations] > > While those are all valid, they don't address the underlying issue which is > C code taking five times longer than Pyth

Re: [sqlite] Big difference in performance between Python and gcc

2011-06-01 Thread Dominique Pellé
Alessandro Marzocchi wrote: > Hello, >    I made some preliminary tests for an application storing big > chunks of data in a sqlite database. I did firsts tests with python > and they gave me quite impressive results. I then tried to make the > same test using C. I expected to get execution times

Re: [sqlite] usleep bug?

2010-09-27 Thread Dominique Pellé
Dustin Sallings wrote: > On Sep 27, 2010, at 5:43, Simon Slavin wrote: > >> I think this differs for different platforms.  For instance: >> >> http://www.manpagez.com/man/3/usleep/ >> >> makes no mention of that fault. > >        It kind of does indirectly.  It says it's built on nanosleep(), and

[sqlite] [patch] fixed memory leak in the ".dump" command of command line tool sqlite-3.7.0.1

2010-08-07 Thread Dominique Pellé
Hi I noticed a memory leak when using the .dump command of command line tool "sqlite3". I initially saw it when using spatialite but the same bug exists in latest sqlite-3.7.0.1. A block of memory is leaked each time the ".dump" command is used. In the following example, I run .dump 3 times

Re: [sqlite] How to update many rows efficiently?

2010-08-05 Thread Dominique Pellé
Igor Tandetnik wrote: > Dominique Pellé <dominique.pe...@gmail.com> wrote: >> For example, given 2 tables t1 and t2, both with 2 columns as follows... >> >> Table t1: >> >>    ID   name >>    --   >>    1     >>    2    NULL

[sqlite] How to update many rows efficiently?

2010-08-04 Thread Dominique Pellé
Hi I'm new to SQLite and I'm still learning. It's unclear to me how to update many records efficiently in a table. For example, given 2 tables t1 and t2, both with 2 columns as follows... Table t1: ID name -- 1 2NULL 3NULL 4 (~1