Re: [sqlite] Version 3.1.0

2005-01-21 Thread Roger Binns
I think all this discussion has missed the original question. Quite simply, is there any way in which the SQLite *source code* could be changed in order to get better performance on 64 bit machines. If there is then following questions should be answered: - Has it already been done (or should

Re: [sqlite] Sqlite with MFC SDI

2005-01-30 Thread Roger Binns
SQLite as an application file format is one of the suggested uses for SQLite at http://www.sqlite.org/whentouse.html. I've done this (though using Tcl/Tk as the programming language, not C++) and it has worked *very* well. I use it as the native format in BitPim. BitPim is written in Python. The

Re: RE(1): [sqlite] SQLite Advocacy

2005-01-31 Thread Roger Binns
Not true at all. In fact, from experience, the Linux OS is much more full of holes than Windows. It appears most hate Microsoft so thier OS gets the most virus and hackers. All I can say is we independently did a test with Linux and Windows we isntalled a default OS and put it on the net wit

Re: [sqlite] Accesing database remotely

2005-01-31 Thread Roger Binns
I've also seen Google search results where "optimistic" or "opportunistic" caching on Windows clients may cause data corruption, in the context of other file-based database libraries. Oplocks by themselves won't cause corruption. The way oplocks work is that the file server tells the client they a

Re: [sqlite] tricky date time problem

2005-02-21 Thread Roger Binns
I have a query which calculates the number of events during an hour by the minute. It needs to work out which minute has the most events and the average events during that hour. So it should return an array of 60 results for an hour where I can use the MAX() feature in php to find the peak mi

Re: [sqlite] sqlite performance variationin linux and windows

2005-02-25 Thread Roger Binns
build a version up to 10-15% faster I saw the same kind of speedup (and sometimes up to 25%) depending on if assertions are enabled. I believe they are enabled by default. Adding -DNDEBUG turns them off. Roger

Re: [sqlite] what is "in-memory SQLite database"?

2005-03-04 Thread Roger Binns
i am writing an application where i need to define couple of huge hash tables. What is your idea of huge and what operating system/CPU are you running on? Roger

Re: [sqlite] Does "PRAGMA full_column_names = 1" work with APSW and 3.1.3?

2005-03-05 Thread Roger Binns
Note your comment: sql="select * from testnn" # testnn is empty Note the error: apsw.ExecutionCompleteError: Can't get description for statements that have completed execution The statement ran to completion. Asking what the columns are afterwards can't be done because it is complete. Am I doing s

Re: [sqlite] thoughts on a web-based front end to sqlite3 db?

2005-03-07 Thread Roger Binns
APSW is not DBAPI compliant but it's close enough Note that I do document how APSW differs from DBAPI. http://www.rogerbinns.com/apsw.html#dbapinotes APSW is very nice. Thanks :-) I've used it and PySQLite and I'd recommend APSW. It's "thinner". Note that it is almost impossible to do a DBAPI c

Re: [sqlite] Best practices for forward conversion of database formats?

2018-08-30 Thread Roger Binns
On 30/08/18 09:51, Randall Smith wrote: > is how to convert existing data from one DB format to another, given some > arbitrary set of changes in the database schema in the interim. I use SQLite's user pragma. It starts at zero. https://sqlite.org/pragma.html#pragma_user_version My code ends u

[sqlite] sqlite3_complete and comments

2018-10-14 Thread Roger Binns
I use sqlite3_complete in my shell in order to determine when a complete statement has been input and can be run. (Otherwise a continuation "sqlite> " prompt is shown.) If the line entered is: -- hello Then the sqlite shell does not issue a continuation and "executes" the text. However sqli

Re: [sqlite] SQLite between forks

2018-03-26 Thread Roger Binns
On 26/03/18 13:30, Simone Mosciatti wrote: > However I fail to see how this can be a problem for purely in-memory > database. When a process forks, only the thread that called fork is kept in the new child process. Also note that semaphores (and locks in general) are left in the same state as at

Re: [sqlite] Size of the SQLite library

2018-05-31 Thread Roger Binns
On 31/05/18 10:15, Richard Hipp wrote: > Size is still important. But having useful features is important too. > I'm continuing to work to find the right balance between these > competing goals. A pattern used in other projects is to have standard downloads, as well as custom ones. With the latt

Re: [sqlite] Size of the SQLite library

2018-06-05 Thread Roger Binns
On 01/06/18 13:46, Warren Young wrote: > Your jQuery example later on doesn’t much apply here, for several reasons: Note that I was showing how the site let you choose whatever features you want, and then gave you a download matching exactly that. > 1. JavaScript is a dynamic language, while C is

Re: [sqlite] Size of the SQLite library

2018-06-06 Thread Roger Binns
On 05/06/18 15:07, Warren Young wrote: > All right, so include [multi-component source control and build process] ... I'm still not sure what point you are trying to make. Yes *you* can do that. Should *every* SQLite user who wants non-default options *have* to go through a similar amount of fri

Re: [sqlite] Size of the SQLite library

2018-06-06 Thread Roger Binns
On 06/06/18 09:24, Bob Friesenhahn wrote: > A local tool which makes it easy to configure sqlite from local files > sounds useful ... It already exists. It is what the SQLite team uses to produce the amalgamations etc, and is part of the SQLite code base. > but depending on a "web site" (baby-bi

[sqlite] Bug: sqlite3_memory_used/highwater truncate to 32 bits

2015-05-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The sqlite3_memory_used and highwater interfaces are defined to return a 64 bit value. They carefully call the 32 bit limited sqlite3_status method and then cast to 64 bit. Instead they should call sqlite3_status64. Reported indirectly via https://g

[sqlite] Issue: Inconsistency of sqlite3_dbstat_register declaration

2015-05-10 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There is a forward declaration of sqlite3_dbstat_register (currently line 130871 in the 3.8.10.1 amalgamation): int sqlite3_dbstat_register(sqlite3*); Later is the actual implementation (currently line 155838 in the 3.8.10.1 amalgamation): SQLIT

[sqlite] Issue: Inconsistency of sqlite3_dbstat_register declaration

2015-05-11 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/11/2015 01:34 AM, Jan Nijtmans wrote: > 2) If sqlite is not compiled with -DSQLITE_ENABLE_DBSTAT_VTAB=1, > or the function is renamed (as it should) it wouldn't even work. That was my scenario. I just -DSQLITE_ENABLE_DBSTAT_VTAB and the code di

[sqlite] Suggestion for .import improvement

2015-05-11 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/11/2015 12:24 PM, tonyp at acm.org wrote: > It?s a real nuisance to have to edit out the headers every time, > especially when the files are very large and they do not load fast > (or at all) into a text editor. Plus, it?s semi-automatic. I sug

[sqlite] xBestIndex() implementation question

2015-05-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/13/2015 08:06 PM, Jilong Kuang wrote: > Sure, that is one option. But I'm just wondering if inside the > xBestIndex() we can get the value field, we can do a much better > job to give the cardinality info. Note that the value could be something

[sqlite] Extending VFS documentation

2015-05-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/24/2015 02:39 AM, Philip Bennefall wrote: > I have looked at the example VFS implementations, but it is hard > to determine which parts of the code that are implementation > details as opposed to being part of the stable public API so to > speak.

[sqlite] [AGAIN] SQLite on network share

2015-11-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/13/2015 10:31 AM, Richard Hipp wrote: > Just be warned that there are many network filesystems that claim > to implement locks correctly, and do most of the time, but > sometimes mess up It is also worth mentioning that SQLite trusts the filesys

[sqlite] [AGAIN] SQLite on network share

2015-11-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/13/2015 10:46 AM, A. Mannini wrote: > 1) is there a list of FS where SQLite works fine? I don't know of any. Network filesystems are very hard to implement (so many corner cases), and there is a lot of complexity if you also want them to be per

[sqlite] [AGAIN] SQLite on network share

2015-11-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/13/2015 11:55 AM, A. Mannini wrote: > About VistaDB it support use on network share look at > http://www.gibraltarsoftware.com/Support/VistaDB/Documentation/WebFram e.html#VistaDB_Introduction_SupportedPlatforms.html > > and confirmed from its

[sqlite] SQLITE_CANTOPEN after days of operation

2015-10-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/11/2015 12:56 PM, Andrew Miles wrote: >>> 5) I ran lsof on the db, only one process (this one) had the >>> file open Have you run lsof on your monitoring process? You may be running out of file descriptors that are accessing other files or netw

[sqlite] SIGSEGV in sqlite3_open

2015-10-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/28/2015 01:48 AM, Lohmann, Niels, Dr. (CQTN) wrote: > #0 0x010385c4 in SignalProcmask_r () from > C:\QNX650\target\qnx6/armle-v7/lib/libc.so.3 Have you tried turning off all compiler optimisations? Some older/embedded system compilers can prod

[sqlite] Question about Style

2015-10-30 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/30/2015 11:08 AM, Ramar Collins wrote: > I'm working on a static site generator and want to use SQLite to > store metadata. I'm using C and a small library to get the > majority of the work done. My question is, do you have any > suggestions

[sqlite] Feature Request: Support for MMAP beyond 2GB

2015-10-30 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 https://sqlite.org/mmap.html SQLite supports memory mapping databases, but only does so for the first 2GB of the file. My databases are around 4 to 20GB, and completely fit within RAM on my 64 bit systems. The 2GB mmap limit means that only a portio

[sqlite] ESC_BAD_ACCESS when using a background thread

2015-09-01 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/31/2015 11:28 PM, Jeff M wrote: > All my bad -- I'm fessing up. Can you tell us how you found the root causes of the problems? It would be nice to know what tools and techniques worked. Roger -BEGIN PGP SIGNATURE- Version: GnuPG v2 i

[sqlite] Potential bug: Database on gvfs mount cannot be committed to

2015-09-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/06/2015 06:16 AM, Markus Weiland wrote: > I've discovered a potential bug in handling of SQLite database > files on gvfs mounted network shares. SQLite doesn't support being stored on the network for several reasons, including that network file

[sqlite] Potential bug: Database on gvfs mount cannot be committed to

2015-09-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/06/2015 10:20 AM, Markus Weiland wrote: > I see. Since this was working under Ubuntu 14.04, I assume this is > a regression with gvfs. I'll check over there. Nope. SQLite can not maintain data integrity when used with *any* network filesystem.

[sqlite] Potential bug: Database on gvfs mount cannot be committed to

2015-09-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/06/2015 11:13 AM, Florian Weimer wrote: > Surely that's not true, and NFS and SMB are fine as long as there > is no concurrent access? And no program crashes, no network glitches, no optimisation in the protocols to deal with latency, nothing el

[sqlite] Recent downloads gone? Bad http returns

2016-04-28 Thread Roger Binns
I can download 3.12.2 as http://sqlite.org/2016/sqlite-autoconf-3120200.tar.gz but 3.12.1 and 3.12.0 are now gone, giving a 404 error. URLs were http://sqlite.org/2016/sqlite-autoconf-3120100.tar.gz and http://sqlite.org/2016/sqlite-autoconf-312.tar.gz which used to work. Is this intentional?

[sqlite] Recent downloads gone? Bad http returns

2016-04-28 Thread Roger Binns
On 28/04/16 12:56, Richard Hipp wrote: > I intentionally removed 3.12.0 and 3.12.1 because they can (under > obscure circumstances) generate incompatible databases. I have no problems with removing the links to those downloads, but removing the downloads themselves seems a bit extreme. Don't ma

[sqlite] Use of __builtin_expect in SQLite

2016-02-07 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/16 00:56, Dominique Pell? wrote: > I'm curious about the outcome on SQLite benchmarks. About a year ago I tried them out on some tight code (non-SQLite) that absolutely had to use less CPU time. I couldn't get them to make any difference out

[sqlite] Use of __builtin_expect in SQLite

2016-02-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/16 22:39, Matthias-Christian Ott wrote: > Amdahl's law is not applicable here and describes a completely > different problem. SQLite does not involve concurrency. Amdahl's law very much applies, and doesn't explicitly only involve concurrency

[sqlite] Why SQLite does not use a web-based forum?

2016-02-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/02/16 17:19, admin at shuling.net wrote: > Why SQLite does not utilize a web-based forum for all users > discuss problems? I think that will be more convenient and can help > more people. Here you go in several different formats: http://dir.g

[sqlite] applyng schema changes to several databases

2016-02-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 15/02/16 01:01, Luca Ferrari wrote: > While I'm pretty sure a simple sheel script that will execute, > file per file, the alter table (within a transaction) will do the > job I'm wondering if there's a better approach or a more automated > one. The

[sqlite] SQLite tuning advice for caching scenario

2016-02-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/02/16 06:37, Dave Baggett wrote: > I'd welcome any suggestions How about two databases? Create an in memory database for the cache. Then whenever it hits a certain size (eg 64MB) or time passed (eg 5 minutes), copy/move data from the memory da

[sqlite] SQLite tuning advice for caching scenario

2016-02-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/02/16 08:20, Dave Baggett wrote: > One issue, though is that I'd have to run two FTS searches to > search -- one on the disk-based database, and one on the > memory-based one You see issues, I see features :-) The memory based cache would conta

[sqlite] applyng schema changes to several databases

2016-02-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/02/16 03:38, Luca Ferrari wrote: > The above syntax> does not look familiar to me, It is made up to be concise and get the point across. > and in the meantime I've wrapped the user_version pragma get/set > into a shell script. You can't do if

[sqlite] Incorrect missing download behaviour on SQLite.org website

2016-01-10 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 If you request a download that doesn't exist, the SQLite website behaves in a very non-standard way. For example the current download should be from a 2016 directory, but if you instead request from a 2015 directory, you instead get HTTP 302 (temporar

[sqlite] Database Corrupt While Disk Full

2016-01-14 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/01/16 00:44, sanhua.zh wrote: > Recently, my monitoring system showed that the error code > SQLITE_FULL and SQLITE_CORRUPT increasing in same trend. Just as another data point, I had SQLite using code in a library that was used across a bunch of

[sqlite] Using sqlite3.exe as a subprocess

2016-01-15 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/01/16 19:53, Matthew Allen wrote: > It seems that sqlite3.exe (console) doesn't work as a subprocess > with pipes. There is a bit of a problem with using apps via pipes. Generally when stdout is a terminal, output will be line buffered (ie you

[sqlite] Using sqlite3.exe as a subprocess

2016-01-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 15/01/16 13:16, Matthew Allen wrote: > Yeah in hindsight it wasn't the best. I was trying to do: > > while still running: p.communicate etc > > Poll is not the right method to call. It especially is the wrong method in your scenario since you wan

[sqlite] Incompatible change in unix vfs xCurrentTime

2016-01-20 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 SQLite 3.10 changed the xCurrentTime (note not -64 version) method for the "unix" VFS from a function to a NULL pointer. This has broken things for me, because I have a VFS that calls back into the "unix" vfs. A SQLite shared library upgrade will now

[sqlite] Incompatible change in unix vfs xCurrentTime

2016-01-22 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 20/01/16 20:11, Roger Binns wrote: > SQLite 3.10 changed the xCurrentTime (note not -64 version) method > for From the documentation it is hard to tell who is at "fault" > here. xCurrentTime is documented as optional now, but wa

[sqlite] Slight problem with sqlite3_compileoption_get

2016-01-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/01/16 11:41, Bart Smissaert wrote: > My question is how could I have foreseen this problem, so how could > I have known that using long is no good here? I am somewhat confused about what you wrote. SQLite provides a C level api. You can ignore

[sqlite] Slight problem with sqlite3_compileoption_get

2016-01-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/01/16 13:14, Bart Smissaert wrote: >> I am somewhat confused about what you wrote. > > This has to do with making a .tlb (type library) to access > sqlite3.dll from a VB6 ActiveX dll. That much was clear. > Sofar I have mapped SQLite int with

[sqlite] Slight problem with sqlite3_compileoption_get

2016-01-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/01/16 17:36, Bart Smissaert wrote: > What is different though about sqlite3_compileoption_get, so that > long in the IDL causes the mentioned problems and int doesn't? Absolutely nothing is different about that API. If something was then one of

[sqlite] Slight problem with sqlite3_compileoption_get

2016-01-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/01/16 02:19, Bart Smissaert wrote: > You didn't mention it but how arguments are called from VB6 is also > very important, that is ByVal or ByRef. I think that unless the > argument is going to be altered I have to pass them always ByVal. Sort o

[sqlite] Slight problem with sqlite3_compileoption_get

2016-01-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/01/16 13:12, Bart Smissaert wrote: >>> I think that unless the argument is going to be altered I have >>> to pass > them always ByVal. That wasn't quite right, eg: int > sqlite3_close(sqlite3*); sqlite3* is not going to be altered, but I > can se

[sqlite] Changing the default page_size in 3.12.0

2016-03-04 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/03/16 07:48, Richard Hipp wrote: > The tip of trunk (3.12.0 alpha) changes the default page size for > new database file from 1024 to 4096 bytes. ... This seems like a > potentially disruptive change, so I want to give you, the user > community,

[sqlite] Reserved column names

2016-03-19 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/03/16 03:30, Paul Sanderson wrote: > I know that keywords can be quoted but I am interested in just > those that can be used unquoted (even if not advisable) Out of curiousity, why? My rule of thumb is to always quote (using square brackets) wh

[sqlite] Reserved column names

2016-03-21 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21/03/16 03:32, Dominique Devienne wrote: > SQLite tries to be compatible with non-standard extensions from > various popular RDBMS', but when a standard alternative exists, it > should be preferred IMHO. --DD That depends on the code and project.

[sqlite] Understanding table and database locking mechanism in shared cache mode and normal mode

2016-03-30 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30/03/16 16:58, Simon Slavin wrote: > In both modes (whether you're using 'shared cache' or not) use > either > > https://www.sqlite.org/c3ref/busy_timeout.html The last time I dealt with shared cache mode, the busy timeout did not apply. You had

[sqlite] SQLite vs GPLed software

2016-05-02 Thread Roger Binns
On 02/05/16 16:42, Richard Hipp wrote: > But some of the discussion did get me > thinking about the extent of GPLed software versus SQLite. Something that needs be made abundantly clear is the GPL is *not* about popularity. The GPL is about freedom (think freedom of speech, not price). Even then

[sqlite] SQLite vs GPLed software

2016-05-02 Thread Roger Binns
On 02/05/16 20:00, Richard Hipp wrote: > I said I won't participate in that debate and I mean it. The relative > merits of GPL vs whatever is *not* the question on the floor. Apologies for not being clear. I am in no way debating the merits of either, nor expressing my opinion on the merits, go

[sqlite] Incremental backup/sync facility?

2016-05-06 Thread Roger Binns
On 06/05/16 05:32, Stephan Buchert wrote: > Having the copies of the file synced becomes increasingly tedious > as their sizes increase. Ideal would be some kind of > incremental backup/sync facility. Out of curiousity, would an approach of using multiple databases and using ATTACH to "unify" them

[sqlite] 2 different SQLite versions inside the same process space

2016-05-10 Thread Roger Binns
On 10/05/16 10:42, Andrey Gorbachev wrote: > I am a bit worried that the initialisation of 2 different versions of SQLite > would interfere with one another. Any advice? There is a way to do it, and I use it for my Python APSW extension as the recommended build instructions. It is especially hel

[sqlite] 2 different SQLite versions inside the same process space

2016-05-11 Thread Roger Binns
On 10/05/16 23:43, Dominique Devienne wrote: > That explains how to avoid symbol mixups, to have two or more SQLite > "instances" > in a single process, but doesn't address the concern about POSIX locks > DRH mentioned. > if more than one of those same-process instances access the same file. --DD

[sqlite] Podcast with Dr Hipp: SQLite history, success and funding

2016-05-16 Thread Roger Binns
On 16/05/16 10:36, Richard Hipp wrote: > I find that when you are thinking long-term, it changes your > perspective on which patches land on trunk. In addition to your (plural) fantastic work, saying yes/no is probably by far the most important piece. There are constant calls for things to be add

Re: [sqlite] WAL: no schema after close

2016-06-03 Thread Roger Binns
On 03/06/16 08:28, Максим Дементьев wrote: > Thank you, I'll try this "yet another python SQLite wrapper" in the near > future, it looks interesting. Disclosure: I am the author of APSW. I recommend looking at the page showing the differences between APSW and pysqlite: https://rogerbinns.gith

Re: [sqlite] WAL: no schema after close

2016-06-05 Thread Roger Binns
On 03/06/16 18:56, Gelin Yan wrote: >APSW is great, I have used it for years. I want to know whether it > support pypy. I have switched to pypy for a while. APSW at its heart is very much a CPython extension and uses that C API to bind to SQLite. I did port APSW to pypy a few years ago which

Re: [sqlite] Bad db feature request

2016-06-29 Thread Roger Binns
On 29/06/16 07:51, Dominique Devienne wrote: > I wish for the day SQLite has page checksums to detect any such random > corruption. Agreed. The SQLite team rejected doing so: http://www.sqlite.org/src/tktview/72b01a982a84f64d4284 > Yes, I know, it's a format change, and will likely slow thing

Re: [sqlite] Bad db feature request

2016-06-29 Thread Roger Binns
On 29/06/16 09:45, Drago, William @ CSG - NARDA-MITEQ wrote: > Aren't there things like that [checksums] already built in to the hard disk > controllers (CRC, Reed Solomon, etc.)? They are at a different level and can only detect issues in what they see. For example SQLite can create a page of d

Re: [sqlite] page checksums (was Re: Bad db feature request)

2016-06-29 Thread Roger Binns
On 29/06/16 19:13, Scott Robison wrote: > Given the nature of VFS, it is trivial* for anyone to create a module to > provide this very functionality. So you can write it yourself! > > *Not really trivial, but probably not horribly difficult either. VFS is one way you can't reasonably do it. The

Re: [sqlite] builtin functions and strings with embedded nul characters

2016-07-01 Thread Roger Binns
On 01/07/16 05:04, Simon Slavin wrote: > On 1 Jul 2016, at 10:18am, Rob Golsteijn wrote: > >> For the tests below I assumed that the intention is that a string ends at >> the first embedded nul character. > > I'm not sure that this is the intent. > > The idea that null is a terminating charact

Re: [sqlite] The upcoming "pi" release of SQLite

2016-07-24 Thread Roger Binns
On 23/07/16 08:16, Richard Hipp wrote: > Draft change log: https://www.sqlite.org/draft/releaselog/3_14_0.html Please please don't make the new trace/profile API expand the SQL by default. There are two problems with expanding by default: - The text no longer matches what the programmer had in

Re: [sqlite] client app crashes frequently and points sqlite DLL

2016-09-02 Thread Roger Binns
On 02/09/16 03:11, Frantz FISCHER wrote: > I'm almost out of tracks to follow. Any idea on what I could check next? Are you using threads? Note that in this case whatever is happening is making SQLite be the victim. The cause is almost certainly some other C code in your process. Roger sign

Re: [sqlite] Database malformed after 6000 Inserts?

2016-10-04 Thread Roger Binns
On 04/10/16 03:11, Werner Kleiner wrote: > ... after 6000 records. > > Is there a limitation with huge inserts? While there may be "Lite" in the name, SQLite copes very well with "huge" stuff. That means many many gigabytes in database sizes, many many millions of rows, up to 2GB per row etc. 6

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Roger Binns
On 17/11/16 19:14, Kevin O'Gorman wrote: > SO: I need help bifurcating this problem. For instance, how can I tell if > the fault lies in SQLite, or in python? Or even in the hardware, given that > the time to failure is so variable? Are you using threads, threading related settings etc in any way

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Roger Binns
On 18/11/16 08:55, Kevin O'Gorman wrote: >> I am not. All of the python code is a single thread. The closest I come > is a few times where I use subprocess.Popen to create what amounts to a > pipeline, and one place where I start a number of copies of a C program in > parallel, but each is a sepa

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Roger Binns
On 18/11/16 15:19, James K. Lowden wrote: > Good catch, Roger. It's a liability, but I slightly disagree with your > characterization. > >> - Running any Python code (destructors can be called which then run in >> the parent and child) > > Yes, if those destructors affect shared resources. Th

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-21 Thread Roger Binns
On 19/11/16 08:08, Kevin O'Gorman wrote: > System with problems: Running Xubuntu Linux 16.04.1, Python 3.5.2. [...] > System without this problem: Running Ubuntu Linux 14.04.5, Python 3.4.3. You are good on Python versions then. My remaining recommendation is to make the process that does SQLite

Re: [sqlite] problem or not problem around 'begin immediate'

2016-11-25 Thread Roger Binns
On 25/11/16 12:02, Richard Hipp wrote: > Rather, I presume > that Python has recently started using the sqlite3_stmt_readonly() > interface in a new way. The bigger picture may be helpful. There is a third party module developed under the name "pysqlite" which has a long and storied history. At

Re: [sqlite] Pragma to flag unknown pragma?

2016-11-25 Thread Roger Binns
On 22/11/16 16:08, Scott Hess wrote: > Something like "PRAGMA pedantic_pragma = on" would be much slicker. SQLite lets you do "natrual" joins too. I'd argue all this falls under a lint mode that helps conscientious developers make sure everything is working correctly under the hood. https://www

Re: [sqlite] Scope of sqlite3_update_hook?

2016-12-04 Thread Roger Binns
On 01/12/16 16:51, Jens Alfke wrote: > If so, then does that include connections in other OS processes? (I'm looking > for a way to detect this.) You can't get a callback when other processes change the database for many reasons. However it is possible to detect if the database has changed: h

Re: [sqlite] Interactive shell tab completion for queries

2016-12-09 Thread Roger Binns
On 09/12/16 06:29, daveparr...@tutanota.com wrote: > I'm writing to ask if it is possible to have table completion for SQL queries > in the interactive shell? The APSW shell (compatible with the SQLite one) has completion and colouring: https://rogerbinns.github.io/apsw/shell.html Disclaimer

Re: [sqlite] Interactive shell tab completion for queries

2016-12-09 Thread Roger Binns
On 09/12/16 14:18, Dominique Devienne wrote: > How did you implement completion in APSW? [Long answer] at bottom. You get a callback from the readline or equivalent library when completion is needed, with it expecting you to provide 0 or more completions. You can get the input line, the current

Re: [sqlite] Interactive shell tab completion for queries

2016-12-09 Thread Roger Binns
On 09/12/16 14:18, Dominique Devienne wrote: > How did you implement completion in APSW? [Long answer] at bottom. You get a callback from the readline or equivalent library when completion is needed, with it expecting you to provide 0 or more completions. You can get the input line, the current

Re: [sqlite] Speaking of Ruby & Sqlite...a weekend oddity for someone

2016-12-09 Thread Roger Binns
On 09/12/16 14:09, Don V Nielsen wrote: > However, > it fails using the sqlite3 gem. The specific exception is "in > 'initialize': near "with": syntax error That will be a SQLite version issue - older SQLite's didn't support with. While you made some effort around versions, whatever is happening

Re: [sqlite] Why this query plan?

2017-01-12 Thread Roger Binns
On 11/01/17 16:49, Richard Hipp wrote: > For years I have threatened to make it a feature of SQLite that it > really does output the rows in some random order if you omit the ORDER > BY clause - specifically to expose the common bug of omitting the > ORDER BY clause when the order matters. And for

Re: [sqlite] [sqlite-dev] Beginning of release testing for version 3.17.0

2017-02-07 Thread Roger Binns
On 07/02/17 08:56, James K. Lowden wrote: > I must be having a bad day. Both Google and cscope fail to turn up any > reference to NULL_TRIM. What are you referring to, and what does it > do? I started at the changelog posted at the beginning of this thread which makes everything clear/linkable

Re: [sqlite] Beginning of release testing for version 3.17.0

2017-02-09 Thread Roger Binns
On 08/02/17 11:41, Cecil Westerhof wrote: > ​OK, glad to help. What should I do? It is nicest if whatever software/tools you already have also has some sort of testing (ideally automated, but a manual checklist works too). Then run the testing with the existing version of SQLite, and repeat with

Re: [sqlite] Linux top command and sqlite

2017-02-21 Thread Roger Binns
On 21/02/17 10:22, Kevin O'Gorman wrote: > Some of my stuff takes a while to run, and I like to keep tabs on it. > Right now, I'm running one of those, and the Linux top command shows > extremely small CPU usage, and a status ("S" column) of "D" which the man > page defines as "uninterruptable slee

Re: [sqlite] Concrete example of corruption

2013-12-01 Thread Roger Binns
On 01/12/13 06:10, L. Wood wrote: > D. Richard Hipp, are there any plans to make this more robust in the > future, so reading/writing a corrupt database (with no -journal file > available) will return an *error* instead of causing further damage? There has been a ticket languishing for many year

Re: [sqlite] "Last Modified" file attribute is not updated

2013-12-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/12/13 10:30, Felipe Farinon wrote: > I'm sorry to repost, but I just want to confirm that there is no > interest in fixing this, so that I can handle this with a workaround in > my application. Note that the effect will be to make SQLite slower

Re: [sqlite] Does not detect invalid column name when subexpression optimized away

2013-12-19 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 19/12/13 15:36, RSmith wrote: > With this query you essentially ask the RDBMS to evaluate and supply > you with the result of (X and 0) - my guess is the optimiser pounces > directly on the fact that (X and 0) will always be 0 no matter what X > is

Re: [sqlite] Is that same between rebinding every parameter and

2013-12-30 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 30/12/13 06:18, Igor Tandetnik wrote: > sqlite3_clear_binding is very rarely needed, in my experience. In fact, > I have not yet encountered a reason to use it. I use it in my Python wrapper (APSW). The reason is because I have an automatic statem

Re: [sqlite] Saw something interesting in the debugger...

2014-01-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/01/14 08:53, Ward Willats wrote: > I found the UI thread and a worker thread, both in the DB, both in the > default busy handler, both taking a 1 second sleep. > > I expected to see a third thread in the DB doing some work while the > other two

Re: [sqlite] fsync on iOS

2014-01-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16/01/14 11:43, Ward Willats wrote: > So it looks like fsync() is taking more than the 5 second timeout I've > set. This is not uncommon on mobile devices using flash based storage. There is a lot of volatility in read and write performance. I do

Re: [sqlite] Solving the Sudoku with SQlite 3.8.3 trunk

2014-02-06 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/02/14 23:15, big stone wrote: > APSW looks indeed great for specialised installations. The intention behind APSW is a Python wrapper for SQLite3. It does everything the SQLite way where applicable. It advances with SQLite meaning new versions

Re: [sqlite] New

2014-02-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/02/14 16:54, Ashleigh wrote: > I'm trying to view files from my iphone backup I'm not sure which > program it is it says sqlite it is a black box like the windows command > If any one knows a better way to read and understand the files I would >

Re: [sqlite] Compiling SQLite on VxWorks 6.3 (DKM)

2014-03-01 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/02/14 06:37, deltuo wrote: > i compile sqlite 3.8.3 to vxworks 6.9, i first compile sqlite in dkm > and get xx.a lib file, and then test it in vip project, but meet disk > i/o error, can you help me ? thank you , my email is del...@126.com h

Re: [sqlite] Why would batched write operations NOT be faster than individual ones

2014-03-03 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/03/14 03:00, Simon Slavin wrote: > What the heck ? Is this a particular implementation of RAID ... The technical term is "write hole" and can occur at many RAID levels: http://www.raid-recovery-guide.com/raid5-write-hole.aspx You can mitiga

Re: [sqlite] SQLITE_MAX_PAGE_COUNT

2014-03-05 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/03/14 10:59, Raheel Gupta wrote: > If you point out to me the changes required I will do it and have it > reviewed. The changes required are to update the test suites (there are several) to hit/cross the current limit, to modify all relevant co

Re: [sqlite] How good is pragma integrity_check

2014-03-21 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 20/03/14 18:06, Simon Slavin wrote: > All useful as far as SQLite itself goes, and better than nothing. > Unfortunately, failing hard disks do weird things in weird orders. And > the interaction between the physical hard disk and the on-board cache

Re: [sqlite] How good is pragma integrity_check

2014-03-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21/03/14 15:24, Simon Slavin wrote: > Checksums stored with the page index lists, SQLite already has the ability to carve out data on each page for other uses. For example the encryption extension uses this. > Nevertheless, the basic SQLite engin

<    5   6   7   8   9   10   11   12   13   14   >