[sqlite] editline/readline.h' file not found
On 11/9/15, jungle Boogie wrote: > On 8 November 2015 at 19:51, jungle Boogie wrote: >> sqlite3 /home/jungle/fossil-repos/sqlite3/src/shell.c >> ./.libs/libsqlite3.a -ledit -lpthread -Wl,-rpath -Wl,/usr/local/lib >> /home/jungle/fossil-repos/sqlite3/src/shell.c:71:11: fatal error: >> 'editline/readline.h' file not found >> # include >> ^ >> 1 error generated. >> *** Error code 1 >> >> Stop. >> make: stopped in /usr/home/jungle/fossil-repos/sqlite3 > > > Can anyone confirm this is a problem on non-gnu/linux systems? AKA, > does it work on your *BSD system? > Works for me on OpenBSD 5.8. No issues at all. -- D. Richard Hipp drh at sqlite.org
[sqlite] Minor errors in the documentation, building sqlite
On 11/9/15, Richard Hipp wrote: > It is traditional, in a permuted index, to include the complete > unedited title of each document in the index, even if that title > begins with a stop-word. You'll notice that documents that have > "SQLite" as an interior word are not listed among the "SQLite" > documents in the permuted index - only those documents that actually > being with the word "SQLite". You'll likewise notice that there are > examples of titles in the permuted index that begin with "The", though > interior "The" words in titles are not indexed. That make sense. Thank you for the explanation. -- Ambrus
[sqlite] editline/readline.h' file not found
On 9 November 2015 at 19:29, Richard Hipp wrote: > Works for me on OpenBSD 5.8. No issues at all. Failing on freebsd with this make file: https://www.sqlite.org/src/artifact/4469ed8b02a9934f Introduced with this check-in: https://www.sqlite.org/src/info/866f0277781dedf0 Commenting out line 57 and will get sqlite3 to build and install. -- --- inum: 883510009027723 sip: jungleboogie at sip2sip.info xmpp: jungle-boogie at jit.si
[sqlite] 64-bit length BLOB
Hello, Could you please confirm that BLOBs created by sqlite3_bind_blob64()/sqlite3_bind_zeroblob64() can still be read with sqlite3_blob_read()/sqlite3_blob_bytes() or sqlite3_column_blob /sqlite3_column_bytes() because their size can never exceed the limit specified here: http://sqlite.org/limits.html#max_length "The current implementation will only support a string or BLOB length up to 231-1 or 2147483647." Thanks.
[sqlite] Random performance issues with SQLite
On 2015/11/09 6:33 PM, Brice Andr? wrote: > OK, but if this scenario was at the origin of my problem, I guess I would > have such a log each time a db query freezes ? > > If yes, then there is probably another problem, as I am sure I got such > freezes with no such logs from sqlite. > > By the way, if the antivirus is at the origin of the problem, which > practical solution may I find to the problem ? I cannot request my client > to deactivate his antivirus. And he is using a very common one (norton). > So, I would probably not be the only one to have this problem ? Actually, you are not the only one. This problem has been discussed here a lot and is very common. Keeping updates in transactions may help, but a lot of the time you have no control over how many times a user requests things from the DB. Having the Antivirus exclude the folder (or DB file at a minimum) usually helps for us. (It's part of our systems' documentation even). All this assumes it /IS/ in fact the anti-virus causing the problem - you need to check that first. Best of luck! Ryan
[sqlite] 64-bit length BLOB
On 9 Nov 2015, at 7:52pm, gwenn wrote: > Could you please confirm that BLOBs created by > sqlite3_bind_blob64()/sqlite3_bind_zeroblob64() can still be read with > sqlite3_blob_read()/sqlite3_blob_bytes() or sqlite3_column_blob > /sqlite3_column_bytes() because their size can never exceed the limit > specified here: Yes, you can mix and match these routines. Make sure you use the right size for your pointers, and are using hardware, OS and programming language which can handle pointers of that size and that amount of memory. Make sure you close your BLOB handles before trying to read/replace your values atomically. A BLOB handle should not, ideally, be held open which waiting for input from users or an external device. But it will still work correctly as long as thing else tries to mess with the field or row before the handle is closed. Simon.
[sqlite] editline/readline.h' file not found
On 8 November 2015 at 19:51, jungle Boogie wrote: > sqlite3 /home/jungle/fossil-repos/sqlite3/src/shell.c > ./.libs/libsqlite3.a -ledit -lpthread -Wl,-rpath -Wl,/usr/local/lib > /home/jungle/fossil-repos/sqlite3/src/shell.c:71:11: fatal error: > 'editline/readline.h' file not found > # include > ^ > 1 error generated. > *** Error code 1 > > Stop. > make: stopped in /usr/home/jungle/fossil-repos/sqlite3 Can anyone confirm this is a problem on non-gnu/linux systems? AKA, does it work on your *BSD system? -- --- inum: 883510009027723 sip: jungleboogie at sip2sip.info xmpp: jungle-boogie at jit.si
[sqlite] Random performance issues with SQLite
OK, but if this scenario was at the origin of my problem, I guess I would have such a log each time a db query freezes ? If yes, then there is probably another problem, as I am sure I got such freezes with no such logs from sqlite. By the way, if the antivirus is at the origin of the problem, which practical solution may I find to the problem ? I cannot request my client to deactivate his antivirus. And he is using a very common one (norton). So, I would probably not be the only one to have this problem ? Is there a way to request SQlite to don't release control of the file between two operations ? As I only open one descriptor, this could maybe help if it was possible ? Maybe I could ask sqlite to take control of the file at the beginning of my program, and request it to release control at the end ? Thanks for your help, Regards, Brice 2015-11-09 17:26 GMT+01:00 Richard Hipp : > So the problem is occurring on a call to GetFileAttributesEx(). > > I'm guessing that this is an adverse interaction with an anti-virus > (AV) program. > > The scenario might be something like this. > > (1) SQLite writes one small part of a complex change to the database file > (2) The AV seizes control and scans the entire database, taking a long > time to do so (because the file is large) and denying SQLite access > while it does. > (3) The AV finally releases control of the file > (4) Goto 1. > > > On 11/9/15, Brice Andr? wrote: > > Dear Richard, > > > > Thanks for your answer. The version of sqlite I am using is the 3.9.1 > > amalgamation. > > > > But, as stated above, I am not sure my freeze problem is linked to the > > strange log observed at my client as this log does not systematically > > appears on database queries that freeze. Those logs sometimes appear on > > queries that do not freeze, and sometimes, queries that freeze do not > have > > those logs. > > > > I checked in the different logs I have collected, and it seems to be > always > > the same line (39428) that causes this strange log. > > > > Regards, > > > > Brice > > > > 2015-11-09 16:36 GMT+01:00 Richard Hipp : > > > >> On 11/9/15, Brice Andr? wrote: > >> > > >> > 08-11-2015 (13h25): Exception: SQLiteDataBase : Error callback -> 27 : > >> > delayed 25ms for lock/sharing conflict at line 39428 > >> > >> The "line 39428" refers to a specific source code line number in the > >> "sqlite3.c" file, which in turn depends on exactly which version of > >> SQLite you are running - information which you failed to share with > >> us. If you will let us know which version of SQLite you are running, > >> we might be able to figure out what is going wrong. > >> -- > >> D. Richard Hipp > >> drh at sqlite.org > >> ___ > >> sqlite-users mailing list > >> sqlite-users at mailinglists.sqlite.org > >> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > >> > > ___ > > sqlite-users mailing list > > sqlite-users at mailinglists.sqlite.org > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > > > > -- > D. Richard Hipp > drh at sqlite.org > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Random performance issues with SQLite
On Mon, Nov 9, 2015 at 4:41 PM, R Smith wrote: > All this assumes it /IS/ in fact the anti-virus causing the problem - you > need to check that first. Just let him run with the AV disabled for a while or ignore the folder and try to replicate the problem. However, I would **uninstall the program completely**, as many AVs like to keep some things running even when they are "disabled". -- Bernardo Sulzbach
[sqlite] Random performance issues with SQLite
Dear Richard, Thanks for your answer. The version of sqlite I am using is the 3.9.1 amalgamation. But, as stated above, I am not sure my freeze problem is linked to the strange log observed at my client as this log does not systematically appears on database queries that freeze. Those logs sometimes appear on queries that do not freeze, and sometimes, queries that freeze do not have those logs. I checked in the different logs I have collected, and it seems to be always the same line (39428) that causes this strange log. Regards, Brice 2015-11-09 16:36 GMT+01:00 Richard Hipp : > On 11/9/15, Brice Andr? wrote: > > > > 08-11-2015 (13h25): Exception: SQLiteDataBase : Error callback -> 27 : > > delayed 25ms for lock/sharing conflict at line 39428 > > The "line 39428" refers to a specific source code line number in the > "sqlite3.c" file, which in turn depends on exactly which version of > SQLite you are running - information which you failed to share with > us. If you will let us know which version of SQLite you are running, > we might be able to figure out what is going wrong. > -- > D. Richard Hipp > drh at sqlite.org > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Random performance issues with SQLite
On 9 Nov 2015, at 3:21pm, Brice Andr? wrote: > So, does someone has an idea on what can cause such performance issues on > some sqlite DB requests ? Since you 1) are getting delays and not errors, and 2) you are using very common OS and hardware, and 3) you are accessing a database from an internal disk, I strongly suspect that any fault is not in SQLite. From experience this sort of problem is caused by either A) an anti-virus package or B) a hard disk which is just starting to fail and will get worst until it fails completely Diagnosis and fixes should relate to trying a different disk or temporarily disabling the anti-virus software, not to anything inside SQLite. Simon.
[sqlite] Random performance issues with SQLite
Dear all, I experience some performance issues with SQLite. I have an application that uses SQLite to store user data, including agenda information. One of my client has a relatively big database (around 1.3Gb). I have indexed the database so that the queries take a reasonable time to execute. In the general case, the complete update process of the more demanding tasks (handling of user action, search in database and screen refresh) takes less than 1 second. But, at very sparse moments, for reasons I cannot identify, one of my clients experiences freezes in the application that can raise up to several minutes ! After those freezes, the application continues to work normally I instrumented my application to find the origin of the freeze. From this instrumentation, I can guarantee that the time is spent in the database request. But what is strange is that executing exactly the same request, with exactly the same parameters on exactly the same database content just after, or just before the problem occurs does not cause any performance issue. I logged all information on what was happening on my client database (I currently only have one client that encounters this strange behaviour), and the only strange thing that I collected from his computer is the following kind log from sqlite : 08-11-2015 (13h25): Exception: SQLiteDataBase : Error callback -> 27 : delayed 25ms for lock/sharing conflict at line 39428 08-11-2015 (13h25): Exception: SQLiteDataBase : Error callback -> 27 : delayed 75ms for lock/sharing conflict at line 39428 27 is the SQLIte error code, and the text located after is the sqlite message. The occurrence of such warnings do not seem to be direcly linked to my problem, as they can appear on database access that do not suffer from performance issues, and some database access with performance issues can appear without those strange logs. But, for me, those logs are strange because : - I never encountered them on other computers (even if it's true that I did not pay attention to those logs before this problem was reported by my client) - my application is supposed to be the only process to access the database, and only one thread of my application is accessing the database. So, does someone has an idea on what can cause such performance issues on some sqlite DB requests ? Thanks in advance, Brice PS: the problem is present at a client computer running a Windows 7 over a relatively old computer (intel core 2 duo) with 4G ram. The version of sqlite is the last one (because I upgraded hoping this would solve the issue), but was also observerd with an older version of sqlite.
[sqlite] Minor errors in the documentation, building sqlite
On 11/8/15, Zsb?n Ambrus wrote: > > 4. The permuted index > "http://www.sqlite.org/draft/sitemap.html#pindex; has several entries > starting with "SQLite". Most of those are superfluous, because the > whole index is about SQLite documentation only. I think you should > remove most of them. It is traditional, in a permuted index, to include the complete unedited title of each document in the index, even if that title begins with a stop-word. You'll notice that documents that have "SQLite" as an interior word are not listed among the "SQLite" documents in the permuted index - only those documents that actually being with the word "SQLite". You'll likewise notice that there are examples of titles in the permuted index that begin with "The", though interior "The" words in titles are not indexed. -- D. Richard Hipp drh at sqlite.org
[sqlite] field name in UDF
Stephan Beal Sent: Saturday, November 07, 2015 3:54 AM > On Fri, Nov 6, 2015 at 6:50 PM, Nelson, Erik - 2 < > erik.l.nelson at bankofamerica.com> wrote: > > > I have a user-defined function something like > > > > void quarter_sqlite3(sqlite3_context *context, int argc, > sqlite3_value > > **argv); > > > > for each sqlite3_value being passed in, it would sometimes be helpful > > to have the associated field (if any) that the value is associated > with. > > > > Is there any way to retrieve that? > > An idea came to mind, but i'm not sure if it's usable for your case > (it's only potentially useful if the queries are static, not entered by > end > users): UDFs are variadic, so you could pass an optional 2nd parameter > with any information as a string to the final parameter, e.g.: > > select quarter(t.a, 't.a'), quarter(4,'four') from t; That's not a bad idea, but in my case it's a user query so it doesn't work so well, as you mention. -- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
[sqlite] A little light reading
On Sun, Nov 8, 2015 at 4:17 PM, Nicolas J?ger wrote: > people with non-latin-1 name knows problem too... even in their own country... > > regards, > Nicolas Yes, but I think this is getting better as there is more high-level programming and better and more tested libraries being used. Still, the problem is not going to disappear anytime soon. -- Bernardo Sulzbach
[sqlite] Random performance issues with SQLite
On 11/9/15, Brice Andr? wrote: > OK, but if this scenario was at the origin of my problem, I guess I would > have such a log each time a db query freezes ? > > If yes, then there is probably another problem, as I am sure I got such > freezes with no such logs from sqlite. > > By the way, if the antivirus is at the origin of the problem, which > practical solution may I find to the problem ? I cannot request my client > to deactivate his antivirus. And he is using a very common one (norton). > So, I would probably not be the only one to have this problem ? > > Is there a way to request SQlite to don't release control of the file > between two operations ? As I only open one descriptor, this could maybe > help if it was possible ? Maybe I could ask sqlite to take control of the > file at the beginning of my program, and request it to release control at > the end ? > If you run your large update inside a transaction, it should run to completion without getting interference from AV software. -- D. Richard Hipp drh at sqlite.org
[sqlite] Random performance issues with SQLite
So the problem is occurring on a call to GetFileAttributesEx(). I'm guessing that this is an adverse interaction with an anti-virus (AV) program. The scenario might be something like this. (1) SQLite writes one small part of a complex change to the database file (2) The AV seizes control and scans the entire database, taking a long time to do so (because the file is large) and denying SQLite access while it does. (3) The AV finally releases control of the file (4) Goto 1. On 11/9/15, Brice Andr? wrote: > Dear Richard, > > Thanks for your answer. The version of sqlite I am using is the 3.9.1 > amalgamation. > > But, as stated above, I am not sure my freeze problem is linked to the > strange log observed at my client as this log does not systematically > appears on database queries that freeze. Those logs sometimes appear on > queries that do not freeze, and sometimes, queries that freeze do not have > those logs. > > I checked in the different logs I have collected, and it seems to be always > the same line (39428) that causes this strange log. > > Regards, > > Brice > > 2015-11-09 16:36 GMT+01:00 Richard Hipp : > >> On 11/9/15, Brice Andr? wrote: >> > >> > 08-11-2015 (13h25): Exception: SQLiteDataBase : Error callback -> 27 : >> > delayed 25ms for lock/sharing conflict at line 39428 >> >> The "line 39428" refers to a specific source code line number in the >> "sqlite3.c" file, which in turn depends on exactly which version of >> SQLite you are running - information which you failed to share with >> us. If you will let us know which version of SQLite you are running, >> we might be able to figure out what is going wrong. >> -- >> D. Richard Hipp >> drh at sqlite.org >> ___ >> sqlite-users mailing list >> sqlite-users at mailinglists.sqlite.org >> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >> > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > -- D. Richard Hipp drh at sqlite.org
[sqlite] Random performance issues with SQLite
On 11/9/15, Brice Andr? wrote: > > 08-11-2015 (13h25): Exception: SQLiteDataBase : Error callback -> 27 : > delayed 25ms for lock/sharing conflict at line 39428 The "line 39428" refers to a specific source code line number in the "sqlite3.c" file, which in turn depends on exactly which version of SQLite you are running - information which you failed to share with us. If you will let us know which version of SQLite you are running, we might be able to figure out what is going wrong. -- D. Richard Hipp drh at sqlite.org
[sqlite] Random performance issues with SQLite
I am not a user of Norton but I expect it has the same features as others. In Windows Defender you can specify files to exclude and that may help. *Jim Dodgen* On Mon, Nov 9, 2015 at 8:44 AM, Richard Hipp wrote: > On 11/9/15, Brice Andr? wrote: > > OK, but if this scenario was at the origin of my problem, I guess I would > > have such a log each time a db query freezes ? > > > > If yes, then there is probably another problem, as I am sure I got such > > freezes with no such logs from sqlite. > > > > By the way, if the antivirus is at the origin of the problem, which > > practical solution may I find to the problem ? I cannot request my client > > to deactivate his antivirus. And he is using a very common one (norton). > > So, I would probably not be the only one to have this problem ? > > > > Is there a way to request SQlite to don't release control of the file > > between two operations ? As I only open one descriptor, this could maybe > > help if it was possible ? Maybe I could ask sqlite to take control of the > > file at the beginning of my program, and request it to release control at > > the end ? > > > > If you run your large update inside a transaction, it should run to > completion without getting interference from AV software. > > -- > D. Richard Hipp > drh at sqlite.org > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
