Re: [sqlite] more test suite problems on Windows
> When testing, it is my usual practice to compile with > -DSQLITE_NO_SYNC=1 which turns off disk syncing. This > very definitely makes the tests run way faster. I > wonder if the Makefiles you are using are not setting > this option by default on windows but are on linux? On windows this option doesn't do anything, and with FlushFileBuffers() allways executing tests run really slow. -- NIE KUPUJ!!! ...zanim nie porownasz cen >> http://link.interia.pl/f1a5e - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] CREATE TABLE fails if SQL starts with comment
Update to version version 3.3.17 has solved the problem. The issue existed in version 3.3.14 - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] more test suite problems on Windows
[EMAIL PROTECTED] wrote:
When testing, it is my usual practice to compile with
-DSQLITE_NO_SYNC=1 which turns off disk syncing. This
very definitely makes the tests run way faster. I
wonder if the Makefiles you are using are not setting
this option by default on windows but are on linux?
Richard,
I have included the diffs between the two makefiles I'm using on Linux
and Windows. Both makefiles were generated by a simple configure command.
../sqlite/configure --with-tcl=
On linux I have the readline libs installed.
--- F:\Makefile.linuxFri Apr 27 11:16:36 2007
+++ C:\SQLite\SQLiteV3\build\MakefileFri Apr 27 12:09:35 2007
@@ -15,7 +15,7 @@
# The toplevel directory of the source tree. This is the directory
# that contains this "Makefile.in" and the "configure.in" script.
#
-TOP = ../sqlite-3.3.17
+TOP = ../sqlite
# C Compiler and options for use in building executables that
# will run on the platform that is doing the build.
@@ -36,19 +36,19 @@
# Compiler options needed for programs that use the TCL library.
#
-TCC += -I/usr/include/tcl8.4
+TCC += -IC:/Tcl/include
# The library that programs using TCL must link against.
#
-LIBTCL = -L/usr/lib -ltcl8.4 -ldl -lpthread -lieee -lm
+LIBTCL = -LC:/Tcl/lib -ltcl84
# Compiler options needed for programs that use the readline() library.
#
-READLINE_FLAGS = -DHAVE_READLINE=1 -I/usr/include/readline
+READLINE_FLAGS = -DHAVE_READLINE=0
# The library that programs using readline() must link against.
#
-LIBREADLINE = -lreadline -lreadline
+LIBREADLINE =
# Should the database engine be compiled threadsafe
#
@@ -81,8 +81,8 @@
# Filename extensions
#
-BEXE =
-TEXE =
+BEXE = .exe
+TEXE = .exe
# The following variable is "1" if the configure script was able to locate
# the tclConfig.sh file. It is an empty string otherwise. When this
@@ -102,7 +102,7 @@
prefix = /usr/local
exec_prefix = ${prefix}
libdir = ${exec_prefix}/lib
-INSTALL = /usr/bin/install -c
+INSTALL = /bin/install -c
LIBTOOL = ./libtool
ALLOWRELEASE =
@@ -112,7 +112,7 @@
LTINSTALL = $(LIBTOOL) --mode=install $(INSTALL)
# nawk compatible awk.
-NAWK = mawk
+NAWK = gawk
# You should not have to change anything below this line
###
@@ -281,7 +281,7 @@
libtclsqlite3.la:tclsqlite.lo libsqlite3.la
$(LTLINK) -o libtclsqlite3.la tclsqlite.lo \
-$(LIBOBJ) -L/usr/lib -ltclstub8.4 $(LIBPTHREAD) \
+$(LIBOBJ) -LC:/Tcl/lib -ltclstub84 $(LIBPTHREAD) \
-rpath $(libdir)/sqlite \
-version-info "8:6:8"
I can't see anything significantly different between the two. The only
place the SQLITE_NO_SYNC symbol appears is in the rule for building the
testfixture target. It is the same, set to 1, for both Makefiles.
testfixture$(TEXE):$(TOP)/src/tclsqlite.c libsqlite3.la $(TESTSRC)
$(LTLINK) -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 \
-DSQLITE_NO_SYNC=1 $(TEMP_STORE) \
-o testfixture $(TESTSRC) $(TOP)/src/tclsqlite.c \
libsqlite3.la $(LIBTCL)
Are you suggesting that -DSQLITE_NO_SYNC=1should be added to TCC so it
affects all compiles?
Dennis Cote
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] CREATE TABLE fails if SQL starts with comment
"Yuriy Martsynovskyy" <[EMAIL PROTECTED]> wrote: > When I execute the SQL code below on a newly created DB file I get an > error 'table Tab already exists', and it creates a table > > -- comment > CREATE TABLE Tab(ID); > > Code below works without error messages: > > CREATE TABLE Tab(ID); > I am unable to reproduce this problem. Both statements work fine when I try them. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] more test suite problems on Windows
Dennis Cote <[EMAIL PROTECTED]> wrote: > > Do you, or anyone else, know why the linux version would not be > committing its changes to disk? > When testing, it is my usual practice to compile with -DSQLITE_NO_SYNC=1 which turns off disk syncing. This very definitely makes the tests run way faster. I wonder if the Makefiles you are using are not setting this option by default on windows but are on linux? -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] more test suite problems on Windows
[EMAIL PROTECTED] wrote: I think that was fixed by check-in [3868]. Richard, It seems you were right again. :-) I'm not sure what caused my build to have a problem, but after building and running the test suite on Linux with no problems, I rebooted into Windows. I did a make clean and make to rebuilt and notices that the version number 3.3.16 was being copied into the sqlite3.h file. The version number is hard coded into the Makefile by configure, so I completely cleared my build directory, re ran configure, and rebuilt the library and reran the test. Everything except the alter-7.1 test now passes. I'm sorry for wasting your time. I did notice something strange that make be related to the recent discussion about SQLite's speed on various flavors of Windows. When I run the test suite in Windows XP (Pro SP2 with all updates) it takes about 30 min to complete with the hard drive light on for almost the entire test. Task Manager shows the testfixture application using between 0 and 10% of the CPU and usually less than 5%. This is about what I would expect for an I/O bound program like this. Thread-specific data deallocated properly 1 errors out of 26993 tests Failures on these tests: alter-7.1 make: *** [test] Error 1 real31m30.063s user0m0.092s sys 0m0.015s When I run the test suite under Linux (Ubuntu 7.04 Feisty Fawn) using a reiserfs file system it only takes about 2 min to complete and the hard drive light is mostly off, blinking on only occasionally. Top show the testfixture running at between 60% to 90% of the CPU with most time spent at about 75% with another process running fairly constantly at about 20% CPU. This looks very much like a CPU bound program. Thread-specific data deallocated properly 0 errors out of 28824 tests Failures on these tests: real1m42.484s user0m43.899s sys 0m17.633s Do you, or anyone else, know why the linux version would not be committing its changes to disk? This was the same program executing on the same CPU. The two OS's are on different hard drives. Windows is on a Maxtor 6E040L0, while Linux is on a Segate ST3802110A. These drives are fairly evenly matched drive6E040L0 ST3802110A size 40 GB 80 GB speed 7200 rpm 7200 rpm access10 ms 11 ms cache 2 MB 2 MB interface ATA133ATA100 Both drives are on the same IDE interface on the CPU. I am going to repeat the test using the same FAT32 partition from both Win XP and Linux to eliminate this difference. How long does the test suite (quick test) take to execute on your linux system? Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] CREATE TABLE fails if SQL starts with comment
When I execute the SQL code below on a newly created DB file I get an error 'table Tab already exists', and it creates a table -- comment CREATE TABLE Tab(ID); Code below works without error messages: CREATE TABLE Tab(ID); - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Don't increment on "INSERT or IGNORE" conflict
Hi! I have been reading through the mailing list and found this post in which someone had the same problem as i had (message 15355 I think, posted by drh), where I don't want the autoincrement column to autoincrement when INSERT or IGNORE is forced to ignore the insert. I tried the replace IGNORE for FAIL solution with the command line tool, but I get an error: "column is not unique" when trying a simple insert or ignore test. So, is there any other solution to this? I have yet to experiment this with SQLite.NET and with transactions, but, correct me if I am wrong, won't FAIL stop further INSERTS of the current transaction? Thank you. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd performance issue under Windows - correction
John Elrick wrote: John Stanton wrote: John Elrick wrote: John Stanton wrote: I would look at the disk controller/disk drive hardware and the software driver to see if they are reporting correctly to the OS. Some of your numbers are too fast for regular disk technology and suggest that there are either hardware defects or intentional shortcuts to always perform some form of lazy write. Even the fastest 15,000 RPM disks will take a minimum of 8 millisecs for a secure disk write, and it is that disk latency which is the limiting factor in secure buffer flushing. One performance timed example from an older machine with our application was 128 INSERTS in 15.069 seconds. That would be 110 milliseconds per write. The faster machine was showing about 2.0 seconds for the same operation, or 15 milliseconds per write. The Ruby script I posted showed 8 seconds to save 1000 rows, or just at the edge of what you show as a minimum possibility, however it showed 115 seconds or around 115 milliseconds per write on the "slower" machine. While I agree that 8 milliseconds is just too darn good to be true, I'm a little hesitant to call 110 milliseconds a reasonable number for a secure disk write. But then again, I could be wrong. I'm no expert in low level hard drive programming, which I why I have asked for ideas here. The 8mS is for the fastest disks available and assuming infinite speed of the rest of the machine. The 100mS number would not be unexpected from say 5,400 RPM disks and an OS like Windows which has many distractions to divert it from its purpose. I should be very suspicious of the integrity of the machine which performed in 8Ms. - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd performance issue under Windows - correction
John Stanton wrote: John Elrick wrote: John Stanton wrote: I would look at the disk controller/disk drive hardware and the software driver to see if they are reporting correctly to the OS. Some of your numbers are too fast for regular disk technology and suggest that there are either hardware defects or intentional shortcuts to always perform some form of lazy write. Even the fastest 15,000 RPM disks will take a minimum of 8 millisecs for a secure disk write, and it is that disk latency which is the limiting factor in secure buffer flushing. One performance timed example from an older machine with our application was 128 INSERTS in 15.069 seconds. That would be 110 milliseconds per write. The faster machine was showing about 2.0 seconds for the same operation, or 15 milliseconds per write. The Ruby script I posted showed 8 seconds to save 1000 rows, or just at the edge of what you show as a minimum possibility, however it showed 115 seconds or around 115 milliseconds per write on the "slower" machine. While I agree that 8 milliseconds is just too darn good to be true, I'm a little hesitant to call 110 milliseconds a reasonable number for a secure disk write. But then again, I could be wrong. I'm no expert in low level hard drive programming, which I why I have asked for ideas here. John - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd performance issue under Windows - correction
John Elrick wrote: John Stanton wrote: The real time with the pragma off is 1.78 seconds. The real time on the "faster" machine is 8.4 seconds. When I set the synchronous pragma to off on the "faster" machine, the time drops to 1.64. John Do your various machines use the same hard disk controller and similar technology disks? No. Some of the machines are actually in the hands of other team members and the client. My local machines range from my son's EMachines desktop to a brand new HP laptop running Vista. So we have eight machines with widely varying hardware configurations, yet five of them are within a second or so of being 10 times slower than the average of other three for this particular operation. John I would look at the disk controller/disk drive hardware and the software driver to see if they are reporting correctly to the OS. Some of your numbers are too fast for regular disk technology and suggest that there are either hardware defects or intentional shortcuts to always perform some form of lazy write. Even the fastest 15,000 RPM disks will take a minimum of 8 millisecs for a secure disk write, and it is that disk latency which is the limiting factor in secure buffer flushing. - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd performance issue under Windows - correction
John Stanton wrote: The real time with the pragma off is 1.78 seconds. The real time on the "faster" machine is 8.4 seconds. When I set the synchronous pragma to off on the "faster" machine, the time drops to 1.64. John Do your various machines use the same hard disk controller and similar technology disks? No. Some of the machines are actually in the hands of other team members and the client. My local machines range from my son's EMachines desktop to a brand new HP laptop running Vista. So we have eight machines with widely varying hardware configurations, yet five of them are within a second or so of being 10 times slower than the average of other three for this particular operation. John - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd performance issue under Windows - correction
John Elrick wrote: John Elrick wrote: John Elrick wrote: Griggs, Donald wrote: John Elrick wrote: "what the heck is happening that is creating a better than order of magnitude difference in execution time on five out of seven Windows machines?". John, If the database is opened and closed just once for your testing, I think you can find out if buffer flushing is involved by *temporarily* adding Pragma synchronous=OFF; At the beginning of your test. Thanks Donald. The results are now identical. So, it has to do with buffer flushing differences between the machines. Now I have to figure out what the differences are given that the XP laptop is configured as shipped and is showing the performance improvement. Corrected text: "Thanks Donald. The results are now in line." The real time with the pragma off is 1.78 seconds. The real time on the "faster" machine is 8.4 seconds. When I set the synchronous pragma to off on the "faster" machine, the time drops to 1.64. John Do your various machines use the same hard disk controller and similar technology disks? - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] May I ask why the source distribution mechanism was changed starting with 3.3.14?
Please provide the processed source package, as it has been the case for SQLite before. "The Amalgamation" big c file (plus header file) might be an (optional) neat package, but the standard processed source package shall be still available. The http://www.sqlite.org/sqlite-source-3_3_17.zip shall link to the processed source package, and another URI shall link to the amalgamation package. Maybe it's an option for you. Every non unix user will thank you! Best regards, Klemens Friedl - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] May I ask why the source distribution mechanism was changed starting with 3.3.14?
In general, I agree. I miss the zipped set of pre-processed C source. Since you have the Linux-based build system at your disposal, you can get what you're used to having with make target_source on the Linux system. This creates a tsrc directory containing the familiar pre-processed C source for use in your Windows build. For what it's worth, I was able to build the sqlite3.dll on Windows (VC6) from the "amalgam" sqlite3.c file without having to have lemon or any of the other SQLite-specific build components - except that pesky sqlite3.def file. I still have to download the precompiled Win32 DLL to get that. Looking at the generated makefile (from the configure script) on Linux, I see there's a rule for making sqlite3.def. The algorithm (grep ... | sed ...) basically works on Windows, using the sqlite3.obj, but the resulting def is short a few lines compared to the one I downloaded. Last, I tried to build the shell from the compiled amalgam, but it errors-out on the link phase complaining about unresolved externals (sqlite3_iotrace or somesuch). And I don't have time right now to figure it out. I'm not a C programmer, and am unfamiliar with compilers, linkers, compiler options, and all that. I usually just have to do trial and error (lots of error) and eventually I figure it out. -Clark - Original Message From: "Bennett, Patrick" <[EMAIL PROTECTED]> To: sqlite-users@sqlite.org Sent: Friday, April 27, 2007 6:57:38 AM Subject: [sqlite] May I ask why the source distribution mechanism was changed starting with 3.3.14? The last time I downloaded SQLite was version 3.3.12. For that version (and many prior versions), I could download a preprocessed archive containing all of the source code, except parse.h, parse.c and opcode.h(? - this is from memory) were the 'generated' versions. The source for the command-line shell was there as well as all other source code. This was close to ideal for me, as I was able to use it in our (custom) build system and build for Windows and Linux with almost no effort at all. Now, I have two choices: 1) Download a tar of everything and have a version of lemon available (which nothing else here uses), as well awk and other tools which don't fit well within our windows-side compiles, or.. 2) Download a preprocessed archive that contains only two files (sqlite3.c and sqlite3.h) losing the ability to easily track changes (or make local patches / fixes if necessary) as well as no longer having the shell code. I'll have to download both archives and piece together my own build. Hopefully this doesn't come off as too much of a nag, but the way it was before was quite convenient and the new method seems to have taken the 'preprocessed' notion to the extreme. If this is how it's going to be from now on, I'll just have to adjust, but if there wasn't any specific reason for changing, you can count this as a vote for the 'old' way. :) Cheers... Patrick Bennett - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] May I ask why the source distribution mechanism was changed starting with 3.3.14?
The last time I downloaded SQLite was version 3.3.12. For that version (and many prior versions), I could download a preprocessed archive containing all of the source code, except parse.h, parse.c and opcode.h(? - this is from memory) were the 'generated' versions. The source for the command-line shell was there as well as all other source code. This was close to ideal for me, as I was able to use it in our (custom) build system and build for Windows and Linux with almost no effort at all. Now, I have two choices: 1) Download a tar of everything and have a version of lemon available (which nothing else here uses), as well awk and other tools which don't fit well within our windows-side compiles, or.. 2) Download a preprocessed archive that contains only two files (sqlite3.c and sqlite3.h) losing the ability to easily track changes (or make local patches / fixes if necessary) as well as no longer having the shell code. I'll have to download both archives and piece together my own build. Hopefully this doesn't come off as too much of a nag, but the way it was before was quite convenient and the new method seems to have taken the 'preprocessed' notion to the extreme. If this is how it's going to be from now on, I'll just have to adjust, but if there wasn't any specific reason for changing, you can count this as a vote for the 'old' way. :) Cheers... Patrick Bennett - To unsubscribe, send email to [EMAIL PROTECTED] -
