Re: [sqlite] test errors on OSX

[Stephen C. Gilardi]

The fix is to NOT disable twolevel namespaces, but to fix SQLite's 
Makefile.in to not hoark the linking of the testfixture.   Once this 
is fixed, all of the unit tests pass all of the time on Mac OS X.
Thanks very much for diagnosing the problem.
I took a look at the Makefile and I see that "libtool" is used for most 
of the compilation and linking, but for testfixture, some compilation 
is done without it.  I can see that leading to the bad behavior you 
described.

It seems like one solution would be to build the TESTSRC files using 
libtool as well.  I'm gathering that (at least for the shared library 
case) this would cause them to be linked them using the 
"-flat_namespace" option.

I don't currently know enough about Makefiles to know how to build .o 
files from .c files differently for different targets.  We would need, 
for example, two .o files for btree.c depending on whether or not 
SQLITE_TEST is defined.  Perhaps the "sqlite" library and a new 
"sqlitetest" library could have different subdirectories for their 
component object files.

Having run said unit tests approximately 8 bazillion times as a part 
of my day job, I'm 100% confident that this is the case. ;-)
I'd like to be able to run the tests successfully as well.  Would it be 
possible for you to send me a working Makefile.in to use as an example?

I'd be happy to study it until I understand it and then write up the 
changes as a bug report to sqlite.org if you don't want to do that for 
some reason.

I'd appreciate any help.
Thanks,
--Steve
On Apr 1, 2005, at 2:54 PM, [EMAIL PROTECTED] wrote:
On Apr 1, 2005, at 11:10 AM, Stephen C. Gilardi wrote:
For several months now, I've gotten similar errors when I run the 
test on my PowerBook G4.  For me, the set of tests that fail is 
different from test run to test run, but they do seem to be 
concentrated in the "*ioerr" tests for the most part.

I've run the test several times on a G5 PowerMac and I have never 
seen the problem.  I know at least some of the official Mac OS X 
testing for sqlite is done on a G5.  That may explain why this hasn't 
been seen a lot by the development team.

I've enclosed the results of two test runs from today's cvs sources 
checked out into a new empty directory on my PowerBook G4.  A similar 
run on a G5 PowerMac succeeded.  In all cases, this is Mac OS X 
10.3.8 with all current updates.

The intermittent nature of the failure suggests to me that it is some 
kind of timing-related problem.  I'll be happy to work with anyone to 
try to debug this or test possible fixes.
This is actually due to a bug in SQLite's Makefile.in for the 
testfixture target.  In particular, it recompiles most of the library 
and then links in a fashion such that many of the modules -- the .o 
files -- are effectively linked twice.  Because they are linked twice 
and twolevel namespaces are enabled, the modules use of static 
variables becomes very, very confused.   Effectively, you end up with 
two instances of a static variable with the different instances being 
used in different contexts, depending on which codepath is followed.

The fix is to NOT disable twolevel namespaces, but to fix SQLite's 
Makefile.in to not hoark the linking of the testfixture.   Once this 
is fixed, all of the unit tests pass all of the time on Mac OS X.
Having run said unit tests approximately 8 bazillion times as a part 
of my day job, I'm 100% confident that this is the case. ;-)

The good news is that this bug only affects the testfixture, not the 
library itself.

b.bum

Re: [sqlite] test errors on OSX

[Stephen C. Gilardi]

For several months now, I've gotten similar errors when I run the test 
on my PowerBook G4.  For me, the set of tests that fail is different 
from test run to test run, but they do seem to be concentrated in the 
"*ioerr" tests for the most part.

I've run the test several times on a G5 PowerMac and I have never seen 
the problem.  I know at least some of the official Mac OS X testing for 
sqlite is done on a G5.  That may explain why this hasn't been seen a 
lot by the development team.

I've enclosed the results of two test runs from today's cvs sources 
checked out into a new empty directory on my PowerBook G4.  A similar 
run on a G5 PowerMac succeeded.  In all cases, this is Mac OS X 10.3.8 
with all current updates.

The intermittent nature of the failure suggests to me that it is some 
kind of timing-related problem.  I'll be happy to work with anyone to 
try to debug this or test possible fixes.

--Steve

Run 1:

% make test > 045111010405.log 2>& 1
% grep -v Ok 045111010405.log
[... build steps omitted...]
creating sqlite3
./testfixture ../sqlite/test/quick.test
attach-5.2...
Expected: [0 {}]
 Got: [1 {database is locked}]
attach-5.3...
Expected: [1 {trigger r5 cannot reference objects in database orig}]
 Got: [1 {no such trigger: r5}]
attach-5.4...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
attach-5.5...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
attach-5.6...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
attach-5.7...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
attach-5.7...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
attach-5.8...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
attach-5.9...
Expected: [1 {trigger r5 cannot reference objects in database temp}]
 Got: [1 {no such table: main.t5}]
autovacuum-ioerr-5.38.3...
Expected: [1]
 Got: [0]
autovacuum-ioerr2-4.132.3...
Expected: [1]
 Got: [0]
time with cache: 2097973 microseconds per iteration
time without cache: 4028552 microseconds per iteration
11 errors out of 19709 tests
Failures on these tests: attach-5.2 attach-5.3 attach-5.4 attach-5.5 
attach-5.6 attach-5.7 attach-5.7 attach-5.8 attach-5.9 
autovacuum-ioerr-5.38.3 autovacuum-ioerr2-4.132.3
make: *** [test] Error 1


Run 2:

% make test > 594512010405.log 2>& 1
% grep -v Ok 594512010405.log
[... build steps omitted...]
creating sqlite3
./testfixture ../sqlite/test/quick.test
autovacuum-ioerr-1.27.3...
Expected: [1]
 Got: [0]
autovacuum-ioerr2-4.34.3...
Expected: [1]
 Got: [0]
ioerr-2.187.1...
Error: database is locked
/[... pwd ...]/594512010405/bld/.libs/testfixture: database is locked
while executing
"ifcapable vacuum {
  do_ioerr_test ioerr-2 -cksum true -sqlprep {
BEGIN;
CREATE TABLE t1(a, b, c);
INSERT INTO t1 VALUES(1, randstr(50,..."
(file "../sqlite/test/ioerr.test" line 57)
invoked from within
"source $testfile"
("foreach" body line 4)
invoked from within
"foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
  set tail [file tail $testfile]
  if {[lsearch -exact $EXCLUDE $tail]>=0} continue
  so..."
(file "../sqlite/test/quick.test" line 45)
make: *** [test] Error 1
--Steve
On Mar 30, 2005, at 10:08 AM, Jason Jobe wrote:
I don't know if this the right forum for this but I just did a CVS 
update, built and ran the standard "make test".

10 errors out of 19855 tests
Failures on these tests: autovacuum-ioerr-6.1.1 autovacuum-ioerr-6.2.1 
autovacuum-ioerr-6.3.1 autovacuum-ioerr-6.4.1 autovacuum-ioerr-6.5.1 
ioerr-6.1.1 ioerr-6.2.1 ioerr-6.3.1 ioerr-6.4.1 ioerr-6.5.1

Is there anything I can do to help narrow down the problem.
-jason

Re: [sqlite] quick question regarding INTEGER PRIMARY KEY default value

[Stephen C. Gilardi]

It is documented to start at 1:

If a table contains a column of type INTEGER PRIMARY KEY, then that 
column becomes an alias for the ROWID.

[...]
If no ROWID is specified on the insert, an appropriate ROWID is created 
automatically. The usual algorithm is to give the newly created row a 
ROWID that is one larger than the largest ROWID in the table prior to 
the insert. If the table is initially empty, then a ROWID of 1 is used. 
If the largest ROWID is equal to the largest possible integer 
(9223372036854775807 in SQLite version 3.0 and later) then the database 
engine starts picking candidate ROWIDs at random until it finds one 
that is not previously used.

--Steve
On Feb 2, 2005, at 11:25 AM, Andrew Shakinovsky wrote:
Will the default value of an INTEGER PRIMARY KEY column always start at
1?
Example:
CREATE TABLE xyz (id INTEGER PRIMARY KEY);
INSERT INTO xyz VALUES (NULL);
Will the value of the id column always be 1 in this example? I tried
numerous tests, and it seems to always start at 1, I wanted to just 
make
sure this would always be the case before I start relying on it in my
code.

Re: [sqlite] Accesing database remotely

[Stephen C. Gilardi]

You may get it to work, but using an sqlite database file on a network 
file system has been reported to be troublesome.  Here is a quote from 
 in a section entitled "How To 
Corrupt Your Database Files":


SQLite uses POSIX advisory locks to implement locking on Unix. On 
windows it uses the LockFile(), LockFileEx(), and UnlockFile() system 
calls. SQLite assumes that these system calls all work as advertised. 
If that is not the case, then database corruption can result. One 
should note that POSIX advisory locking is known to be buggy or even 
unimplemented on many NFS implementations (including recent versions of 
Mac OS X) and that there are reports of locking problems for network 
filesystems under windows. Your best defense is to not use SQLite for 
files on a network filesystem.


Reliable locking is necessary for sqlite to succeed in allowing more 
than one process to access the database contents without corruption.

There is more discussion of this in the mailing list archives and in 
the sqlite source files.

--Steve
On Jan 31, 2005, at 4:44 PM, Richard Boyd wrote:
Hi,
I’m writing a database that will be accessed remotely over Ethernet. I 
had started to write a protocol that would float on top of TCP/IP but 
after reading some posts here it seems that it would be overkill. Is 
it practical to access the database file remotely and run commands on 
it from the remote PC? Are there going to be problems with sharing 
conflicts with more than one application accessing the database 
simultaneously? I know from reading some Sqlite documentation that 
there is some protection built into Sqlite. Does it work automatically 
or is there something I’d have to do at the operating system level?

FYI: I’m using Linux for the computer where the database is stored and 
windows for the machine that will remotely access the database over 
Ethernet.

Thanks again,
Richard.
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.8.1 - Release Date: 27/01/2005

Re: [sqlite] version 3.0.7 giving unresolved external symbol with _sqlite_free_table, _sqlite_open, _sqlite_close, _sqlite_get_table, _sqlite_exec

[Stephen C. Gilardi]

The prefix for several like-named routines in sqlite 3.0.7 is "sqlite3".
For example: sqlite3_open
Could it be that somehow you've #included "sqlite.h" instead of 
"sqlite3.h" in your sources?

--Steve
On Oct 5, 2004, at 12:45 PM, Steve Frierdich wrote:
I made a static library out of the source code for  Sqlite for version 
3.0.7 and when I used the library in an MFC program I get linker error 
on unresolved external symbol  for the following symbols:
_sqlite_close
_sqlite_open
_sqlite_free_table
_sqlite_get_table
_sqlite_exec
_sqlite_error_string
_sqlite_freemem

I made the same type of library for the source for Sqlite for version 
2.8.15 and used that library in the same MFC program and it worked 
find. Can anyone tell me why or how to solve the linker error problem 
for version 3.0.7?

Thanks
Steve Frierdich
727 455 4668
J

Re: [sqlite] api 3, bind help

[Stephen C. Gilardi]

The argument is a "pointer to function returning void and taking void* 
as its argument" (like "free");

Here's a sample program:
#include 
void func(void (*)(void*));
void freemystuff(void* in);
int main(int argc, char* argv[])
{
  func(freemystuff);
}
void func(void (*callback)(void*) )
{
  callback(0);
}
void freemystuff(void* in)
{
  std::cout << "freemystuff called " << in << std::endl;
}
--Steve
On Jun 21, 2004, at 10:55 AM, Andy Colson wrote:
So, I'm guessing the fifth arg is the eCopy param?  My C isnt very 
strong, so void(*)(void*) is a pointer to a function like this:
void* someFunc(void*), right?

My guess is the params are:
int sqlite3_bind_blob(sqlite3_stmt* statement, int paramNumber, const 
void* fieldValue, int fieldSize, callback); /*where callback can be 
one of the const's SQLITE_STATIC, SQLITE_TRANSIENT, or a pointer to a 
function that returns a void* and takes one void* as an arg. */