[sqlite] Re: errors @ "./testfixture conflict.test"
i've got an apparent RESOLUTION for some/most of the errors, & a subsequent/new QUESTION ... looking at the test code, the problem seems to be that, ::sqlite_opentemp_count 0 is never incremented from init val = 0. that sure looks like temp file opening. related, my current ./configure *is* with, --enable-tempstore=yes ( --enable-tempstore Use an in-ram database for temporary tables (never,no,yes,always) ) changing that to, --enable-tempstore=no and re-config+building, then ./testfixture ../sqlite-3.3.8/test/conflict.test finishes with NO errors. Memory leaked: 0 bytes in 0 allocations Thread-specific data deallocated properly 0 errors out of 127 tests Failures on these tests: checking also further, ./testfixture ../sqlite-3.3.8/test/lock2.test (...) 0 errors out of 12 tests (...) ./testfixture ../sqlite-3.3.8/test/malloc2.test (...) 0 errors out of 18130 tests (...) ./testfixture ../sqlite-3.3.8/test/lock2.test (...) 0 errors out of 12 tests (...) ./testfixture ../sqlite-3.3.8/test/pragma.test (...) 0 errors out of 101 tests (...) which resolves most of the last 0.04% of test failures so, QUESTION ... is this a problem with 'tempstore' setting in just the TESTS, or does this indicate that tempstore should stay OUT of ram in/for the app itself? shouldn't the ::sqlite_opentemp_count increment on opening of RAM-based files? but, ./testfixture ../sqlite-3.3.8/test/utf16.test now reports, (...) 9 errors out of 12034 tests Failures on these tests: utf16-delete-8.1 utf16-delete-8.2 utf16-delete-8.3 utf16-delete-8.4 utf16-delete-8.5 utf16-delete-8.6 utf16-temptable-6.2 utf16-temptable-6.3 utf16-temptable-6.6 (...) which are 'new' FAILS, where previously, the fail'd tests were, utf16-conflict-6.2 utf16-conflict-6.3 utf16-conflict-6.7 utf16-conflict-6.8 utf16-conflict-6.9 utf16-conflict-6.10 utf16-conflict-6.11 utf16-conflict-6.12 utf16-conflict-6.13 utf16-lock2-1.1 utf16-lock2-1.3 utf16-lock2-1.4 utf16-lock2-1.6 utf16-lock2-1.7 utf16-lock2-1.10 i've traded in one set of errors for another with this one :-/ ideas? thanks. - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] 2.8.6 Documentation
Hi there, Is there a url or somewhere I can get the sqlite 2.8.6 syntax documentation? Thanks!
Re: [sqlite] SQLite Corruption - Probably Related to auto-vacuum
--- [EMAIL PROTECTED] wrote: > "Ron Avriel" <[EMAIL PROTECTED]> wrote: > > > > >If you only run a single process at a time does the problem go > > >away. (If it does, that indicates that the problem is in the > > >locking code - an area where Fedora has given us no end of problems > > >in the past - not in the BTree layer.) > > > > I believe the problem is a very subtle locking problem during auto-vacuum. > > In pager.c near line 33 is an "#if 0" which if you change to "#if 1" > will turn on debugging printfs in the pager module. Please do this > and send me the output on a run that fails. Is the patch http://www.sqlite.org/cvstrac/chngview?cn=3547 related to this autovaccum issue? __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] errors @ "./testfixture conflict.test"
whittling down the last few testsuite errors,
% ./testfixture ../sqlite-3.3.8/test/conflict.test
reports,
conflict-1.0... Ok
(...)
conflict-6.1... Ok
conflict-6.2...
Expected: [0 {7 6 9} 1 1]
Got: [0 {7 6 9} 1 0]
conflict-6.3...
Expected: [0 {6 7 3 9} 1 1]
Got: [0 {6 7 3 9} 1 0]
conflict-6.4... Ok
conflict-6.5... Ok
conflict-6.6... Ok
conflict-6.7...
Expected: [0 {6 7 3 9} 1 1]
Got: [0 {6 7 3 9} 1 0]
conflict-6.8...
Expected: [0 {7 6 9} 1 1]
Got: [0 {7 6 9} 1 0]
conflict-6.9...
Expected: [0 {6 7 3 9} 1 1]
Got: [0 {6 7 3 9} 1 0]
conflict-6.10...
Expected: [0 {7 6 9} 1 1]
Got: [0 {7 6 9} 1 0]
conflict-6.11...
Expected: [0 {6 7 3 9} 1 1]
Got: [0 {6 7 3 9} 1 0]
conflict-6.12...
Expected: [0 {6 7 3 9} 1 1]
Got: [0 {6 7 3 9} 1 0]
conflict-6.13...
Expected: [0 {7 6 9} 1 1]
Got: [0 {7 6 9} 1 0]
conflict-6.14... Ok
(...)
conflict-11.6... Ok
Memory leaked: 0 bytes in 0 allocations
Thread-specific data deallocated properly
9 errors out of 127 tests
Failures on these tests: conflict-6.2 conflict-6.3 conflict-6.7
conflict-6.8 conflict-6.9 conflict-6.10 conflict-6.11 conflict-6.12
conflict-6.13
checking in,
conflict.test,
@,
263: do_test conflict-6.0 {
...
287: foreach {i conf1 cmd t0 t1 t2 t3} {
the tests that FAIL, all have config param "t0 == 0", where,
# t0 True if there is an error from $cmd
i.e.,
2 REPLACE UPDATE 0 {7 6 9}1 1
3 IGNORE UPDATE 0 {6 7 3 9} 1 1
7 REPLACE {UPDATE OR IGNORE} 0 {6 7 3 9} 1 1
8 IGNORE {UPDATE OR REPLACE} 0 {7 6 9}1 1
9 FAIL {UPDATE OR IGNORE} 0 {6 7 3 9} 1 1
10 ABORT{UPDATE OR REPLACE} 0 {7 6 9}1 1
11 ROLLBACK {UPDATE OR IGNORE} 0 {6 7 3 9} 1 1
12 {} {UPDATE OR IGNORE} 0 {6 7 3 9} 1 1
13 {} {UPDATE OR REPLACE} 0 {7 6 9}1 1
and differ in
"Expected" & "Got" in "t3"
# t3 Number of temporary files created
i _think_ that's the "what" in this error.
now, the "why" ...
ideas?
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] odd indexing behavior
--- michael cuthbertson <[EMAIL PROTECTED]> wrote: > This query: > select * from tbl where num = 2563351070 and > filetype = 'cpp' and > time1 > '2003-01-01' and time1 < '2006-05-05' > returns 18k rows in 600ms and index for time1 is used. > > If I 'plus' the time1 columns, the exec time drops to 420ms and > no index is used. > If I use 'name' instead of 'filetype' with time1 "plussed", no index is > used. > Also, if I use 'order by name', no index is used. > > To summarize, in this query with indexes on most columns, SQLite will only > use time1 index and is slower doing so. > Could someone please explain to me why this happens? Hitting both the index and the underlying table can often be slower than just doing a table scan if you have to examine every row anyway. I suspect that most rows in your table return true for the condition: time1 > '2003-01-01' and time1 < '2006-05-05', so the index is ineffective in this case. Had your time1 constraint been narrower, then using the time1 index could result in a performance gain. Anyway, you've found an acceptable workaround with +time1. If you think the difference between 420ms and 600ms is significant, perhaps running ANALYZE might help. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
> well, a 0.04% test failure rate ain't bad! > Are you, perchance, running this on a network filesystem of some kind? nope. all on my 'local' box. just starting to look over these ... i had NOT *defined* UTF8 as the encoding of choice ... or, at all, actually. so, iirc, that means defaault to ISO8859, yes? wondering if the utf16* errors require UTF8? just guessing atm ... - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
FWIW, my MacBook has nothing special installed. The "About This Mac" dialog box says "Mac OS X Version 10.4.8. 2GHz Intel Core Duo". I searched through the Makefile that configure generated and there was no sign of a -static flag anywhere. "gcc -v" says version 4.0.1, (Apple Computer, Inc. build 5250). I'm really curious to know why it works and nobody elses does. as am i :-/ couple of things of note -- both you & pkishor (next post) are on IntelMacs. _all_ of my Macs are G4/G5 boxes. if not that, then, i _suspect_ (no evidence yet ...) that the difference may well be in the dev environment. mine is, % gcc -v Using built-in specs. Target: powerpc-apple-darwin8 Configured with: /private/var/tmp/gcc/gcc-5367.obj~1/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib --build=powerpc-apple-darwin8 --host=powerpc-apple-darwin8 --target=powerpc-apple-darwin8 Thread model: posix gcc version 4.0.1 (Apple Computer, Inc. build 5367) % automake --version automake (GNU automake) 1.10 % autoconf --version autoconf (GNU Autoconf) 2.61 % sed --version GNU sed version 4.1.5 % gawk --version GNU Awk 3.1.5 % glibtool --version ltmain.sh (GNU libtool) 1.5.22 (1.1220.2.365 2005/12/18 22:14:06) all of which are newer -- some much! -- thatn Apple's 'stock' versions; in particular, note my glibtool @ v1.5.22. the ltmain.sh _bundled_ with sqlite388 is v1.5.2, iirc. also, my are _your_ dev targets set as, % env | grep -i target MACOSX_DEVELOPMENT_TARGET=10.4 MACOSX_DEPLOYMENT_TARGET=10.4 % ? Call me cranky if you want. leaning in that direction ;-) That is the privilege of people who have seen as much water under the bridge as I have. heh. how 'bout we can compare stacks of keypunch cards at another time ... This is not an important patch and does not close any real bug that I can see. It probably just makes some gee-whiz syntax checker tool stop complaining. Whatever... i'm certainly making no judgement as to relative value. my only goal is to end up with a clean build, and well-behaved test suite, so as to be able to label the result as 'high(er) confidence', and pass all on to staff that actaully _deserve_ to be paid for their efforts! if you're interested in sleuthing a bit -- and determining if there's something that needs to be fixed, correct version dependencies, etc etc -- i'm happy to play guinea-pig. otherwise, i'll simply focus on workarounds for the problem, and the remoaining bug(lets) i'm seeing ... thanks! - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Re: [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
snowcrash+sqlite <[EMAIL PROTECTED]> wrote: > ... > Thread-specific data deallocated properly > 32 errors out of 80314 tests > Failures on these tests: conflict-6.2 conflict-6.3 conflict-6.7 > conflict-6.8 conflict-6.9 conflict-6.10 conflict-6.11 conflict-6.12 > conflict-6.13 lock2-1.1 lock2-1.3 lock2-1.4 lock2-1.6 lock2-1.7 > lock2-1.10 malloc2.1.5 pragma-9.10 utf16-conflict-6.2 > utf16-conflict-6.3 utf16-conflict-6.7 utf16-conflict-6.8 > utf16-conflict-6.9 utf16-conflict-6.10 utf16-conflict-6.11 > utf16-conflict-6.12 utf16-conflict-6.13 utf16-lock2-1.1 > utf16-lock2-1.3 utf16-lock2-1.4 utf16-lock2-1.6 utf16-lock2-1.7 > utf16-lock2-1.10 > make: *** [fulltest] Error 1 > > well, a 0.04% test failure rate ain't bad! > Are you, perchance, running this on a network filesystem of some kind? -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] sqlite performance, locking & threading
"Emerson Clarke" <[EMAIL PROTECTED]> wrote: > > Firstly can i clarify what you mean regarding the same moment. Do you > mean that no two threads can be executing the call, or that no two > threads can be in the middle of stepping through a series of results > using the step function (assuming there is some context behind the > scenes). This first. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
maybe this will help, maybe not... On 1/2/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: snowcrash+sqlite <[EMAIL PROTECTED]> wrote: > > ... > > On my system (which is generic as far as I know) this runs > > to completion without any errors. > > ... > > I'm not sure what the difference is. > > tbh, i have _no_ concrete idea either as to why you are NOT seeing > these issues on your MacBook :-/ FWIW, my MacBook has nothing special installed. The "About This Mac" dialog box says "Mac OS X Version 10.4.8. 2GHz Intel Core Duo". I searched through the Makefile that configure generated and there was no sign of a -static flag anywhere. "gcc -v" says version 4.0.1, (Apple Computer, Inc. build 5250). I'm really curious to know why it works and nobody elses does. Lucknow:~/Projects/sqlite-3.3.8 punkish$ gcc -v .. gcc version 4.0.1 (Apple Computer, Inc. build 5363) I don't have any -static flag in my generated Makefile as well, and I have had no problem with any of the tests iirc. Everything fancily works. Macbook Pro Intel Core2 Duo 2.33 GHz. -- Puneet Kishor http://punkish.eidesis.org/ Nelson Inst. for Env. Studies, UW-Madison http://www.nelson.wisc.edu/ Open Source Geospatial Foundation https://edu.osgeo.org/ - collaborate, communicate, compete = - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Re: [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
snowcrash+sqlite <[EMAIL PROTECTED]> wrote:
> > ...
> > On my system (which is generic as far as I know) this runs
> > to completion without any errors.
> > ...
> > I'm not sure what the difference is.
>
> tbh, i have _no_ concrete idea either as to why you are NOT seeing
> these issues on your MacBook :-/
FWIW, my MacBook has nothing special installed. The "About This Mac"
dialog box says "Mac OS X Version 10.4.8. 2GHz Intel Core Duo". I
searched through the Makefile that configure generated and there was
no sign of a -static flag anywhere. "gcc -v" says version 4.0.1,
(Apple Computer, Inc. build 5250). I'm really curious to know why
it works and nobody elses does.
> diff -ru sqlite-3.3.8ORIG/ sqlite-3.3.8
> diff -re sqlite-3.3.8ORIG/Makefile.in sqlite-3.3.8/Makefile.in
> --- sqlite-3.3.8ORIG/Makefile.in 2007-01-02 13:17:56.0 -0800
> +++ sqlite-3.3.8/Makefile.in 2007-01-02 13:39:48.0 -0800
> @@ -271,7 +271,7 @@
> # of the most recently modified source code file
> #
> last_change: $(SRC)
> - cat $(SRC) | grep '$$Id: ' | sort -k 5 | tail -1 \
> + cat $(SRC) | grep '$$Id: ' | sort -k 5 | tail -n 1 \
>| $(NAWK) '{print $$5,$$6}' >last_change
yeah, yeah. I used to be able to say "sort -5" too. I resisted
making the change to "-k 5" for as long as I could. Changes to
the unix interface like this come from the crowd who things that
it is more important to be consistent than correct. In reality,
consistency is the hobgloblin of little minds. I've been happily
typing "tail -1" since before many of the readers of this message
were born, and I will continue to resist the change to "tail -n 1"
for as long as I can. Call me cranky if you want. That is the
privilege of people who have seen as much water under the bridge
as I have.
> diff -ru sqlite-3.3.8ORIG/tool/lemon.c sqlite-3.3.8/tool/lemon.c
> --- sqlite-3.3.8ORIG/tool/lemon.c 2007-01-02 13:17:55.0 -0800
> +++ sqlite-3.3.8/tool/lemon.c 2007-01-02 13:30:36.0 -0800
> @@ -1322,15 +1322,15 @@
>va_start(ap, format);
>/* Prepare a prefix to be prepended to every output line */
>if( lineno>0 ){
> -sprintf(prefix,"%.*s:%d: ",PREFIXLIMIT-10,filename,lineno);
> +snprintf(prefix,sizeof prefix,"%.*s:%d:
> ",PREFIXLIMIT-10,filename,lineno);
Lemon was written before snprintf() was invented. And it is a
developer tool, not really subject to exploitation. Note that none
of this code is actually in the SQLite core. Note also the
careful limiting of the resulting string length using ".*"
before the "s" in "%s". This is not an important patch and does
not close any real bug that I can see. It probably just makes
some gee-whiz syntax checker tool stop complaining. Whatever...
--
D. Richard Hipp <[EMAIL PROTECTED]>
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
[sqlite] Re: [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
now, to check 'make fulltest', with, % chown -R snowcrash /projects/b1 % chown -R snowcrash /projects/sqlite-3.3.8 % sudo -u snowcrash make fulltest and see if this 'holds'. with the mods i've made, make fulltest reports, ... Thread-specific data deallocated properly 32 errors out of 80314 tests Failures on these tests: conflict-6.2 conflict-6.3 conflict-6.7 conflict-6.8 conflict-6.9 conflict-6.10 conflict-6.11 conflict-6.12 conflict-6.13 lock2-1.1 lock2-1.3 lock2-1.4 lock2-1.6 lock2-1.7 lock2-1.10 malloc2.1.5 pragma-9.10 utf16-conflict-6.2 utf16-conflict-6.3 utf16-conflict-6.7 utf16-conflict-6.8 utf16-conflict-6.9 utf16-conflict-6.10 utf16-conflict-6.11 utf16-conflict-6.12 utf16-conflict-6.13 utf16-lock2-1.1 utf16-lock2-1.3 utf16-lock2-1.4 utf16-lock2-1.6 utf16-lock2-1.7 utf16-lock2-1.10 make: *** [fulltest] Error 1 well, a 0.04% test failure rate ain't bad! now, to track down these. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] sqlite performance, locking & threading
Nico, I have implemented all three strategies (thead specific connections, single connection multiple threads, and single thread server with multiple client threads). The problem with using thread specific contexts is that you cant have a single global transaction which wraps all of those contexts. So you end up having to use fine grained transactions, which decreases performance. The single connection multiple thread alternative apparently has problems with sqlite3_step being active on more than one thread at the same moment, so cannot easily be used in a safe way. But it is by far the fastest and simplest alternative. The single thread server solution involves message passing between threads, and even when this is done optimally with condition variables (or events on windows) and blocking ive found that it results in a high number of context switches and decreased performance. It does however make a robust basis for a wrapper api, since it guarantees that things will always be synchronised. But using this arrangement can also result in various static initialisation problems, since the single thread server must always be up and running before anything which needs to use it. Emerson On 1/2/07, Nicolas Williams <[EMAIL PROTECTED]> wrote: On Sat, Dec 30, 2006 at 03:34:01PM +, Emerson Clarke wrote: > Technically sqlite is not thread safe. [...] Solaris man pages describe APIs with requirements like SQLite's as "MT-Safe with exceptions" and the exceptions are listed in the man page. That's still MT-Safe, but the caller has to play by certain rules. Anyways, this is silly. SQLite API is MT-Safe with one exception and that exception is rather ordinary, common to other APIs like it that have a context object of some sort (e.g., the MIT krb5 API), and not really a burden to the caller. In exchange for this exception you get an implementation of the API that is lighter weight and easier to maintain than it would have been without that exception; a good trade-off IMO. Coping with this exception is easy. For example, if you have a server app with multiple worker threads each of which needs a db context then you could use a thread-specific key to track a per-thread db context; use pthread_key_create(3C) to create the key, pthread_setspecific(3C) once per-thread to associate a new db context with the calling thread, and pthread_getspecific(3C) to get the calling thread's db context when you need it. If you have a protocol where you have to step a statement over multiple message exchanges with a client, and you don't want to have per-client threads then get a db context per-client/exchange and store that and a mutext in an object that represents that client/exchange. And so on. Nico -- - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] sqlite performance, locking & threading
On Sat, Dec 30, 2006 at 03:34:01PM +, Emerson Clarke wrote: > Technically sqlite is not thread safe. [...] Solaris man pages describe APIs with requirements like SQLite's as "MT-Safe with exceptions" and the exceptions are listed in the man page. That's still MT-Safe, but the caller has to play by certain rules. Anyways, this is silly. SQLite API is MT-Safe with one exception and that exception is rather ordinary, common to other APIs like it that have a context object of some sort (e.g., the MIT krb5 API), and not really a burden to the caller. In exchange for this exception you get an implementation of the API that is lighter weight and easier to maintain than it would have been without that exception; a good trade-off IMO. Coping with this exception is easy. For example, if you have a server app with multiple worker threads each of which needs a db context then you could use a thread-specific key to track a per-thread db context; use pthread_key_create(3C) to create the key, pthread_setspecific(3C) once per-thread to associate a new db context with the calling thread, and pthread_getspecific(3C) to get the calling thread's db context when you need it. If you have a protocol where you have to step a statement over multiple message exchanges with a client, and you don't want to have per-client threads then get a db context per-client/exchange and store that and a mutext in an object that represents that client/exchange. And so on. Nico -- - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] [RESOLVED/PATCH] re: "Error: no such function: randstr" @ v3.3.8 on OSX
this, apparently, is a real/known/old issue, at least @/by the 'fink'
proj (and, apparently, by ubuntu proj ...)
after working with the folks at IRC(#fink), and porting their 'old
version' patch,
# orig PATCH:
http://fink.cvs.sourceforge.net/fink/dists/10.4/unstable/main/finkinfo/database/sqlite3.patch?revision=1.1=markup
to sqlite v3.3.8, the 'errors' are gone.
(NOTE: from "dmacks" @ #fink
"that -static patch is not really portable upstream ...
The tail syntax change is backward-compatible, but is the only one
recognized by new versions of tail, and the sprintf->snprintf changes
are to avoid some crasher/exploit bugs")
I have my MacBook Pro sitting on my desk
...
I used the very latest code from CVS
...
On my system (which is generic as far as I know) this runs
to completion without any errors.
...
I'm not sure what the difference is.
tbh, i have _no_ concrete idea either as to why you are NOT seeing
these issues on your MacBook :-/
anyway, the details:
% rm -rf /projects/sqlite*
% cd /Users/snowcrash/projects
% rm -rf sqlite*
% wget http://www.sqlite.org/sqlite-3.3.8.tar.gz
% gnutar zxvf sqlite-3.3.8.tar.gz
% vi /projects/patch.sqlite338
-
diff -ru sqlite-3.3.8ORIG/ sqlite-3.3.8
diff -re sqlite-3.3.8ORIG/Makefile.in sqlite-3.3.8/Makefile.in
--- sqlite-3.3.8ORIG/Makefile.in2007-01-02 13:17:56.0 -0800
+++ sqlite-3.3.8/Makefile.in2007-01-02 13:39:48.0 -0800
@@ -271,7 +271,7 @@
# of the most recently modified source code file
#
last_change:$(SRC)
- cat $(SRC) | grep '$$Id: ' | sort -k 5 | tail -1 \
+ cat $(SRC) | grep '$$Id: ' | sort -k 5 | tail -n 1 \
| $(NAWK) '{print $$5,$$6}' >last_change
libsqlite3.la: $(LIBOBJ)
##
## Prevent link time warnings with multiply defined symbols when these
## two binaries linking against libsqlite3.dylib. Instead, force a
## link against libsqlite3.a.
##
@@ -476,7 +476,7 @@
libsqlite3.la $(LIBTCL)
testfixture$(TEXE): $(TOP)/src/tclsqlite.c libsqlite3.la $(TESTSRC)
- $(LTLINK) -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 \
+ $(LTLINK) -static -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 \
-DSQLITE_NO_SYNC=1 $(TEMP_STORE) \
-o testfixture $(TESTSRC) $(TOP)/src/tclsqlite.c \
libsqlite3.la $(LIBTCL)
##
## This patch was taken from Ubuntu's sqlite-2.8.16-1 package.
##
diff -ru sqlite-3.3.8ORIG/tool/lemon.c sqlite-3.3.8/tool/lemon.c
--- sqlite-3.3.8ORIG/tool/lemon.c 2007-01-02 13:17:55.0 -0800
+++ sqlite-3.3.8/tool/lemon.c 2007-01-02 13:30:36.0 -0800
@@ -1322,15 +1322,15 @@
va_start(ap, format);
/* Prepare a prefix to be prepended to every output line */
if( lineno>0 ){
-sprintf(prefix,"%.*s:%d: ",PREFIXLIMIT-10,filename,lineno);
+snprintf(prefix,sizeof prefix,"%.*s:%d: ",PREFIXLIMIT-10,filename,lineno);
}else{
-sprintf(prefix,"%.*s: ",PREFIXLIMIT-10,filename);
+snprintf(prefix,sizeof prefix,"%.*s: ",PREFIXLIMIT-10,filename);
}
prefixsize = strlen(prefix);
availablewidth = LINEWIDTH - prefixsize;
/* Generate the error message */
- vsprintf(errmsg,format,ap);
+ vsnprintf(errmsg,sizeof errmsg,format,ap);
va_end(ap);
errmsgsize = strlen(errmsg);
/* Remove trailing '\n's from the error message. */
@@ -2863,7 +2863,7 @@
while( cfp ){
char buf[20];
if( cfp->dot==cfp->rp->nrhs ){
-sprintf(buf,"(%d)",cfp->rp->index);
+snprintf(buf,sizeof buf,"(%d)",cfp->rp->index);
fprintf(fp,"%5s ",buf);
}else{
fprintf(fp," ");
@@ -2909,7 +2909,7 @@
c = *cp;
*cp = 0;
path = (char *)malloc( strlen(argv0) + strlen(name) + 2 );
-if( path ) sprintf(path,"%s/%s",argv0,name);
+if( path ) snprintf(path,sizeof path,"%s/%s",argv0,name);
*cp = c;
}else{
extern char *getenv();
@@ -2922,7 +2922,7 @@
if( cp==0 ) cp = [strlen(pathlist)];
c = *cp;
*cp = 0;
-sprintf(path,"%s/%s",pathlist,name);
+snprintf(path,sizeof path,"%s/%s",pathlist,name);
*cp = c;
if( c==0 ) pathlist = "";
else pathlist = [1];
@@ -3002,14 +3002,16 @@
cp = strrchr(lemp->filename,'.');
if( cp ){
-sprintf(buf,"%.*s.lt",(int)(cp-lemp->filename),lemp->filename);
+snprintf(buf,sizeof buf,"%.*s.lt",(int)(cp-lemp->filename),lemp->filename);
}else{
-sprintf(buf,"%s.lt",lemp->filename);
+snprintf(buf,sizeof buf,"%s.lt",lemp->filename);
}
if( access(buf,004)==0 ){
tpltname = buf;
}else if( access(templatename,004)==0 ){
tpltname = templatename;
+ }else if( access("/usr/share/lemon/lempar.c",004)==0 ){
+tpltname = "/usr/share/lemon/lempar.c";
}else{
tpltname = pathsearch(lemp->argv0,templatename,0);
}
@@ -3021,7 +3023,7 @@
}
in = fopen(tpltname,"rb");
if( in==0 ){
-fprintf(stderr,"Can't open the template file \"%s\".\n",templatename);
+fprintf(stderr,"Can't open the template file
Re: [sqlite] calculate age
Jay Sprenkle wrote: LOL! You should look at a function to determine if a day is a holiday. Talk about ugly! In some places you literally need to know the weather and the phase of the moon! -- [EMAIL PROTECTED] ~/movie]$ pom The Moon is Waning Gibbous (100% of Full) [EMAIL PROTECTED] ~/movie]$ which pom /usr/games/pom - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] odd indexing behavior
Hello and Happy New year! I have an indexing problem with a very simple query. The correct index is not used or no index is used, according to Explain Query Plan. In the following query, all columns but num have indexes, using default. num is integer; filetype is Varchar; time1 is Datetime; name is varchar. This query: select * from tbl where num = 2563351070 and filetype = 'cpp' and time1 > '2003-01-01' and time1 < '2006-05-05' returns 18k rows in 600ms and index for time1 is used. If I 'plus' the time1 columns, the exec time drops to 420ms and no index is used. If I use 'name' instead of 'filetype' with time1 "plussed", no index is used. Also, if I use 'order by name', no index is used. To summarize, in this query with indexes on most columns, SQLite will only use time1 index and is slower doing so. Could someone please explain to me why this happens? Thanks. Michael - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] v338 build seems ok, but 'make test' complains a lot
> can you tell me what, specifically, does 'just' this, > > cd b1 > ./testfixture ../sqlite/test/autovacuum_ioerr2.test > > report for you? > Thread-specific data deallocated properly 0 errors out of 1761 tests well, for the life o' me, here, (now on 2 boxes @ home, and 2 @ work ...) cd projects ./testfixture ../sqlite/test/autovacuum_ioerr2.test autovacuum-ioerr2-1.1.1... Error: no such function: randstr autovacuum-ioerr2-1.1.2... Ok autovacuum-ioerr2-1.1.3... Expected: [1] Got: [0] autovacuum-ioerr2-2.1.1... Error: no such function: randstr autovacuum-ioerr2-2.1.2... Ok autovacuum-ioerr2-2.1.3... Expected: [1] Got: [0] autovacuum-ioerr2-3.1.1... Ok autovacuum-ioerr2-3.1.2... Ok autovacuum-ioerr2-3.1.3... Ok autovacuum-ioerr2-4.1.1... Error: no such function: randstr autovacuum-ioerr2-4.1.2... Ok autovacuum-ioerr2-4.1.3... Expected: [1] Got: [0] Thread-specific data deallocated properly 6 errors out of 13 tests Failures on these tests: autovacuum-ioerr2-1.1.1 autovacuum-ioerr2-1.1.3 autovacuum-ioerr2-2.1.1 autovacuum-ioerr2-2.1.3 autovacuum-ioerr2-4.1.1 autovacuum-ioerr2-4.1.3 and, cd projects make fulltest starts as, --> http://paste.lisp.org/display/33965/raw and the errors continue from there. and, again, the errors are, for me, exactly the same on each/all of the four boxes. something's different bet our setups ... i'd guess it *is* environmental. the question is, what? - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] v338 build seems ok, but 'make test' complains a lot
snowcrash+sqlite <[EMAIL PROTECTED]> wrote: > > On my system (which is generic as far as I know) this runs > > to completion without any errors. (Actually, it is still > > running as I type this, but no errors so far - certainly not > > the cascade of errors you saw.) > > > > I'm not sure what the difference is. > > hrm. frustrating. > > can you tell me what, specifically, does 'just' this, > > cd b1 > ./testfixture ../sqlite/test/autovacuum_ioerr2.test > > report for you? > Thread-specific data deallocated properly 0 errors out of 1761 tests -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] v338 build seems ok, but 'make test' complains a lot
On my system (which is generic as far as I know) this runs to completion without any errors. (Actually, it is still running as I type this, but no errors so far - certainly not the cascade of errors you saw.) I'm not sure what the difference is. hrm. frustrating. can you tell me what, specifically, does 'just' this, cd b1 ./testfixture ../sqlite/test/autovacuum_ioerr2.test report for you? thanks. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Creating a database from inside a program
Rob Richardson wrote: So, I thought I could just issue the following command from inside my program: Sqlite3 newdatabase.db .read schemafile.txt But, when I issue this command from the DOS prompt, it gives me an error message saying that there is no command named "read". Rob, You can pass your schema command file into sqlite in 3 ways. The first is using the -init option. sqlite3 -init schemafile.txt newdatabase.db The second is using input redirection. sqlite3 newdatabase.db < schemafile.txt The third is supplying a read meta command on the command line. Note, you must quote the command so it gets passed to sqlite as a single string. sqlite3 newdatabase.db ".read schemafile.txt" HTH Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] v338 build seems ok, but 'make test' complains a lot
just fyi, i've now tested this on a different osx box, in order to try to rule out something/anything 'environmental'. this box is, basically, a 'vanilla' osx install. same problem -- same errors. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Cross compile error!
checking readline header files... not specified: still searching... checking readline.h usability... no checking readline.h presence... no checking for readline.h... no checking for /usr/include/readline.h... configure: error: cannot check for file existence when cross compiling not sure on your platform, but on my osx & suse boxes, my readline.h is in /usr/include/readline/readline.h not, /usr/include/readline.h you may wish to specify the dir in CPPFLAGS, or change Makefile's READLINE_FLAGS ... hth - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Cross compile error!
"Francesco Andrisani" <[EMAIL PROTECTED]> wrote: > Hi comunity, > i've a problem when i try to cross compile sqlite-3.3.8 for mipsel > architecture. Suggested approach. 1. Configure for the host. 2. Type "make target_source". This puts a bugs of C source files in a subdirectory named "tsrc". 3. Remove tclsqlite.c and possibly also shell.c from tsrc. 4. Compile all the other C files in tsrc using your cross-compiler. for i in *.c; do xcc -c $i.c; done ar r libsqlite3.a *.o The key step is to do "make target_source" on your host machine. There is a lot of generated code in SQLite. That step will do all the automatic code generation and leave you with a batch of ordinary C source code files, which are much easier to deal with using a cross-compiler. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
[sqlite] Cross compile error!
Hi comunity, i've a problem when i try to cross compile sqlite-3.3.8 for mipsel architecture. I lauch configure file with follow params: [EMAIL PROTECTED] sqlite-3.3.8]$ ./configure --prefix=/home/frank/Router/OpenWRT/whiterussian/openwrt/staging_dir_mipsel --host=mipsel-linux --disable-tcl I've by-passed first error: Version set to 3.3 Release set to 3.3.8 Version number set to 3003008 checking for mipsel-linux-gcc... (cached) mipsel-linux-gcc checking whether we are using the GNU C compiler... (cached) yes checking whether mipsel-linux-gcc accepts -g... (cached) yes checking for mipsel-linux-gcc option to accept ANSI C... (cached) none needed configure: error: unable to find a compiler for building build tools ...changing configure file, but if i re-lauch it show new error: checking readline header files... not specified: still searching... checking readline.h usability... no checking readline.h presence... no checking for readline.h... no checking for /usr/include/readline.h... configure: error: cannot check for file existence when cross compiling Can someone suggest the solution of my problems? Regards
Re: [sqlite] can i dynamically move backward and forward in a record-set ?
My original use of this method was in a program which would literally run for years. An implementation which would checkerboard the heap would ultimately cause problems despite the absence of leaks. Both Unix and Windows support this POSIX feature, and both permit anonymous allocation so it is a good way of dynamically allocating memory in programs portable between Unix and Windows and for implementing shared memory cacheing, buffering or communication in a portable manner. A simple and high performance way of implementing a modest sized single table is to mmap it as a text file and use a fast string search like Boyer-Moore to locate records (rows) in a table search. It can be used to implement tiny footprint flexible data lookups given reasonable size limits. Teg wrote: Hello John, Thanks. I might look into that. The memory manager I use is a block allocator with sub-allocator and coalesces free'd memory as it's returned to the allocator. I don't actually ever "malloc" or "free". I let STL handle all the memory management so, I don't really have to worry about memory leaks. C Tuesday, January 2, 2007, 8:34:45 AM, you wrote: JS> A very simple and efficient method of holding temporary results is to JS> just read it into a temporary file which is memory mapped. The VM JS> manager takes care of the memory buffering so you do not have to be JS> concerned about upper size limits being constrained by memory resources. JS> The file can be anonymous to make the process tidier (subject to a few JS> limitations). JS> Traversing the list is just a matter of reading forwards and back either JS> by using a file handles and the read API or by using a pointer. It have JS> not tried using a pointer to such a structure in a list control, but it JS> may be possible. JS> Avoiding using mallocs for dynamic memory allocation makes for less JS> possibility of checkerboarding in long running processes. JS> Teg wrote: Hello Roger, - To unsubscribe, send email to [EMAIL PROTECTED] -
Re[2]: [sqlite] can i dynamically move backward and forward in a record-set ?
Hello John, Thanks. I might look into that. The memory manager I use is a block allocator with sub-allocator and coalesces free'd memory as it's returned to the allocator. I don't actually ever "malloc" or "free". I let STL handle all the memory management so, I don't really have to worry about memory leaks. C Tuesday, January 2, 2007, 8:34:45 AM, you wrote: JS> A very simple and efficient method of holding temporary results is to JS> just read it into a temporary file which is memory mapped. The VM JS> manager takes care of the memory buffering so you do not have to be JS> concerned about upper size limits being constrained by memory resources. JS> The file can be anonymous to make the process tidier (subject to a few JS> limitations). JS> Traversing the list is just a matter of reading forwards and back either JS> by using a file handles and the read API or by using a pointer. It have JS> not tried using a pointer to such a structure in a list control, but it JS> may be possible. JS> Avoiding using mallocs for dynamic memory allocation makes for less JS> possibility of checkerboarding in long running processes. JS> Teg wrote: >> Hello Roger, >> -- Best regards, Tegmailto:[EMAIL PROTECTED] - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] v338 build seems ok, but 'make test' complains a lot
g'morning. I think the -DSQLITE_TEST=1 option needs to be added only when building testfixture, not when doing a regular build. Doing this should clear up your randstr() problems. joe w. also pointed this out. thanks. but, as per my results in this thread/post, http://article.gmane.org/gmane.comp.db.sqlite.general/24916 i _think_ i'm doing exacly that, but _still_ see the randstr error(s). likely, i'm still missing something. no? thanks! - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] can i dynamically move backward and forward in a record-set ?
A very simple and efficient method of holding temporary results is to just read it into a temporary file which is memory mapped. The VM manager takes care of the memory buffering so you do not have to be concerned about upper size limits being constrained by memory resources. The file can be anonymous to make the process tidier (subject to a few limitations). Traversing the list is just a matter of reading forwards and back either by using a file handles and the read API or by using a pointer. It have not tried using a pointer to such a structure in a list control, but it may be possible. Avoiding using mallocs for dynamic memory allocation makes for less possibility of checkerboarding in long running processes. Teg wrote: Hello Roger, The data's from Usenet newsgroups so, they have to know what's there before they can filter down to what they're looking for. Some users will just download all 60,000 items in one go. Some will want to sub-search within the N files listed and some do other things. You know how users are, you plan for something, say no more than 5000 items at a time and they decide they want to see them all. I don't even question it anymore. I just just code it to do what they want. It's so fast because the DB's just used for storage and to resolve dependency's for download. Nothing can touch in memory performance. I've had XOVER data for as many as 60 million posts in memory at a time. Some of the largest groups in Usenet have 60 million or more posts in them. It's tricky keeping memory usage small and still make resolving the information needed to download reasonably fast. Essentially, worker threads waiting on semaphores and queued jobs handle the resolution behind the scenes. The point of the original reply though was, that loading data from DB into another efficient data structure, hash_sets is what I use, performs better than trying to cursor forwards and backwards through the data. It's not always possible but, if the data sizes permit it, that's what I do. I don't shy away from using 100-200 megs of RAM if that's what the user's asking for. C Monday, January 1, 2007, 12:47:45 PM, you wrote: RB> -BEGIN PGP SIGNED MESSAGE- RB> Hash: SHA1 RB> Teg wrote: I do that fairly often in my program, I've found performance wise it's better to just load them up. Even to use a thread to load them and post partial results to the GUI so, the loading's dynamic. 10-60,000 items can be efficiently displayed, at least in Windows if you use a virtual list control. Much over 60,000 items and I see a touch of stalling on column sorts and the like. I've had as many as 200-250,000 items in a list control and it's somewhat slow though still usable. RB> I can't imagine any user would scroll through 60,000 items! You are far RB> better off listing the first 250 or similar amount and providing a way RB> for the user to refine their query, or really request all items. RB> Roger RB> -BEGIN PGP SIGNATURE- RB> Version: GnuPG v1.4.6 (GNU/Linux) RB> iD8DBQFFmUlBmOOfHg372QQRAucQAKDIuiHkKcQT/2T1HcKix08dOLNy1wCfQJ5y RB> KRy/uIxJ5yI6Fx7d8r//Q+0= RB> =JK/+ RB> -END PGP SIGNATURE- RB> - RB> To unsubscribe, send email to [EMAIL PROTECTED] RB> - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] v338 build seems ok, but 'make test' complains a lot
snowcrash+spamassassin <[EMAIL PROTECTED]> wrote: > i've found that i need libtool to get the .dylibs built. > > currently, as long as i DISABLE, > >TCC += -DSQLITE_TEST=1 > > then make & install complete ok, resulting in, > > but then, > > make fulltest > > is unhappy, complaining abt 'missing' testsuite, or some such. a bit > of fumbling around suggests that ENABLING, > >TCC += -DSQLITE_TEST=1 > > should fix _that_, but, with it enabled, just > > make > > fails... > I think the -DSQLITE_TEST=1 option needs to be added only when building testfixture, not when doing a regular build. Doing this should clear up your randstr() problems. -- D. Richard Hipp <[EMAIL PROTECTED]> - To unsubscribe, send email to [EMAIL PROTECTED] -
