Re: [sqlite] Version 3.1.0

[sganz]

The one aspect that I find usefull is for accessing a larger memory space.
For example on your typical linux (rh9 32bit) you start to run out of mem
(per process) at about 2gig. I can immediatly take the app an run on a
64bit machine and get at least an additional gig of ram from the
allocators. This is just due to the O/S implementation and some other
stuff the 64 bit o/s does.

Now compile under 64bit native and I can allocate a much (technical term)
Huger chunk of memory. So I would hope that if can use :memory: with
SQLite on a 64bit system (or huge cache) with enough ram to stuff the
entire db in ram.

An example is the MonetDB (IMDB) which on a 32 bit system can't deal with
DB's lager then available ram. On a 64bit system it can go past ram, and
then let the vmm do the swapping (not ideal, but can do db's larger then
ram). They also have options for trying to keep things 32bit sized when
necessary, as the 64bit transition can cost as well, ptrs, etc are now
double in size.

Other folks might have other reasons, but for me it is address space :-)

Sandy

> On Fri, 2005-01-21 at 20:56 -0500, Robert L Cochran wrote:
>> Does this version take advantage of 64-bit cpu's like the AMD Athlon 64?
>
> I don't know.  What do you mean by "take advantage of"?
> What does a 64-bit CPU do that you can't do with a 32-bit CPU?
>
> There are a few limits in SQLite that are based on the number of
> bits in an integer.  For example, you can't have more than 32
> tables in a join.  Except on a 64-bit machine where you can have
> 64 tables in a join.  Is that what you mean by taking advantage
> of a 64-bit CPU?  But note that we could get 32-bit CPUs to
> provide 64 tables in a join just by changing a single line of
> code, namely changing
>
> typedef unsigned int Bitmask;
>
> into
>
> typedef unsigned long long int Bitmask;
>
> So how does that really "take advantage of" the 64-bitness of
> some CPUs?
>
> I hear a lot of excitement about 64-bit CPUs which I really
> do not understand.  Please explain this to me.
>
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
>

Re: [sqlite] Version 3.1.0

[Robert L Cochran]

Yes -- this is what I'm talking about.
Bob
Roger Binns wrote:
I think all this discussion has missed the original question.
Quite simply, is there any way in which the SQLite *source code* could 
be changed in order to get better performance on 64 bit machines. If 
there is then following questions should be answered:

- Has it already been done (or should it be done)
- Can the changes also be used on a 32 bit machine, or is the codebase 
forked

One thing you can do on 64 bit machines is memory map entire files, so 
there is no explicit need for page caches and things like that. This 
is possible due to the larger address space available which would need 
to be larger than the file size. (On 32 bit machines programs 
typically can only memory map files under 2GB in size).

Roger




smime.p7s
Description: S/MIME Cryptographic Signature

Re: [sqlite] Version 3.1.0

[Clay Dowling]

Robert L Cochran wrote:
1. My wife, on her 32 bit Intel laptop, running Microsoft Access, 
finds her system slowed down to a crawl whenever she works with one 
table that has 109,000 rows. This is just the beginning of the huge 
tables she creates. She has a bunch of others.

I moved this table to my Athlon 64 machine and converted it to the 
Linux x86_64 version of MySQL version 4.1.7.  Then I queried the 
database using a PHP script to "select distinct" from 3 different 
columns and stuff the result sets into HTML select tags so that the 
client can pick valid column values for those columns. 3 queries, 
109,000 rows...and they executed so fast the web page seems to come up 
instantly.

I plan to try this same query with SQLite as soon as I have time.
I think that in this case you'll find that the biggest speed advantage 
was in switching from MS-Access to MySQL.  MySQL is very fast, and 
Access is very much not fast.  If you want to really help your wife get 
a speed up, install MySQL on her laptop and give her the MySQL GUI 
client.  Likewise, she could do very well with SQLite. I've been very 
impressed with SQLite compared with other disk-based databases on 
Windows and I86 hardware.  It's also highly likely that you were running 
on faster hardware, as others have mentioned.

Clay Dowling

Re: [sqlite] Version 3.1.0

[Doug Currie]

Friday, January 21, 2005, 5:41:00 PM, Dan wrote:

>> autovacuum-ioerr2.4.0...
>> Error: error copying "test.db" to "backup.db": no such file or
>> directory autovacuum-ioerr2-4.1.1...
>> Error: error copying "backup.db" to "test.db": no such file or
>> directory autovacuum-ioerr2-4.1.2... Ok
>> autovacuum-ioerr2-4.1.3... Ok
>> c:\dev\sqlite3\bld\testfixture.exe: invalid command name "db"
>>while executing
>> "db close"

> I had the same problem. It's caused by a bug in the version of TCL
> that comes with mingw. If you upgrade TCL (I used ActiveTcl 8.4.9)
> it should work.

Thank-you Dan and Dennis... I installed Tcl 8.4.9 from source and now
'make test' yields...

  1 errors out of 18124 tests
  Failures on these tests: bind-4.1

as expected.

e

Re: [sqlite] Version 3.1.0

[Roger Binns]

I think all this discussion has missed the original question.
Quite simply, is there any way in which the SQLite *source code* could 
be changed in order to get better performance on 64 bit machines.  If 
there is then following questions should be answered:

 - Has it already been done (or should it be done)
 - Can the changes also be used on a 32 bit machine, or is the codebase 
forked

One thing you can do on 64 bit machines is memory map entire files, so 
there is no explicit need for page caches and things like that.  This 
is possible due to the larger address space available which would need 
to be larger than the file size.  (On 32 bit machines programs 
typically can only memory map files under 2GB in size).

Roger

Re: [sqlite] Version 3.1.0

[Greg Miller]

Robert L Cochran wrote:
I should have indicated in my earlier post that magazines which do 
side-by-side hardware testing are saying that the AMD Athlon 64 is 
indeed faster than the Pentium 4; for example look in the January issue 
of Maximum PC magazine. they have a followup test supplementing an 
earlier review of Athlon vs. Pentium systems. Their testing results 
speak better than my faltering around with words. Tom's Hardware Guide 
has also compared Socket 939 Athlon systems to the latest that Intel 
has; I can't remember THG's exact judgement but it was overall more 
favorable to the Athlon systems.
Well, that's not an interesting or relevant comparison. A better 
comparison would be the same test run with a 32-bit application and a 
64-bit version of the same application on the same Athlon system.

Comparisons to other chips are irrelevant. That's different hardware.
--
http://www.velocityvector.com/ | http://www.indie-games.com/
http://www.classic-games.com/  | http://glmiller.blogspot.com/
"If my forgeries looked as bad as the CBS documents, it would
have been 'Catch Me In Two Days'"   -- Frank Abagnale, Jr.

Re: [sqlite] Version 3.1.0

[Greg Miller]

Robert L Cochran wrote:
I don't know how to explain this "excitement" myself, except through 
examples that might bore you because I don't know the details of how to 
write a program that takes advantage of a 64 bit cpu. The excitement is 
mainly about speed, I would say. To make compiled software harness that 
speed, it may be that only a compiler flag has to be turned on; I'm not 
real sure and should research it. But here are some examples:
64-bit code, in general, tends to be a little slower than 32-bit code on 
the same CPU. There are exceptions, of course.

AMD64 has the advantage of adding a number of new registers, and that 
can boost performance enough to make up for it. AMD claims something 
like a 5% net increase with AMD64 vs. IA32 on their own chips. This is 
not enough to really be visible. Unless you happen to have one of those 
apps that's really well suited to AMD64, you won't benefit much.

Where the Athlon 64 and Opteron really shine is the integrated memory 
controller. Without it, early Opterons and A64s would have been slower 
than the Athlon XPs that were released before them, according to AMD's 
own performance numbers.

Also keep in mind that many of the most demanding apps (e.g., games and 
workstation graphics) are heavy on floating point math. Those won't see 
any benefit, and many companies have struggled to even match the 
performance of 32-bit code, much less exceed it. The Unreal Engine (used 
in numerous games) is one notorious example.

In answer to your question, the compiler will automatically attempt to 
make use of the extra registers when compiling on your AM64 system. Just 
don't expect too much.
--
http://www.velocityvector.com/ | http://www.indie-games.com/
http://www.classic-games.com/  | http://glmiller.blogspot.com/
"If my forgeries looked as bad as the CBS documents, it would
have been 'Catch Me In Two Days'"   -- Frank Abagnale, Jr.

Re: [sqlite] Version 3.1.0

[Robert L Cochran]

I should have indicated in my earlier post that magazines which do 
side-by-side hardware testing are saying that the AMD Athlon 64 is 
indeed faster than the Pentium 4; for example look in the January issue 
of Maximum PC magazine. they have a followup test supplementing an 
earlier review of Athlon vs. Pentium systems. Their testing results 
speak better than my faltering around with words. Tom's Hardware Guide 
has also compared Socket 939 Athlon systems to the latest that Intel 
has; I can't remember THG's exact judgement but it was overall more 
favorable to the Athlon systems.

Bob
Cory Nelson wrote:
You should see an increase automatically due to more registers being
available (assuming the compiler makes good use of them).  Other than
that, I suspect the only increase you will see is in parts that use
"long long" or __int64.
Also keep in mind, compilers are very new to 64bit compiling - it will
take a while for them to optimize to the full potential.
On Fri, 21 Jan 2005 20:23:44 -0800, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
 

As for SQLite, it is unlikely that you will see any performance
difference between a 32 bit or 64 bit compile of SQLite unless you are
specifically working with working sets that are larger than 4GB -- that
is, working sets that would not fit in a 32 bit process.
When compiled to run as a 64 bit process, any program will use more
memory simply because data alignments are typically on a wider boundary
and pointer sizes are that much larger.
"64 bit computing" is one of the great myths of marketing foisted upon
the computing industry.  Now that "ghz ghz ghz" seems to be largely
dead as a marketing myth, it would seem that "64 bits or die" is the
next great frontier.
There are real advantages to 64 bit, but "everything is just faster
with a recompile" is definitely not one of them.
b.bum
   


 




smime.p7s
Description: S/MIME Cryptographic Signature

Re: [sqlite] Version 3.1.0

[John Richard Moser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Uh.

The core is better.  Try benching a Barton (32 bit) next to a P4, you'll
see.  I think it takes something like a 266MHz barton to clock with a
1GHz P4.

I estimate a clawhammer core at probably ~1.25-1.5x faster than a Barton
core; but with the extra registers and with the code in long mode (built
- -march=k8), the total execution speed including those from compiler
gains is about 2.0x.  PIC has no overhead (something like 0.01%), where
it has something like 0.99002% on x86.

All you need is a compiler that spits out x86-64 code.  C is high level;
you don't do stupid shit like code for a CPU.


Robert L Cochran wrote:
> D. Richard Hipp wrote:
> 
>> On Fri, 2005-01-21 at 20:56 -0500, Robert L Cochran wrote:
>>  
>>
>>> Does this version take advantage of 64-bit cpu's like the AMD Athlon
>>> 64?   
>>
>>
>> I don't know.  What do you mean by "take advantage of"?
>> What does a 64-bit CPU do that you can't do with a 32-bit CPU?
>>
>> There are a few limits in SQLite that are based on the number of
>> bits in an integer.  For example, you can't have more than 32
>> tables in a join.  Except on a 64-bit machine where you can have
>> 64 tables in a join.  Is that what you mean by taking advantage
>> of a 64-bit CPU?  But note that we could get 32-bit CPUs to provide 64
>> tables in a join just by changing a single line of
>> code, namely changing
>>
>>typedef unsigned int Bitmask;
>>
>> into
>>
>>typedef unsigned long long int Bitmask;
>>
>> So how does that really "take advantage of" the 64-bitness of
>> some CPUs?
>>
>> I hear a lot of excitement about 64-bit CPUs which I really
>> do not understand.  Please explain this to me.
>>
>>  
>>
> I don't know how to explain this "excitement" myself, except through
> examples that might bore you because I don't know the details of how to
> write a program that takes advantage of a 64 bit cpu. The excitement is
> mainly about speed, I would say. To make compiled software harness that
> speed, it may be that only a compiler flag has to be turned on; I'm not
> real sure and should research it. But here are some examples:
> 
> 1. My wife, on her 32 bit Intel laptop, running Microsoft Access, finds
> her system slowed down to a crawl whenever she works with one table that
> has 109,000 rows. This is just the beginning of the huge tables she
> creates. She has a bunch of others.
> 
> I moved this table to my Athlon 64 machine and converted it to the Linux
> x86_64 version of MySQL version 4.1.7.  Then I queried the database
> using a PHP script to "select distinct" from 3 different columns and
> stuff the result sets into HTML select tags so that the client can pick
> valid column values for those columns. 3 queries, 109,000 rows...and
> they executed so fast the web page seems to come up instantly.
> 
> I plan to try this same query with SQLite as soon as I have time.
> 
> 2. I can install Fedora Core x86_64 in about 25 minutes on my Athlon
> machine. On a 32 bit Intel machine, I can take a lunch hour break and
> the install still might not be finished, depending on the machine's
> resources. And these are all NFS image installs. I don't do them from CD
> or DVD any longer. Using the network is faster.
> 
> 3. I generally enjoy blindingly fast compile times when I have to do
> compiles of large codebases such as PHP from source. I haven't yet tried
> compiling Mozilla on this machine, but I'll get there -- I've done it on
> my Pentium machines and it takes a minimum of 45 minutes to complete.
> 
> The sum total is that speaking for myself, I won't go back to an Intel
> 32 bit system if I can help it. 64 bit systems are fast! They save a lot
> of time.
> 
> Perhaps I'm making a stupid point, but perhaps there is a compiler
> switch you can turn on somewhere (or maybe the configure script in the
> *tar.gz source code already does it) that optimizes for a 64 bit system.
> Hmmm, perhaps I can rerun autoconf and libtool and get this magic to
> happen.
> 
> Bob Cochran
> Greenbelt, Maryland
> 
> 
> 
> 
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB8eFThDd4aOud5P8RAhAdAKCJmrtQyZ3Q5CT1TbVPuh4rid5PowCdG5NN
PcEmRjZ5lSbhtHu3A0q1efk=
=A4WB
-END PGP SIGNATURE-

Re: [sqlite] Version 3.1.0

[Cory Nelson]

You should see an increase automatically due to more registers being
available (assuming the compiler makes good use of them).  Other than
that, I suspect the only increase you will see is in parts that use
"long long" or __int64.

Also keep in mind, compilers are very new to 64bit compiling - it will
take a while for them to optimize to the full potential.


On Fri, 21 Jan 2005 20:23:44 -0800, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> As for SQLite, it is unlikely that you will see any performance
> difference between a 32 bit or 64 bit compile of SQLite unless you are
> specifically working with working sets that are larger than 4GB -- that
> is, working sets that would not fit in a 32 bit process.
> 
> When compiled to run as a 64 bit process, any program will use more
> memory simply because data alignments are typically on a wider boundary
> and pointer sizes are that much larger.
> 
> "64 bit computing" is one of the great myths of marketing foisted upon
> the computing industry.  Now that "ghz ghz ghz" seems to be largely
> dead as a marketing myth, it would seem that "64 bits or die" is the
> next great frontier.
> 
> There are real advantages to 64 bit, but "everything is just faster
> with a recompile" is definitely not one of them.
> 
> b.bum
> 
> 


-- 
Cory Nelson
http://www.int64.org

Re: [sqlite] Version 3.1.0

[Robert L Cochran]

D. Richard Hipp wrote:
On Fri, 2005-01-21 at 20:56 -0500, Robert L Cochran wrote:
 

Does this version take advantage of 64-bit cpu's like the AMD Athlon 64? 
   

I don't know.  What do you mean by "take advantage of"?
What does a 64-bit CPU do that you can't do with a 32-bit CPU?
There are a few limits in SQLite that are based on the number of
bits in an integer.  For example, you can't have more than 32
tables in a join.  Except on a 64-bit machine where you can have
64 tables in a join.  Is that what you mean by taking advantage
of a 64-bit CPU?  But note that we could get 32-bit CPUs to 
provide 64 tables in a join just by changing a single line of
code, namely changing

   typedef unsigned int Bitmask;
into
   typedef unsigned long long int Bitmask;
So how does that really "take advantage of" the 64-bitness of
some CPUs?
I hear a lot of excitement about 64-bit CPUs which I really
do not understand.  Please explain this to me.
 

I don't know how to explain this "excitement" myself, except through 
examples that might bore you because I don't know the details of how to 
write a program that takes advantage of a 64 bit cpu. The excitement is 
mainly about speed, I would say. To make compiled software harness that 
speed, it may be that only a compiler flag has to be turned on; I'm not 
real sure and should research it. But here are some examples:

1. My wife, on her 32 bit Intel laptop, running Microsoft Access, finds 
her system slowed down to a crawl whenever she works with one table that 
has 109,000 rows. This is just the beginning of the huge tables she 
creates. She has a bunch of others.

I moved this table to my Athlon 64 machine and converted it to the Linux 
x86_64 version of MySQL version 4.1.7.  Then I queried the database 
using a PHP script to "select distinct" from 3 different columns and 
stuff the result sets into HTML select tags so that the client can pick 
valid column values for those columns. 3 queries, 109,000 rows...and 
they executed so fast the web page seems to come up instantly.

I plan to try this same query with SQLite as soon as I have time.
2. I can install Fedora Core x86_64 in about 25 minutes on my Athlon 
machine. On a 32 bit Intel machine, I can take a lunch hour break and 
the install still might not be finished, depending on the machine's 
resources. And these are all NFS image installs. I don't do them from CD 
or DVD any longer. Using the network is faster.

3. I generally enjoy blindingly fast compile times when I have to do 
compiles of large codebases such as PHP from source. I haven't yet tried 
compiling Mozilla on this machine, but I'll get there -- I've done it on 
my Pentium machines and it takes a minimum of 45 minutes to complete.

The sum total is that speaking for myself, I won't go back to an Intel 
32 bit system if I can help it. 64 bit systems are fast! They save a lot 
of time.

Perhaps I'm making a stupid point, but perhaps there is a compiler 
switch you can turn on somewhere (or maybe the configure script in the 
*tar.gz source code already does it) that optimizes for a 64 bit system. 
Hmmm, perhaps I can rerun autoconf and libtool and get this magic to happen.

Bob Cochran
Greenbelt, Maryland




smime.p7s
Description: S/MIME Cryptographic Signature

Re: [sqlite] Version 3.1.0

[Jeff Edwards]

To the unlearned laymen, 64bit CPU's mean that processing is performed 
twice as fast.

The assumption is that if an application is compiled for 64bit, then it 
will be operating faster.

Perhaps this is what Robert's assumption of 'taking advantage' was?
Jeff Edwards
On 22/01/2005, at 11:08 AM, D. Richard Hipp wrote:
On Fri, 2005-01-21 at 20:56 -0500, Robert L Cochran wrote:
Does this version take advantage of 64-bit cpu's like the AMD Athlon 
64?
I don't know.  What do you mean by "take advantage of"?
What does a 64-bit CPU do that you can't do with a 32-bit CPU?

Re: [sqlite] Version 3.1.0

[D. Richard Hipp]

On Fri, 2005-01-21 at 20:56 -0500, Robert L Cochran wrote:
> Does this version take advantage of 64-bit cpu's like the AMD Athlon 64? 

I don't know.  What do you mean by "take advantage of"?
What does a 64-bit CPU do that you can't do with a 32-bit CPU?

There are a few limits in SQLite that are based on the number of
bits in an integer.  For example, you can't have more than 32
tables in a join.  Except on a 64-bit machine where you can have
64 tables in a join.  Is that what you mean by taking advantage
of a 64-bit CPU?  But note that we could get 32-bit CPUs to 
provide 64 tables in a join just by changing a single line of
code, namely changing

typedef unsigned int Bitmask;

into

typedef unsigned long long int Bitmask;

So how does that really "take advantage of" the 64-bitness of
some CPUs?

I hear a lot of excitement about 64-bit CPUs which I really
do not understand.  Please explain this to me.

-- 
D. Richard Hipp <[EMAIL PROTECTED]>

Re: [sqlite] Version 3.1.0

[Robert L Cochran]

D. Richard Hipp wrote:
Version 3.1.0 (alpha) of SQLite is now available on the website.
Release notes are available from a link on the homepage.
This released is labeled "alpha" but it is still very well
tested.  By being "alpha" it means that there is still a
small window of opportunity during when users can suggest
API changes.  Once we go to beta (in about a week) no more
changes will be accepted.  So if you want to suggest changes,
please do so quickly.
 

Does this version take advantage of 64-bit cpu's like the AMD Athlon 64? 
I just compiled it successfully on Fedora Core 3 x86_64, which is 
running on an Athlon 64 3500+, but I'm puzzled as to whether I'm 
generating a 64-bit aware version or not.

Bob Cochran
Greenbelt, Maryland, USA

Re: [sqlite] Version 3.1.0

[John Richard Moser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I'd probably suggest something about how to set BLOB data from a void*
and an int in 1 line of code, if I had any idea at all how blob data is
set now.

I found an example, but it was something like 400 lines long, overly
complex (loading a blob into a db from a file and parsing command lines
etc etc) and had 0 comments.  It was part of "sqlite_examples.tar.gz or
something.  What idiot writes example code for newbies to reference
without adding in comments?

Anyway, still hammering google for C Sqlite3 examples, though I'm just
getting C#.net wrappers and RUBY and PERL APIs and crap.

D. Richard Hipp wrote:
> Version 3.1.0 (alpha) of SQLite is now available on the website.
> Release notes are available from a link on the homepage.
> 
> This released is labeled "alpha" but it is still very well
> tested.  By being "alpha" it means that there is still a
> small window of opportunity during when users can suggest
> API changes.  Once we go to beta (in about a week) no more
> changes will be accepted.  So if you want to suggest changes,
> please do so quickly.
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB8aTghDd4aOud5P8RApIBAJ42DIUyVwbJa80FdSJYnRFPt1cSkgCfdO20
xJ0hGQIQFCoNPfzbxMAFDuk=
=v0Yq
-END PGP SIGNATURE-

Re: [sqlite] Version 3.1.0

[Dan Kennedy]

--- Doug Currie <[EMAIL PROTECTED]> wrote:

> Friday, January 21, 2005, 1:33:35 PM, DRH wrote:
> 
> > Version 3.1.0 (alpha) of SQLite is now available on the website.
> 
> Compiling with MinGW MSYS on WinXP...
> 
> 1. I had to modify my lib/tclConfig.sh to have
>   TCL_LIB_SPEC='-L/mingw/lib -ltcl84'
> instead of
>   TCL_LIB_SPEC=''
> or else testfixture wouldn't build. Dunno why my tclConfig.sh was broken.
> 
> 2. autovacuum-ioerr2.4.0 and onward fail...
> 
> autovacuum-ioerr2.4.0...
> Error: error copying "test.db" to "backup.db": no such file or directory
> autovacuum-ioerr2-4.1.1...
> Error: error copying "backup.db" to "test.db": no such file or directory
> autovacuum-ioerr2-4.1.2... Ok
> autovacuum-ioerr2-4.1.3... Ok

I had the same problem. It's caused by a bug in the version of TCL 
that comes with mingw. If you upgrade TCL (I used ActiveTcl 8.4.9)
it should work.




__ 
Do you Yahoo!? 
Take Yahoo! Mail with you! Get it on your mobile phone. 
http://mobile.yahoo.com/maildemo 

Re: [sqlite] Version 3.1.0

[Dennis Cote]

Doug Currie wrote:
Friday, January 21, 2005, 1:33:35 PM, DRH wrote:
Version 3.1.0 (alpha) of SQLite is now available on the website.
Compiling with MinGW MSYS on WinXP...
1. I had to modify my lib/tclConfig.sh to have
 TCL_LIB_SPEC='-L/mingw/lib -ltcl84'
instead of
 TCL_LIB_SPEC=''
or else testfixture wouldn't build. Dunno why my tclConfig.sh was
broken.
2. autovacuum-ioerr2.4.0 and onward fail...
autovacuum-ioerr2.4.0...
Error: error copying "test.db" to "backup.db": no such file or
directory autovacuum-ioerr2-4.1.1...
Error: error copying "backup.db" to "test.db": no such file or
directory autovacuum-ioerr2-4.1.2... Ok
autovacuum-ioerr2-4.1.3... Ok
c:\dev\sqlite3\bld\testfixture.exe: invalid command name "db"
   while executing
"db close"
   (file "../sqlite/test/autovacuum_ioerr2.test" line 189)
   invoked from within
"source $testfile"
   ("foreach" body line 4)
   invoked from within
"foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
 set tail [file tail $testfile]
 if {[lsearch -exact $EXCLUDE $tail]>=0} continue
 so..."
   (file "../sqlite/test/quick.test" line 44)
make: *** [test] Error 1
e
Doug,
I ran into the same problem with testfixture using MinGW. I fixed it by 
modifying the sqlite Makefile, but your change should be more permanent.

The errors in autovacuum-ioerr2 (and similat errors in the ioerr2 and pager 
test files) are due to a broken 'file copy' command in the 8.4.1 tcl package 
distributed with MinGW. Dan Kennedy pointed this out and suggested ActiveTcl 
as a suitable replacement. I installed ActiveTcl 8.4.9 and got a similar 
error when building testfixture.exe, except now it was including a couple of 
extra dlls on the link line. I again modified the sqlite Makefile to remove 
these dlls from the LIBTCL line. Everything builds, and all the tests run 
(with the exception of bind-4.1 due to a trivial formatting issue).

HTH
Dennis Cote

Re: [sqlite] Version 3.1.0 API suggestion: sqlite3_commit()

[Eric Scouten]

Ulrik Petersen wrote:
Eric,
Eric Scouten wrote:
D. Richard Hipp wrote:
This released is labeled "alpha" but it is still very well
tested.  By being "alpha" it means that there is still a
small window of opportunity during when users can suggest
API changes.  Once we go to beta (in about a week) no more
changes will be accepted.  So if you want to suggest changes,
please do so quickly.
 

I'm making a push to use prepared statements as much as possible and 
I've been unable to make use of a prepared statement that commits a 
transaction.

What I've tried (roughly, I'm doing this from memory since the code 
is gone now):

   sqlite3_exec( "BEGIN TRANSACTION;" );
  // do other stuff...
   sqlite3_stmt commitStmt;
   sqlite3_prepare( db, "COMMIT;", -1, , NULL );
   sqlite3_step( commitStmt );
  // error...
I forget the exact error message and result code, but the jist of it 
was that the transaction couldn't be committed because one or more 
statements were still running. From what I can tell, the only 
statement that hadn't been run to completion was the commit statement 
itself!

Bear in mind that this is a quick-n-dirty example to demonstrate the 
issue. In real life, I would have prepared and reused the "BEGIN 
TRANSACTION" statement (which works, BTW), cached commitStmt, paid 
attention to result codes, etc., etc.

It seems kind of silly to me to have to use sqlite3_exec( ... 
"COMMIT" ... ) to work around that problem, but that's what I'm doing 
at the moment. If there's a way to prepare and use a "COMMIT" 
statement, please let me know. If not, consider this a feature 
request for a sqlite3_commit() function so I don't have to spend the 
time compiling "COMMIT" over and over again.

It's not completely clear how you detect the error, so I have to ask: 
Did you use sqlite3_finalize?
I never got as far as sqlite3_finalize, because the error was reported 
on sqlite3_step.

-Eric

Re: [sqlite] Version 3.1.0

[Raymond Irving]

Hi,

It would be nice if Autovacum can be set for a
database with data and also be set to occur at timely
intervals. Maybe something like every 1000 updates or
deletes.

__
Raymond Irving

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:

> Version 3.1.0 (alpha) of SQLite is now available on
> the website.
> Release notes are available from a link on the
> homepage.
> 
> This released is labeled "alpha" but it is still
> very well
> tested.  By being "alpha" it means that there is
> still a
> small window of opportunity during when users can
> suggest
> API changes.  Once we go to beta (in about a week)
> no more
> changes will be accepted.  So if you want to suggest
> changes,
> please do so quickly.
> 
> -- 
> D. Richard Hipp <[EMAIL PROTECTED]>
> 
> 

Re: [sqlite] Version 3.1.0

[Doug Currie]

Friday, January 21, 2005, 1:33:35 PM, DRH wrote:

> Version 3.1.0 (alpha) of SQLite is now available on the website.

Compiling with MinGW MSYS on WinXP...

1. I had to modify my lib/tclConfig.sh to have
  TCL_LIB_SPEC='-L/mingw/lib -ltcl84'
instead of
  TCL_LIB_SPEC=''
or else testfixture wouldn't build. Dunno why my tclConfig.sh was broken.

2. autovacuum-ioerr2.4.0 and onward fail...

autovacuum-ioerr2.4.0...
Error: error copying "test.db" to "backup.db": no such file or directory
autovacuum-ioerr2-4.1.1...
Error: error copying "backup.db" to "test.db": no such file or directory
autovacuum-ioerr2-4.1.2... Ok
autovacuum-ioerr2-4.1.3... Ok
c:\dev\sqlite3\bld\testfixture.exe: invalid command name "db"
while executing
"db close"
(file "../sqlite/test/autovacuum_ioerr2.test" line 189)
invoked from within
"source $testfile"
("foreach" body line 4)
invoked from within
"foreach testfile [lsort -dictionary [glob $testdir/*.test]] {
  set tail [file tail $testfile]
  if {[lsearch -exact $EXCLUDE $tail]>=0} continue
  so..."
(file "../sqlite/test/quick.test" line 44)
make: *** [test] Error 1


e

Re: [sqlite] Version 3.1.0 API suggestion: sqlite3_commit()

[Ulrik Petersen]

Eric,
Eric Scouten wrote:
D. Richard Hipp wrote:
This released is labeled "alpha" but it is still very well
tested.  By being "alpha" it means that there is still a
small window of opportunity during when users can suggest
API changes.  Once we go to beta (in about a week) no more
changes will be accepted.  So if you want to suggest changes,
please do so quickly.
 

I'm making a push to use prepared statements as much as possible and 
I've been unable to make use of a prepared statement that commits a 
transaction.

What I've tried (roughly, I'm doing this from memory since the code is 
gone now):

   sqlite3_exec( "BEGIN TRANSACTION;" );
  // do other stuff...
   sqlite3_stmt commitStmt;
   sqlite3_prepare( db, "COMMIT;", -1, , NULL );
   sqlite3_step( commitStmt );
  // error...
I forget the exact error message and result code, but the jist of it 
was that the transaction couldn't be committed because one or more 
statements were still running. From what I can tell, the only 
statement that hadn't been run to completion was the commit statement 
itself!

Bear in mind that this is a quick-n-dirty example to demonstrate the 
issue. In real life, I would have prepared and reused the "BEGIN 
TRANSACTION" statement (which works, BTW), cached commitStmt, paid 
attention to result codes, etc., etc.

It seems kind of silly to me to have to use sqlite3_exec( ... "COMMIT" 
... ) to work around that problem, but that's what I'm doing at the 
moment. If there's a way to prepare and use a "COMMIT" statement, 
please let me know. If not, consider this a feature request for a 
sqlite3_commit() function so I don't have to spend the time compiling 
"COMMIT" over and over again.
It's not completely clear how you detect the error, so I have to ask: 
Did you use sqlite3_finalize?

From :
"The sqlite3_finalize() routine deallocates a prepared SQL statement. 
All prepared statements must be finalized before the database can be 
closed. The sqlite3_reset() routine resets a prepared SQL statement so 
that it can be executed again."

HTH
Ulrik P.
--
Ulrik Petersen, MA, B.Sc.

Re: [sqlite] Version 3.1.0 API suggestion: sqlite3_commit()

[Eric Scouten]

D. Richard Hipp wrote:
This released is labeled "alpha" but it is still very well
tested.  By being "alpha" it means that there is still a
small window of opportunity during when users can suggest
API changes.  Once we go to beta (in about a week) no more
changes will be accepted.  So if you want to suggest changes,
please do so quickly.
 

I'm making a push to use prepared statements as much as possible and 
I've been unable to make use of a prepared statement that commits a 
transaction.

What I've tried (roughly, I'm doing this from memory since the code is 
gone now):

   sqlite3_exec( "BEGIN TRANSACTION;" );
  // do other stuff...
   sqlite3_stmt commitStmt;
   sqlite3_prepare( db, "COMMIT;", -1, , NULL );
   sqlite3_step( commitStmt );
  // error...
I forget the exact error message and result code, but the jist of it was 
that the transaction couldn't be committed because one or more 
statements were still running. From what I can tell, the only statement 
that hadn't been run to completion was the commit statement itself!

Bear in mind that this is a quick-n-dirty example to demonstrate the 
issue. In real life, I would have prepared and reused the "BEGIN 
TRANSACTION" statement (which works, BTW), cached commitStmt, paid 
attention to result codes, etc., etc.

It seems kind of silly to me to have to use sqlite3_exec( ... "COMMIT" 
... ) to work around that problem, but that's what I'm doing at the 
moment. If there's a way to prepare and use a "COMMIT" statement, please 
let me know. If not, consider this a feature request for a 
sqlite3_commit() function so I don't have to spend the time compiling 
"COMMIT" over and over again.

FWIW, I think I was using 3.0.7 at the time I encountered this issue.
-Eric

Re: [sqlite] Version 3.1.0

[Eli Burke]

D. Richard Hipp wrote:
This released is labeled "alpha" but it is still very well
tested.  By being "alpha" it means that there is still a
small window of opportunity during when users can suggest
API changes.  Once we go to beta (in about a week) no more
changes will be accepted.  So if you want to suggest changes,
please do so quickly.
 

Here's a suggestion for an API change:
A lot of my code where I don't have fixed queries ends up looking 
something like this:

   int rc;
   char * pSql;
   pSql = sqlite3_mprintf(queryPattern, argList);
   if (pSql == NULL)
   {
   // error case
   }
   else
   {
   rc = sqlite3_exec(database, pSql, 0, 0, 0);
   sqlite3_free(pSql);
   if (rc != SQLITE_OK)
   {
  // error case
   }
   }
For the most part I don't use sqlite3_exec callbacks (for such 
situations there is
prepare/update/finish), nor do I use the sqlite3_exec errmsg, which just 
makes for
another variable to declare and then free in the error cases. (it was 
helpful when I
was first learning about sqlite, but is well expressed by the return code).

So it would be nice if sqlite3_exec or another function combined the 
features of
the current sqlite3_exec and sqlite3_mprintf:

rc = sqlite3_mprintf_exec(database, queryPattern, argList);
Ok yeah maybe it's a trivial change, and easily re-created by the end 
user with a
wrapper function (not helping my argument here, am I...) but for the 
people who
don't think to wrap it, or who don't do much error checking, it would be 
a boon.

-Eli

Re: [sqlite] Version 3.1.0

[Cory Nelson]

Any API changes?


On Fri, 21 Jan 2005 13:33:35 -0500, D. Richard Hipp <[EMAIL PROTECTED]> wrote:
> Version 3.1.0 (alpha) of SQLite is now available on the website.
> Release notes are available from a link on the homepage.
> 
> This released is labeled "alpha" but it is still very well
> tested.  By being "alpha" it means that there is still a
> small window of opportunity during when users can suggest
> API changes.  Once we go to beta (in about a week) no more
> changes will be accepted.  So if you want to suggest changes,
> please do so quickly.
> 
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
> 
> 


-- 
Cory Nelson
http://www.int64.org