Re: [sqlite] Next Version of SQLite

[John Stanton]

[EMAIL PROTECTED] wrote:

Joe Wilson <[EMAIL PROTECTED]> wrote:

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:
Sorry for the confusion. 

No problem.

For what it's worth, I am also curious as to the final form of the 
VM opcode transformation. The number of opcodes generated by the various
SQL statements seems to be roughly the same as the old scheme. At this 
point without sub-expresssion elimination are you seeing any speed 
improvement?




I have not even looked at performance yet.  I'm assuming that
performance will drop during the conversion process and that we
will have to fight to get it back up to previous levels after
the conversion is complete.

But consider would can be done with a register machine that
would couldn't do with the old stack machine.  In a statement
like this:

SELECT * FROM a NATURAL JOIN b;

Suppose tables a and b have column c in common and unique
columns a1, a2, a3 and b1, b2, b3.  With the stack machine, 
the algorithm is roughly this:


foreach each entry in a:
  foreach entry in b with b.c==a.c:
push a.c
push a.a1
push a.a2
push a.a3
push b.b1
push b.b2
push b.b3
return one row of result
  endforeach
endforeach

For each result row, all columns had to be pushed onto the
stack.  Then the OP_Callback opcode would fire, causing
sqlite3_step() to return SQLITE_ROW.  The result columns would
then be available to sqlite3_column_xxx() routines which read
those results off of the stack.  When sqlite3_step() is called
again, all result columns are popped from the stack and 
execution continues with the first operation after the 
OP_Callback.


In the register VM, result columns are stored in a consecutive
sequence of registers.  It is no longer necessary to pop the
stack of prior results at the start of each sqlite3_step().
So the code can look more like this:

foreach entry in a:
  r1 = a.c
  r2 = a.a1
  r3 = a.a2
  r4 = a.a3
  foreach entry in b where c=r1:
r5 = b.b1
r6 = b.b2
r7 = b.b3
return one row of result
  endforeach
endforeach

When result are stored in registers, the computation of the 
first four columns of the result set can be factored out of 
the inner loop.  If there are 10 matching rows in b for every

row in a, this might result in a significant performance boost.

Do not look for this improvement right away, though.  The
first order of business is to get the VM converted over into
a register machine.  Only after that is successfully accomplished
will we look into implementing optimizations such as the above.

--
D. Richard Hipp <[EMAIL PROTECTED]>

This architecture replacement is a very significant step for Sqlite from 
my perspective.  It indicates a transition from the simple, more easily 
implemented technology into a more refined one providing the basis for 
adding more sophistication in optimization and a generally more 
efficient core.


Such ongoing evolution guarantees that Sqlite will not ossify and become
irrelevant.  Congratulation Dr Hipp and team for not leaving well alone.



-
To unsubscribe, send email to [EMAIL PROTECTED]
-




-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Bill KING]

[EMAIL PROTECTED] wrote:
> Bill KING <[EMAIL PROTECTED]> wrote:
>   
>> A colleague brought up a very good point. At least for the first few
>> revisions, is the old engine/code still going to be available until the
>> new engine code base settles down? 
>>
>> 
>
> You can always pull the old code frm CVS and run it against
> the newer code.  We started making the switch with check-in [4652]
> on 2008-01-02.  So any CVS verious prior to that will be a pure
> stack-based machine.  
>   
As expected, will be setting up a set of auto-tests for our particular
use-case, but that's just being a good software engineer ;)
> The changes consist of slowly adding capability to the existing
> VM so that it can work with operands on the stack or in registers.
> This continued through check-in [4700] on 2008-01-09.  After [4700],
> all operations can be done using register operands, though the
> code generator still uses the stack for many things.  Since then
> and moving forward, we are converting the code generator to use
> registers rather than the stack, and we are removing stack
> capabilities from the VM as those abilities lapse from use.
> Toward the end of the process, we will completely remove the
> operand stack from the VM.
>   
One hell of a challenge, but in the end, I do agree with you, there's
been a lot of knowledge around for a long time on register based
optimisations, and with an infinite register machine, some really
amazing optimizations can occur, agreed. Besides, all the cool kids are
doing it too ;)
> During the conversion process, SQLite will likely be slower and
> use more memory.  But all regression tests are suppose to pass 
> after each check-in.  So we hope to avoid introducing any bugs.
> Once the conversion is complete, we hope to be able to reduce
> the memory footprint of each sqlite3_stmt object and make them
> run a little faster too.  But we need to get a correctly functioning
> pure register VM first.
>   
This part I'm happy with "all regression tests are supposed to pass" :)
I'm still going to wait a point or two before production tho, but that's
again just careful management. Looking forward to this tho in the long run.
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
>
>
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
>
>
>   


-- 
Bill King, Software Engineer
Trolltech, Brisbane Technology Park
26 Brandl St, Eight Mile Plains, 
QLD, Australia, 4113
Tel + 61 7 3219 9906 (x137)
Fax + 61 7 3219 9938
mobile: 0423 532 733


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Nicolas Williams]

On Mon, Jan 14, 2008 at 10:55:42AM -0600, Fred Williams wrote:
> >Which means that if the major version number changes,
> > then it'll be for marketing purposes.
> 
> Well seeing how SQLite is FREE, it does its own "marketing" with one
> word, so to speak.  Therefore I doubt marketing will have much impact on
> versioning.  What a quandary develops when one goes against the grain of
> commercial software deployment :-)

I wouldn't knock marketing, even for open source, public domain projects
like this one, even where marketing has been of the word-of-mouth type.

The question is: will 4.0 scare folks ("oh dear, must be a new API, just
like 3.0 meant") or entice them ("oh cool!  must be way better, so I
should go learn it now")?

I have no idea what the answer to that is.

One very good aspect of SQLite 3.x has been stability and evolution
(i.e., few if any incompatible changes).  This too is very important
from a marketing p.o.v.

The safest bet is that 3.6.0 is a good enough move.

Nico
-- 

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

RE: [sqlite] Next Version of SQLite

[Fred Williams]

> -Original Message-
> From: Nicolas Williams [mailto:[EMAIL PROTECTED]
> Sent: Monday, January 14, 2008 10:29 AM
> To: sqlite-users@sqlite.org
> Subject: Re: [sqlite] Next Version of SQLite
>
>
> On Sun, Jan 13, 2008 at 08:46:03PM -0600, Rick Langschultz wrote:
> > I was wondering what would constitute the creation of SQLite 4.0?
>
> IMO major implementation details changes are not necessarily a good
> rationale for bumping the major version number, not from a
> user's p.o.v.

...
>
> My guess is that the 3.x APIs are plenty good enough that incremental
> evolution is far better than incompatible revolution, so I
> bet they are
> here to stay.  Which means that if the major version number changes,
> then it'll be for marketing purposes.
>
> Nico
> --

Well seeing how SQLite is FREE, it does its own "marketing" with one
word, so to speak.  Therefore I doubt marketing will have much impact on
versioning.  What a quandary develops when one goes against the grain of
commercial software deployment :-)

Fred


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Nicolas Williams]

On Sun, Jan 13, 2008 at 08:46:03PM -0600, Rick Langschultz wrote:
> I was wondering what would constitute the creation of SQLite 4.0?

IMO major implementation details changes are not necessarily a good
rationale for bumping the major version number, not from a user's p.o.v.
The authors might think it is though, and from a marketing p.o.v. it
really, really might be (perception matters).

If changing the major version number need not imply changing the C API
prefix, then it's certainly easier to change the major version number.

But I think developers are used to "sqlite3_" as the prefix now --
changing the major version number without changing that prefix will seem
odd, but changing the prefix without substantive changes to the API will
be gratuitous and very painful.

If the 3.x C APIs became a stone around SQLite's neck and couldn't be
incrementally fixed (by deprecating a small number of functions and
adding a few new ones), then the time might be right for changing the
prefix to sqlite4_.  If the DB format had to change radically, then that
too might be a good reason to change the major version number.

My guess is that the 3.x APIs are plenty good enough that incremental
evolution is far better than incompatible revolution, so I bet they are
here to stay.  Which means that if the major version number changes,
then it'll be for marketing purposes.

Nico
-- 

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[drh]

Joe Wilson <[EMAIL PROTECTED]> wrote:
> --- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:
> > Sorry for the confusion. 
> 
> No problem.
> 
> For what it's worth, I am also curious as to the final form of the 
> VM opcode transformation. The number of opcodes generated by the various
> SQL statements seems to be roughly the same as the old scheme. At this 
> point without sub-expresssion elimination are you seeing any speed 
> improvement?
> 

I have not even looked at performance yet.  I'm assuming that
performance will drop during the conversion process and that we
will have to fight to get it back up to previous levels after
the conversion is complete.

But consider would can be done with a register machine that
would couldn't do with the old stack machine.  In a statement
like this:

SELECT * FROM a NATURAL JOIN b;

Suppose tables a and b have column c in common and unique
columns a1, a2, a3 and b1, b2, b3.  With the stack machine, 
the algorithm is roughly this:

foreach each entry in a:
  foreach entry in b with b.c==a.c:
push a.c
push a.a1
push a.a2
push a.a3
push b.b1
push b.b2
push b.b3
return one row of result
  endforeach
endforeach

For each result row, all columns had to be pushed onto the
stack.  Then the OP_Callback opcode would fire, causing
sqlite3_step() to return SQLITE_ROW.  The result columns would
then be available to sqlite3_column_xxx() routines which read
those results off of the stack.  When sqlite3_step() is called
again, all result columns are popped from the stack and 
execution continues with the first operation after the 
OP_Callback.

In the register VM, result columns are stored in a consecutive
sequence of registers.  It is no longer necessary to pop the
stack of prior results at the start of each sqlite3_step().
So the code can look more like this:

foreach entry in a:
  r1 = a.c
  r2 = a.a1
  r3 = a.a2
  r4 = a.a3
  foreach entry in b where c=r1:
r5 = b.b1
r6 = b.b2
r7 = b.b3
return one row of result
  endforeach
endforeach

When result are stored in registers, the computation of the 
first four columns of the result set can be factored out of 
the inner loop.  If there are 10 matching rows in b for every
row in a, this might result in a significant performance boost.

Do not look for this improvement right away, though.  The
first order of business is to get the VM converted over into
a register machine.  Only after that is successfully accomplished
will we look into implementing optimizations such as the above.

--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[drh]

Bill KING <[EMAIL PROTECTED]> wrote:
> A colleague brought up a very good point. At least for the first few
> revisions, is the old engine/code still going to be available until the
> new engine code base settles down? 
> 

You can always pull the old code frm CVS and run it against
the newer code.  We started making the switch with check-in [4652]
on 2008-01-02.  So any CVS verious prior to that will be a pure
stack-based machine.  

The changes consist of slowly adding capability to the existing
VM so that it can work with operands on the stack or in registers.
This continued through check-in [4700] on 2008-01-09.  After [4700],
all operations can be done using register operands, though the
code generator still uses the stack for many things.  Since then
and moving forward, we are converting the code generator to use
registers rather than the stack, and we are removing stack
capabilities from the VM as those abilities lapse from use.
Toward the end of the process, we will completely remove the
operand stack from the VM.

During the conversion process, SQLite will likely be slower and
use more memory.  But all regression tests are suppose to pass 
after each check-in.  So we hope to avoid introducing any bugs.
Once the conversion is complete, we hope to be able to reduce
the memory footprint of each sqlite3_stmt object and make them
run a little faster too.  But we need to get a correctly functioning
pure register VM first.
--
D. Richard Hipp <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Jim Dodgen]

this was a true complement and nothing else.

P Kishor wrote:

On 1/13/08, Joe Wilson <[EMAIL PROTECTED]> wrote:
  

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:


There are people on this mailing list (ex: Joe Wilson) who appear
to read every line of every change that we make to SQLite, within
minutes of making them, and complain if we so much as misspell a
word in a comment.  And I haven't heard a peep from Joe
  

Wow - what prompted that dig against me?

Fixing a spelling mistake in a comment adds no value to the software.
None of those spelling mistake tickets are my doing.

The majority of the tickets I've reported were legitimate problems
in the code - and arguing the case for some of these bugs isn't
exactly the easiest thing. But hey, it's cool. I won't file any
more tickets.





honestly, coming from Richard, that is most definitely a compliment to
an "old friend" indicative of the value of your contribution.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



  



-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Joe Wilson]

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:
> Sorry for the confusion. 

No problem.

For what it's worth, I am also curious as to the final form of the 
VM opcode transformation. The number of opcodes generated by the various
SQL statements seems to be roughly the same as the old scheme. At this 
point without sub-expresssion elimination are you seeing any speed 
improvement?



  

Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ 


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Rick Langschultz wrote:
> I was wondering what would constitute the creation of SQLite 4.0?

An incompatible API, or significant behaviour changes.

> Since the VDBE is being revamped I would consider this a pretty big
> revamp of the SQLite code. 

The VDBE is not visible outside of the SQLite library except when
EXPLAIN is used.  It is immaterial to you as a user of SQLite as long as
it is correctly implemented.  For example if SQLite started counting
internally using roman numerals but still always gave correct (decimal)
results then it doesn't matter.

> I also wanted to know what the difference between stack based and
> register based is. Unfortunately, i do not mess with a lot of C / C++
> code in my work. So I was a little curious.

http://en.wikipedia.org/wiki/Virtual_machine#List_of_virtual_machine_software

Except for trivial ones, all virtual machines have a stack.  For example
it is how you implement recursion with the stack growing and shrinking
as you enter and leave "functions".  It is also where intermediate
values are stored.  Expressions are "compiled" to work on the stack.
For example something like b+2*c would become something like:

  push b
  push 2
  push c
  mulitply
  add

To understand this well, I strongly recommend looking up the Forth
programming language and a few Forth tutorials.

Stack based virtual machines as above are easy to understand and
implement.  However there is a limit to their performance.  Every
operation manipulates the stack and you can end up with a lot of stack
manipulation code as operations such as add and multiply in the example
above only work on the top operands of the stack. You also have to be
careful when generating the opcodes that you don't overflow the stack,
and that functions return the stack in the same state they found it (it
would be disastrous if there were more or less entries on it).

Real hardware uses both a stack and registers.  Registers are a fast
temporary storage location, but you can still use the stack.  For
example an x86 processor in 32 bit mode has 8 registers.  For virtual
machines there is no need to limit how many registers there are.

http://en.wikipedia.org/wiki/Processor_register

Current best practise in compilers is to use registers (and has been for
many decades).  Look up three address code and SSA as well as the
optimisations they allow.  It is also easier to JIT register based code.

Virtual machines up to and including Java tended to be stack based.
.NET, parrot, llvm etc are all register based because of the advantages.

In the case of SQLite I suspect that being register based will result in
fewer operations per program and each operation will execute faster (no
futzing with the stack).  You can try it for yourself by using EXPLAIN
in 3.5.4 vs CVS.

Here is a good description of why the Parrot VM used register rather
than stack:

 http://www.sidhe.org/~dan/blog/archives/000189.html

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHitZnmOOfHg372QQRAljEAJ97yFOuWh6/lOlV572iUjHrHThMfgCfQsjN
LMCBk5uiO5IAsUADf96F9lQ=
=VNz/
-END PGP SIGNATURE-

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Bill KING]

A colleague brought up a very good point. At least for the first few
revisions, is the old engine/code still going to be available until the
new engine code base settles down? (via #defines maybe?). It would lead
towards a good chance of comparison between the two engines too for people.

D. Richard Hipp wrote:
>
> On Jan 13, 2008, at 11:31 AM, Rich Shepard wrote:
>
>> On Sun, 13 Jan 2008, Darren Duncan wrote:
>>
>>> I would think something like that is worthy of a 3.6.0 version
>>> number. Not just a minor version increase that would be more
>>> suitable for minor
>>> changes or bugfixes.
>>
>>   I agree with Darren that massive changes to the core of the system
>> should
>> be reflected by a major version increase (to 4.0); at a minimum to a
>> minor
>> version increease (to 3.6). A version number change from 3.5.4 to 3.5.5
>> tells folks that it's a minor bug fix or simple adjustment, not a
>> wholesale
>> rewrite of the system's core.
>>
>
> There are no user-visible changes to the interface.  The version numbers
> in SQLite reflect user-visible changes only.
>
> Well, there is one minor user-visible change.  The output of EXPLAIN now
> has 7 columns whereas it used to contain only 5.  But the output of
> EXPLAIN
> changes from point release to point release all the time anyway, so I do
> not consider this something worth bumping a version number.
>
> I do not expect significant instability with the next release.  I want
> to gain
> some experience with the new software before I recommend it for millions
> of deployments.  But it should be solid and stable as soon as it is
> released.
> For that matter, the current code in CVS (which is well into the
> conversion
> to a register machine) has not been giving any problems.  There are
> people
> on this mailing list (ex: Joe Wilson) who appear to read every line of
> every
> change that we make to SQLite, within minutes of making them, and
> complain
> if we so much as misspell a word in a comment.  And I haven't heard a
> peep
> from Joe or anybody else, so I'm thinking the code is still working
> correctly
> for everybody despite the massive changes that have already gone in.
> If you find that the current code in CVS gives problems, or if you see
> significant problems emerge as we get closer to releasing 3.5.5, then
> maybe we might consider calling it 3.6.0.  But I do not anticipate any
> serious problems.  You should not underestimate the level of detail to
> which we test SQLite and the thoroughness of the test suite.  Not much
> is likely to slip through the cracks.
>
>
> D. Richard Hipp
> [EMAIL PROTECTED]
>
>
>
>
> -
>
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
>
>
>


-- 
Bill King, Software Engineer
Trolltech, Brisbane Technology Park
26 Brandl St, Eight Mile Plains, 
QLD, Australia, 4113
Tel + 61 7 3219 9906 (x137)
Fax + 61 7 3219 9938
mobile: 0423 532 733


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Darren Duncan]

At 8:46 PM -0600 1/13/08, Rick Langschultz wrote:

I was wondering what would constitute the creation of SQLite 4.0?

Since the VDBE is being revamped I would consider this a pretty big 
revamp of the SQLite code. I am looking forward to testing this out 
new engine out.


I also wanted to know what the difference between stack based and 
register based is. Unfortunately, i do not mess with a lot of C / 
C++ code in my work. So I was a little curious.


Although I would have considered the current large update to be 3.6.0 
material, I definitely do *not* consider it to be 4.0.0 material.


Major version updates (such as to 4.0) should not be done lightly, 
and should only be done for things like large or particularly 
incompatible database file format changes, such as the 2.x to 3.x 
update was, or for large or incompatible API changes, either way 
things that users could have no excuse not to notice.


So I agree with Richard's current versioning plan concerning this.

-- Darren Duncan

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Rick Langschultz]

I was wondering what would constitute the creation of SQLite 4.0?

Since the VDBE is being revamped I would consider this a pretty big  
revamp of the SQLite code. I am looking forward to testing this out  
new engine out.


I also wanted to know what the difference between stack based and  
register based is. Unfortunately, i do not mess with a lot of C / C++  
code in my work. So I was a little curious.


On Jan 13, 2008, at 7:41 PM, D. Richard Hipp wrote:



On Jan 13, 2008, at 7:53 PM, Gerry Snyder wrote:


Joe Wilson wrote:

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:

There are people on this mailing list (ex: Joe Wilson) who appear  
to read every line of every change that we make to SQLite, within  
minutes of making them, and complain if we so much as misspell a  
word in a comment.  And I haven't heard a peep from Joe


Wow - what prompted that dig against me?


I took it as a kidding compliment. I saw an invisible smiley after  
his comment.


Your contributions to SQLite are appreciated by all of us.



Exactly.  My remark was meant to indicate that I am amazed at how
much attention you pay to the code, not any kind of a dig.  Sorry for
the confusion.  You are an important member of the quality assurance
team and I hope that you will continue in that role.


D. Richard Hipp
[EMAIL PROTECTED]




-
To unsubscribe, send email to [EMAIL PROTECTED]
-





smime.p7s
Description: S/MIME cryptographic signature

Re: [sqlite] Next Version of SQLite

[D. Richard Hipp]

On Jan 13, 2008, at 7:53 PM, Gerry Snyder wrote:


Joe Wilson wrote:

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:

There are people on this mailing list (ex: Joe Wilson) who appear  
to read every line of every change that we make to SQLite, within  
minutes of making them, and complain if we so much as misspell a  
word in a comment.  And I haven't heard a peep from Joe


Wow - what prompted that dig against me?


I took it as a kidding compliment. I saw an invisible smiley after  
his comment.


Your contributions to SQLite are appreciated by all of us.



Exactly.  My remark was meant to indicate that I am amazed at how
much attention you pay to the code, not any kind of a dig.  Sorry for
the confusion.  You are an important member of the quality assurance
team and I hope that you will continue in that role.


D. Richard Hipp
[EMAIL PROTECTED]




-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[P Kishor]

On 1/13/08, Joe Wilson <[EMAIL PROTECTED]> wrote:
> --- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:
> > There are people on this mailing list (ex: Joe Wilson) who appear
> > to read every line of every change that we make to SQLite, within
> > minutes of making them, and complain if we so much as misspell a
> > word in a comment.  And I haven't heard a peep from Joe
>
> Wow - what prompted that dig against me?
>
> Fixing a spelling mistake in a comment adds no value to the software.
> None of those spelling mistake tickets are my doing.
>
> The majority of the tickets I've reported were legitimate problems
> in the code - and arguing the case for some of these bugs isn't
> exactly the easiest thing. But hey, it's cool. I won't file any
> more tickets.
>


honestly, coming from Richard, that is most definitely a compliment to
an "old friend" indicative of the value of your contribution.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Gerry Snyder]

Joe Wilson wrote:

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:
  
There are people on this mailing list (ex: Joe Wilson) who appear 
to read every line of every change that we make to SQLite, within 
minutes of making them, and complain if we so much as misspell a 
word in a comment.  And I haven't heard a peep from Joe 



Wow - what prompted that dig against me?


I took it as a kidding compliment. I saw an invisible smiley after his 
comment.


Your contributions to SQLite are appreciated by all of us.

Please continue.

Gerry



-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Joe Wilson]

--- "D. Richard Hipp" <[EMAIL PROTECTED]> wrote:
> There are people on this mailing list (ex: Joe Wilson) who appear 
> to read every line of every change that we make to SQLite, within 
> minutes of making them, and complain if we so much as misspell a 
> word in a comment.  And I haven't heard a peep from Joe 

Wow - what prompted that dig against me?

Fixing a spelling mistake in a comment adds no value to the software. 
None of those spelling mistake tickets are my doing. 

The majority of the tickets I've reported were legitimate problems 
in the code - and arguing the case for some of these bugs isn't 
exactly the easiest thing. But hey, it's cool. I won't file any 
more tickets.



  

Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ 


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[D. Richard Hipp]

On Jan 13, 2008, at 11:40 AM, Marco Bambini wrote:


What will be the main benefits of the new virtual machine?



Optimizations such as common subexpression elimination
and moving subexpressions outside of inner loops will become
much easier.  The code generator will, in general, be easier to
work on and less error prone.  An entire class of errors (stack
overflow) such as the recent ticket #2832 (which could cause
database corruption) will become impossible since the VM will
no longer have a stack to overflow.


D. Richard Hipp
[EMAIL PROTECTED]




-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[D. Richard Hipp]

On Jan 13, 2008, at 11:31 AM, Rich Shepard wrote:


On Sun, 13 Jan 2008, Darren Duncan wrote:

I would think something like that is worthy of a 3.6.0 version  
number. Not just a minor version increase that would be more  
suitable for minor

changes or bugfixes.


  I agree with Darren that massive changes to the core of the  
system should
be reflected by a major version increase (to 4.0); at a minimum to  
a minor
version increease (to 3.6). A version number change from 3.5.4 to  
3.5.5
tells folks that it's a minor bug fix or simple adjustment, not a  
wholesale

rewrite of the system's core.



There are no user-visible changes to the interface.  The version numbers
in SQLite reflect user-visible changes only.

Well, there is one minor user-visible change.  The output of EXPLAIN now
has 7 columns whereas it used to contain only 5.  But the output of  
EXPLAIN

changes from point release to point release all the time anyway, so I do
not consider this something worth bumping a version number.

I do not expect significant instability with the next release.  I  
want to gain

some experience with the new software before I recommend it for millions
of deployments.  But it should be solid and stable as soon as it is  
released.
For that matter, the current code in CVS (which is well into the  
conversion
to a register machine) has not been giving any problems.  There are  
people
on this mailing list (ex: Joe Wilson) who appear to read every line  
of every
change that we make to SQLite, within minutes of making them, and  
complain
if we so much as misspell a word in a comment.  And I haven't heard a  
peep
from Joe or anybody else, so I'm thinking the code is still working  
correctly

for everybody despite the massive changes that have already gone in.
If you find that the current code in CVS gives problems, or if you see
significant problems emerge as we get closer to releasing 3.5.5, then
maybe we might consider calling it 3.6.0.  But I do not anticipate any
serious problems.  You should not underestimate the level of detail to
which we test SQLite and the thoroughness of the test suite.  Not much
is likely to slip through the cracks.


D. Richard Hipp
[EMAIL PROTECTED]




-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Marco Bambini]

What will be the main benefits of the new virtual machine?
I mean, it will be just faster or there will be other improvements in  
the library?


---
Marco Bambini
http://www.sqlabs.net
http://www.sqlabs.net/blog/
http://www.sqlabs.net/realsqlserver/



On Jan 13, 2008, at 3:07 AM, D. Richard Hipp wrote:



On Jan 12, 2008, at 7:55 PM, Shawn Wilsher wrote:


Hey all,

I was wondering when you plan on releasing the next version of  
SQLite.
 Mozilla is currently using 3.5.4, but that does not include some  
OS/2

fixes that were checked in after the release of 3.5.4.  Instead of
patching our local copy of sqlite, I'd like to use a release version,
but at the same time do not want to delay this fix to our OS/2 users
very long.  The specific checkins we are looking at are 4646, 4647,
and 4648.



In case you haven't been watching the timeline
(http://www.sqlite.org/cvstrac/timeline) we are in the middle
of some major changes. The virtual machine inside of SQLite
is being transformed from a stack-based machine into a
register-based machine.  The whole virtual machine and
the code generator is being rewritten.  Slowly.  Piece by
piece.  I haven't done an overall line change count yet, but
we are looking at some pretty serious code churn.  3.5.4 to
3.5.5 is likely to be the biggest single change in the history
of SQLite.

So you might not want to release product with 3.5.5
embedded.  All the regression tests pass, but still

If you like, we can set up a special Mozilla branch off
of 3.5.4 that includes the OS/2 fixes.

On the other hand, if this is not for a release, but rather
for general development work, then please build and test
with the latest code from CVS.  (This applies to *everybody*
not just Mozilla.)  Please report any problems.  The test
suite for SQLite is very thorough, but I have found that users
can be very creative in stressing SQLite in ways that I would
have never imagined, and have not developed tests for.


D. Richard Hipp
[EMAIL PROTECTED]




-- 
---

To unsubscribe, send email to [EMAIL PROTECTED]
-- 
---





-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Rich Shepard]

On Sun, 13 Jan 2008, Darren Duncan wrote:

I would think something like that is worthy of a 3.6.0 version number. 
Not just a minor version increase that would be more suitable for minor

changes or bugfixes.


  I agree with Darren that massive changes to the core of the system should
be reflected by a major version increase (to 4.0); at a minimum to a minor
version increease (to 3.6). A version number change from 3.5.4 to 3.5.5
tells folks that it's a minor bug fix or simple adjustment, not a wholesale
rewrite of the system's core.

Rich

--
Richard B. Shepard, Ph.D.   |  IntegrityCredibility
Applied Ecosystem Services, Inc.|Innovation
 Voice: 503-667-4517  Fax: 503-667-8863

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Darren Duncan]

At 9:07 PM -0500 1/12/08, D. Richard Hipp wrote:

In case you haven't been watching the timeline
(http://www.sqlite.org/cvstrac/timeline) we are in the middle
of some major changes. The virtual machine inside of SQLite
is being transformed from a stack-based machine into a
register-based machine.  The whole virtual machine and
the code generator is being rewritten.  Slowly.  Piece by
piece.  I haven't done an overall line change count yet, but
we are looking at some pretty serious code churn.  3.5.4 to
3.5.5 is likely to be the biggest single change in the history
of SQLite.


I would think something like that is worthy of a 3.6.0 version 
number.  Not just a minor version increase that would be more 
suitable for minor changes or bugfixes.  On the other hand, I can 
understand if you save the larger increments for changes that are 
user-visible and not just plumbing. -- Darren Duncan


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[Shawn Wilsher]

On Jan 12, 2008 9:07 PM, D. Richard Hipp <[EMAIL PROTECTED]> wrote:
> In case you haven't been watching the timeline
> (http://www.sqlite.org/cvstrac/timeline) we are in the middle
> of some major changes. The virtual machine inside of SQLite
> is being transformed from a stack-based machine into a
> register-based machine.  The whole virtual machine and
> the code generator is being rewritten.  Slowly.  Piece by
> piece.  I haven't done an overall line change count yet, but
> we are looking at some pretty serious code churn.  3.5.4 to
> 3.5.5 is likely to be the biggest single change in the history
> of SQLite.
Out of curiosity, why so many changes for a point release?

> If you like, we can set up a special Mozilla branch off
> of 3.5.4 that includes the OS/2 fixes.
That'd be awesome if we could get a 3.5.4.1 type of thing going.  I'm
very much against patching sqlite locally in our tree (harder to
upgrade).

Cheers,

Shawn Wilsher

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Next Version of SQLite

[D. Richard Hipp]

On Jan 12, 2008, at 7:55 PM, Shawn Wilsher wrote:


Hey all,

I was wondering when you plan on releasing the next version of SQLite.
 Mozilla is currently using 3.5.4, but that does not include some OS/2
fixes that were checked in after the release of 3.5.4.  Instead of
patching our local copy of sqlite, I'd like to use a release version,
but at the same time do not want to delay this fix to our OS/2 users
very long.  The specific checkins we are looking at are 4646, 4647,
and 4648.



In case you haven't been watching the timeline
(http://www.sqlite.org/cvstrac/timeline) we are in the middle
of some major changes. The virtual machine inside of SQLite
is being transformed from a stack-based machine into a
register-based machine.  The whole virtual machine and
the code generator is being rewritten.  Slowly.  Piece by
piece.  I haven't done an overall line change count yet, but
we are looking at some pretty serious code churn.  3.5.4 to
3.5.5 is likely to be the biggest single change in the history
of SQLite.

So you might not want to release product with 3.5.5
embedded.  All the regression tests pass, but still

If you like, we can set up a special Mozilla branch off
of 3.5.4 that includes the OS/2 fixes.

On the other hand, if this is not for a release, but rather
for general development work, then please build and test
with the latest code from CVS.  (This applies to *everybody*
not just Mozilla.)  Please report any problems.  The test
suite for SQLite is very thorough, but I have found that users
can be very creative in stressing SQLite in ways that I would
have never imagined, and have not developed tests for.


D. Richard Hipp
[EMAIL PROTECTED]




-
To unsubscribe, send email to [EMAIL PROTECTED]
-