Re: [Zope-dev] SQLite as a Light DB component for Zope and Python

[William Trenker]

At 10:45 AM 4/2/02 -0600, "Ross J. Reedstrom" <[EMAIL PROTECTED]> wrote:
>Scale, as in multiuser? Hardly at all: it's an SQL library that accesses a 
>single, textbased, flatfile for the entire database.

>As a lightweight replacement for gadfly, it looks like it might be pretty 
>good.

Exactly.  I'm proposing this as a lightweight component, just as you say, 
not as a replacement for something like MySQL or PostgreSQL.  What I have 
in mind is a small, simple, built-in SQL engine that could be used as a 
step up from something like TinyTablePlus in Zope and be compact enough to 
even be considered as a module candidate for the standard Python library.

>Note that the scripting language of choice of the author seems to be Tcl, 
>rather than Python. This probably explains the 'everything is a string' 
>approach :-)

If you look into the C interface you will find it is almost trivial to 
build a Python extension module and "bring" SQLite into the realm of Python 
scripting.  SQLite also has it's own C interface for adding expression 
functions and aggregates to the SQL syntax.  I expect this could be hooked 
into Python through the Python extension interface as a callback.

Again, the "everything is a string" approach fits in with the idea of 
"simple, lightweight".  Mind you, SQLite supports SQL expressions so it can 
do things like "SELECT * FROM my_table WHERE my_field / 2 > 23.8", or 
"INSERT INTO my_table VALUES (100 / 30.0)".  SQLite does implicit 
conversions, as required.

>The speed comparisions with PostgreSQL are very much an apples vs. fish 
>sort of thing: the pgsql server was not tuned _at all_, and does a whole 
>lot more that was never tested, such as multi-user writer access.

You could be right.  I'm not an expert with PostgreSQL so I can't 
comment.  But, at the risk of being repetitious, I'm thinking 
"lightweight".  My intent is to propose a small relational tool  that 
doesn't impose a significant overhead on the host system and might be 
simple enough that the Zope and Python developers would consider SQLite, 
together with a Python DB API and Zope DA, for their standard libraries.

Thanks for commenting,
Bill



"The commandments of the LORD are right, bringing joy to the heart. The 
commands of the LORD are clear, giving insight to life . . . For this is 
the love of God, that we keep His commandments. And His commandments are 
not burdensome." (Psalm 19:8, 1John 
5:3)torahteacher.com



---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.343 / Virus Database: 190 - Release Date: 3/22/02

Re: [Zope-dev] SQLite as a Light DB component for Zope and Python

[Ross J. Reedstrom]

On Tue, Apr 02, 2002 at 10:02:22AM -0500, Eron Lloyd wrote:
> Very interesting. I think I've read about this somewhere before. The
> claim of "4x faster than PostgreSQL" raised my brow. It is true that
> Gadfly is becoming quite stale, and only supports a *very* limited
> subset of SQL. It also would be nice to see something a bit more robust
> than just shelve in the Standard Library. Gadfly is ok to begin a
> prototype in, but frustrating once you quickly hit a glass ceiling in
> functionality. I'm interested in seeing how well it can scale, versus
> Postgres, however. Any experience?

Scale, as in multiuser? Hardly at all: it's an SQL library that accesses
a single, textbased, flatfile for the entire database. From the FAQ,
multiple readers are allowed (on Unix), but the entire file (yes,
that's the whole database, not a single table) is locked for one backend
to write.

As a lightweight replacement for gadfly, it looks like it might be pretty
good. Note that the scripting language of choice of the author seems
to be Tcl, rather than Python. This probably explains the 'everything
is a string' approach :-) The speed comparisions with PostgreSQL are
very much an apples vs. fish sort of thing: the pgsql server was not
tuned _at all_, and does a whole lot more that was never tested, such
as multi-user writer access.

Ross

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] SQLite as a Light DB component for Zope and Python

[Eron Lloyd]

Very interesting. I think I've read about this somewhere before. The
claim of "4x faster than PostgreSQL" raised my brow. It is true that
Gadfly is becoming quite stale, and only supports a *very* limited
subset of SQL. It also would be nice to see something a bit more robust
than just shelve in the Standard Library. Gadfly is ok to begin a
prototype in, but frustrating once you quickly hit a glass ceiling in
functionality. I'm interested in seeing how well it can scale, versus
Postgres, however. Any experience?

Thanks for the info,

Eron Lloyd

On Mon, 2002-04-01 at 20:54, William Trenker wrote:
> I have noticed on the DB lists lately some concern about the future of 
> Gadfly.  I have been investigating a marvelous little open-source, no 
> copyright, SQL engine called 
> SQLite: An SQL Database Engine 
> In A C Library.  I am quite experienced with Python, reasonably experienced 
> with Zope but a greenhorn at extending Python yet I had a crude but working 
> Python extension module for SQLite up and running in 2 days (most of that 
> time figuring out the Python extension conventions).  I think Python needs 
> a lightweight SQL engine as a standard module, and I think this would be a 
> good Zope product candidate as well.  I'm proposing SQLite as that 
> engine.  Here is the developer's feature list, taken from the link given above:
> 
> Implements a large subset of SQL92.
> A complete database (with multiple tables and indices) is stored in a 
> single disk file.
> Atomic commit and rollback protect data integrity.
> Small memory footprint: less than 20K lines of C code.
> Four times faster than PostgreSQL. Twice as fast as SQLite 1.0.
> Very simple C/C++ interface requires the use of only three functions and 
> one opaque structure.
> TCL bindings included.
> A TCL-based test suite provides near 100% code coverage.
> Self-contained: no external dependencies.
> Built and tested under Linux and Win2K.
> Sources are uncopyrighted. Use for any purpose.
> The SQLite source code is 35% comment. These comments are another important 
> source of information.
> 
> The author, D. Richard Hipp, is 
> a computer science Ph.D. who knows his stuff.  This is not green software, 
> it is well designed and tested.  It was first released in May 2000 and is 
> very actively updated and supported.
> 
> Thanks for listening.
> 
> Bill Trenker
> Internet Applications Developer
> Kelowna, BC, Canada
> 
> 
> 
> "The commandments of the LORD are right, bringing joy to the heart. The 
> commands of the LORD are clear, giving insight to life . . . For this is 
> the love of God, that we keep His commandments. And His commandments are 
> not burdensome." (Psalm 19:8, 1John 
> 5:3)torahteacher.com
> 
> 
> 

> 
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.343 / Virus Database: 190 - Release Date: 3/22/02


---
[This E-mail scanned for viruses by Declude Virus]


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

[Zope-dev] SQLite as a Light DB component for Zope and Python

[William Trenker]

I have noticed on the DB lists lately some concern about the future of
Gadfly.  I have been investigating a marvelous little open-source,
no copyright, SQL engine called
SQLite: An SQL
Database Engine In A C Library.  I am quite experienced with
Python, reasonably experienced with Zope but a greenhorn at extending
Python yet I had a crude but working Python extension module for SQLite
up and running in 2 days (most of that time figuring out the Python
extension conventions).  I think Python needs a lightweight SQL
engine as a standard module, and I think this would be a good Zope
product candidate as well.  I'm proposing SQLite as that
engine.  Here is the developer's feature list, taken from the link
given above:
Implements a large subset of SQL92.
A complete database (with multiple tables and indices) is stored in a
single disk file.
Atomic commit and rollback protect data integrity.
Small memory footprint: less than 20K lines of C code.
Four times faster than PostgreSQL. Twice as fast as SQLite 1.0.
Very simple C/C++ interface requires the use of only three functions and
one opaque structure.
TCL bindings included.
A TCL-based test suite provides near 100% code coverage.
Self-contained: no external dependencies.
Built and tested under Linux and Win2K.
Sources are uncopyrighted. Use for any purpose.
The SQLite source code is 35% comment. These comments are another
important source of information.
The author, D. Richard
Hipp, is a computer science Ph.D.
who knows his stuff.  This is not green software, it is well
designed and tested.  It was first released in May 2000 and is very
actively updated and supported.
Thanks for listening.
Bill Trenker
Internet Applications Developer
Kelowna, BC, Canada


"The commandments of the LORD are right, bringing joy
to the heart. The commands of the LORD are clear, giving insight to life
. . . For this is the love of God, that we keep His commandments. And His
commandments are not burdensome." (Psalm 19:8, 1John
5:3)   
torahteacher.com




---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.343 / Virus Database: 190 - Release Date: 3/22/02