Re: [sqlite] DeviceSQL

[steveweick]

Good  idea... I'll pass it along to the right folks. Meanwhile, if anyone has
further questions or comments, please feel free to write me here (if they
think the group would be interested) or at [EMAIL PROTECTED]

Steve

I would like to recommend that Encriq create a forum or mailing list of 
their own for those who are interesting in learning more.  For me, what 
might be an interesting product is quickly being overshadowed by this 
thread.


-- 
View this message in context: 
http://www.nabble.com/DeviceSQL-tp14297970p14329799.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] DeviceSQL

[steveweick]

Hi James,

You raise some interesting points.  There is nothing secret about the
benchmarks.  We will make the code that was used to run benchmarks available
to anyone who wants to see it and verify results. If you want to find a
third party to verify, be my guest. The benchmark report goes into some
depth on the design and rationale for the benchmark.  Frankly, as much as I
like the idea about taking DeviceSQL open source, you don't need to do so,
just to verify performance claims.  

Do you need to read the code to verify reliability as your next few
sentences seems to imply? For that to be true, the reader would have to be
able to spot bugs through inspection.  While that is certainly one way to
spot bugs, I seriously doubt that any shop would rely on code inspection,
when millions of dollars of potential recall costs are on the line.

In fact the SQLite marketing does not rely on code inspection as its
argument for why the code is reliable. Check it out. 

All of that said, I do admire the elegance of the SQLite code.  It makes
entertaining reading.  Unfortunately elegance does not translate into
performance or reliability.

Regards,

Steve

James Steward-2 wrote:
> 
> steveweick wrote:
>> Richard has it right this time.  Today DeviceSQL uses no SQLite code. One
>> of
>> the things we might consider is bolting the SQLite parser/front end to
>> our
>> table engine, in theory to get the both worlds.  Just an idea at the
>> moment.
>>   
> Such an interesting discussion to be following.  I must say though, it 
> seems DeviceSQL has opened the door to speculation due to 
> unsubstantiated claims in advertising, as far as I see it.  IMHO, so 
> long as there is no independent, unbiased, side by side test results 
> presented somewhere by some reliable source, there will always be some 
> room for "ifs" and "buts" by both sides.
> 
> Maybe DeviceSQL should go open source, so the public can judge for them 
> selves the qualities of the two products.  There would still be money to 
> be made from paid support.  Who knows, both parties could benefit, and 
> customers too.  At least there'd be a clearer view of the pros and cons. 
> 
> There is something to be said for a product being open source, that is 
> the code is scrutinized by the world.  Closed shop code can possibly 
> still be very good, but without seeing it, how would we know?  Reminds 
> me of a story about a cat: dead or alive, we won't know until we open 
> the box it's in, and prior to that, is it only half dead?
> 
> One only has to look at the MSDN code examples to see the ugliness of 
> closed source  code development...(sorry Bill)
> 
> JS.
> 
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/DeviceSQL-tp14297970p14328763.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] DeviceSQL

[steveweick]

Richard has it right this time.  Today DeviceSQL uses no SQLite code. One of
the things we might consider is bolting the SQLite parser/front end to our
table engine, in theory to get the both worlds.  Just an idea at the moment.

Steve



D. Richard Hipp wrote:
> 
> Joe Wilson <[EMAIL PROTECTED]> wrote:
>> Be careful about speculative comments.
>> 
>> For all anyone knows, said product could use SQLite internally with 
>> a couple of proprietary optimizations here and there that may make it
>> faster in specific cases. 
>> 
>> The sqlite public domain license would allow that sort of thing.
>> 
> 
> Because of the radically different architectures of SQLite
> and DeviceSQL, it seems unlikely that they share a common
> core.  Though, I suppose anything is possible...
> 
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
> 
> 
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/DeviceSQL-tp14297970p14327325.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] DeviceSQL

[steveweick]

As shown another thread, Richard has his facts wrong.  See
http://www.nabble.com/Improving-performance-of-SQLite.-Anyone-heard-of-DeviceSQL--to14280006.html#a14317195

Steve

A. Pagaltzis wrote:
> 
> * John Stanton <[EMAIL PROTECTED]> [2007-12-12 17:55]:
>> In general claims of "20x" or even "5x" imply either serious
>> deficiencies in the compared product or a generous dose of
>> snake oil in the challenger.
> 
> Depends. The outline given by Dr. Hipp about the product’s
> features may the claim quite plausible, because you pay a hefty
> cut in features and reliability in exchange for a very large
> increase in speed; a price that many may well find unacceptable.
> (It is, after all, easy, as they say, to compute the wrong answer
> in constant time.)
> 
> Regards,
> -- 
> Aristotle Pagaltzis // 
> 
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/DeviceSQL-tp14297970p14327322.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] DeviceSQL

[steveweick]

I hope this is not a double post, but this was answered in another thread. 
See 
http://www.nabble.com/Improving-performance-of-SQLite.-Anyone-heard-of-DeviceSQL--to14280006.html#a14317195



D. Richard Hipp wrote:
> 
> John Stanton <[EMAIL PROTECTED]> wrote:
>> I received an email promoting a DeviceSQL web presentation.  It 
>> specifically targets Sqlite and promises 5X performance.
>> 
> 
> If you view their web presentation and/or try out Encirq's
> products, I would be very interested to hear your impressions.
> Even better would be if you could blog about it.
> 
> Encirq has for years been running Google Adsense ads claiming
> to be 20x faster than SQLite.  (Dunno why they have now reduced
> that claim to 5x faster.)  But I have never yet seen an
> independent confirmation of this.  Nor even have I been able
> to find anybody who is actually using DeviceSQL in a product.
> Web searches turn up nothing but marketing literature coming
> directly or indirectly from Encirq.  Some independent analysis
> (regardless of whether it is favorable or unfavorable to SQLite)
> would be appreciated.
> 
> My understanding of DeviceSQL is:
> 
>*  It is NOT transactional.  There is no such thing as ROLLBACK.
>*  If you lose power during a write, your database is toast.
>*  If your database schema changes, you have to recompile
>   your application.
>*  The database file format changes depending on the schema.
>*  DeviceSQL is not a general-purpose database engine.  You
>   compile SQL statements into C code on a development
>   workstation, then compile the C code for your embedded
>   device.
> 
> I can imagine circumstances where the DeviceSQL approach,
> while much less flexible and forgiving than SQLite, might
> be a better way to go, depending on what you are trying to
> do.  But I have not gotten good vibes from Encirq as a 
> company.  And I have no idea how reliable the DeviceSQL 
> product is.  I would really appreciate your thoughts on 
> that subject.
> 
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
> 
> 
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/DeviceSQL-tp14297970p14327299.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Improving performance of SQLite. Anyone heard of DeviceSQL?

[steveweick]

oops, fingers are moving faster than the brain :-)  of course, you are right,
Dennis.

Steve

Dennis Cote wrote:
> 
> steveweick wrote:
>> the tests were done
>> using Windows XP SP2 and Linux FC5 on a 3GHz P4 with 1MB
>>   
> 
> That must be very slow. ;-)
> 
> I'm sure you meant 1GB for windows XP.
> 
> Dennis Cote
> 
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Improving-performance-of-SQLite.-Anyone-heard-of-DeviceSQL--tp14280006p14325082.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

RE: [sqlite] Improving performance of SQLite. Anyone heard of DeviceSQL?

[steveweick]

Hi Sam,

re your points below:

1. I think I said "innovative", not "revolutionary". The scheme involves
using "dirty bits" rather than a log to record the transactional state of a
page.

2. We plan on publishing all the details of the benchmarks in a few days.
But to answer your question about platforms and tests, the tests were done
using Windows XP SP2 and Linux FC5 on a 3GHz P4 with 1MB, Linux 2.4.31-a9-3
on a 200MHz ARM9 with 64MB, and Freescale Embedded Linux 2.6.16.11 on a 466
MHz 5200 with 256MB. The tests were done with relatively simple tables that
ranged in size from 5000 to 1M records. Inserts, deletes, updates, and
various selects were tested against the SQLite prepare/execute interface and
the DeviceSQL compiled and interpreted interfaces.

3. I'm not surprised to hear that SQLite is substantially faster than MSSQL.
We haven't tested MSSQL, but it makes sense, because both SQLite and
DeviceSQL do not pay the MSSQL price of client server interfaces. That said,
the real question comes down whether SQLite will meet your application
performance needs.  If it does, great. By contrast, DeviceSQL customers have
very stringent performance requirements (some even have a "performance
budget") and often view performance as a critical element in achieving
competitive advantage. If your application doesn't fit that mold, then
SQLite is the right choice for you. SQLite performance is poor compared with
that of DeviceSQL, not poor in general. Our customers have confirmed that  a
number of times.

4. I'm not a big fan of DeviceSQL marketing to date either. I think that's
going to change soon... watch this space.


Best regards,

Steve


Steve,

I found the information you posted to be a good contrast and would love to
learn more, but you didn't include any technical details.  You said you have
atomic commits without a rollback journal and instead use some revolutionary
new way of doing commits.  You said DeviceSQL performs significantly faster
than SQLite, can you show what tests you ran, on what platforms, and your
exact results?  I was particularly skeptical when you said "SQLite
performance, while poor on larger PCs" because in our own testing we've
found SQLite to be 4 times faster than MSSQL after we migrated.  If you're
finding SQLite performance to be poor at all, then most likely your
developers are doing something wrong in testing SQLite which of course would
invalidate your comparison to DeviceSQL.

In short, can you provide more details?  Personally I don't install demo
software just to learn what I should be able to get from the company website
(which I would hope is truly technical details, not just marketing fluff).

I tried searching online for information about DeviceSQL but pretty much
everything I found was regurgitation of marketing data from your company.
The only really compelling thing I found was this.

http://www.google.com/trends?q=sqlite%2C+devicesql

Best regards,

Sam

-
-- 
View this message in context: 
http://www.nabble.com/Improving-performance-of-SQLite.-Anyone-heard-of-DeviceSQL--tp14280006p14319009.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Improving performance of SQLite. Anyone heard of DeviceSQL?

[steveweick]

oops, I guess I need to get used to this message list protocol.

First let me apologize for letting Richard get me mad. Most of my friends
would describe me as one of the most laid back people they know. Why am I
mad you ask?

We wrote Richard back in August to correct his misstatements then. He chose
to ignore the letter. Moreover he (or anyone) has been able to download our
product with all of its documentation since February or March of this year.
We encourage people to do so, because using the product is far more
convincing and informative than trying to plow through a bunch of marketing
blather.

By the way, I don't know where Richard got the stuff about me leaving the
mailing list... it never happened.

Steve



steveweick wrote:
> 
> 
> 
> D. Richard Hipp wrote:
>> 
>> Ion Silvestru <[EMAIL PROTECTED]> wrote:
>>> >SW: Richard,  We have written to you directly before to ask you to stop
>>> the
>>> >FUD and incorrect statements, and you have chosen to continue. I
>>> suggest you
>>> >not waste everyone's time by circulating deliberately misleading
>>> >information.
>>>
>>> I think you are very aggressive and I think you must apologise to, not
>>> only Richard, but to us (just see previous messages about DeviceSQL,
>>> full of suppositions).
>>>
>> 
>> Thanks for posting, Ion.  I too found Steve's remarks to be
>> rather insolent.  But I was just going to let it go.  Seeing
>> your response was an encouragement to me since it shows me
>> that I am not the only one who feels that way.  Thanks!
>> 
>> Unfortunately, Steve Weick might not see your comment
>> since he appears to have unsubscribed from the mailing list
>> immediately after sending his inflammatory missive.
>> 
>>> 
>>> These were no "FUD and incorrect statements", nor "misleading
>>> information", these were only suppositions, and this is because it's
>>> hard to find real technical information or specifications on DeviceSQL,
>>> only
>>> marketing information. Maybe DeviceSQL is a good product, but absence
>>> of real info and abundance of marketing make us think and suppose
>>> various things (just see previous messages).
>>> 
>>> All of us are waiting for what Richard stated:
>>> "If you view their web presentation and/or try out Encirq's
>>> products, I would be very interested to hear your impressions.
>>> Even better would be if you could blog about it."
>>> 
>>> Even better if all of us can have access to this web presentation, to
>>> find out maybe more technical info about DeviceSQL.
>>> 
>>> Any way, thank you.
>>> 
>> 
>> --
>> D. Richard Hipp <[EMAIL PROTECTED]>
>> 
>> 
>> -
>> To unsubscribe, send email to [EMAIL PROTECTED]
>> -
>> 
>> 
>> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Improving-performance-of-SQLite.-Anyone-heard-of-DeviceSQL--tp14280006p14316335.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Improving performance of SQLite. Anyone heard of DeviceSQL?

[steveweick]

D. Richard Hipp wrote:
> 
> Ion Silvestru <[EMAIL PROTECTED]> wrote:
>> >SW: Richard,  We have written to you directly before to ask you to stop
>> the
>> >FUD and incorrect statements, and you have chosen to continue. I suggest
>> you
>> >not waste everyone's time by circulating deliberately misleading
>> >information.
>>
>> I think you are very aggressive and I think you must apologise to, not
>> only Richard, but to us (just see previous messages about DeviceSQL,
>> full of suppositions).
>>
> 
> Thanks for posting, Ion.  I too found Steve's remarks to be
> rather insolent.  But I was just going to let it go.  Seeing
> your response was an encouragement to me since it shows me
> that I am not the only one who feels that way.  Thanks!
> 
> Unfortunately, Steve Weick might not see your comment
> since he appears to have unsubscribed from the mailing list
> immediately after sending his inflammatory missive.
> 
>> 
>> These were no "FUD and incorrect statements", nor "misleading
>> information", these were only suppositions, and this is because it's
>> hard to find real technical information or specifications on DeviceSQL,
>> only
>> marketing information. Maybe DeviceSQL is a good product, but absence
>> of real info and abundance of marketing make us think and suppose
>> various things (just see previous messages).
>> 
>> All of us are waiting for what Richard stated:
>> "If you view their web presentation and/or try out Encirq's
>> products, I would be very interested to hear your impressions.
>> Even better would be if you could blog about it."
>> 
>> Even better if all of us can have access to this web presentation, to
>> find out maybe more technical info about DeviceSQL.
>> 
>> Any way, thank you.
>> 
> 
> --
> D. Richard Hipp <[EMAIL PROTECTED]>
> 
> 
> -
> To unsubscribe, send email to [EMAIL PROTECTED]
> -
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Improving-performance-of-SQLite.-Anyone-heard-of-DeviceSQL--tp14280006p14316330.html
Sent from the SQLite mailing list archive at Nabble.com.


-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] Improving performance of SQLite. Anyone heard of DeviceSQL?

[steveweick]

This is Steve Weick, CTO & VP Engineering at Encirq Corp., developers and IP
owners of DeviceSQL. I would like to address D. Richard Hipp’s statements.

RDH:"If you view their web presentation and/or try out Encirq's products, I
would be very interested to hear your impressions. Even better would be if
you could blog about it.Encirq has for years been running Google Adsense ads
claiming to be 20x faster than SQLite. (Dunno why they have now reduced that
claim to 5x faster.) But I have never yet seen an independent confirmation
of this. Nor even have I been able
to find anybody who is actually using DeviceSQL in a product. Web searches
turn up nothing but marketing literature coming directly or indirectly from
Encirq. Some independent analysis (regardless of whether it is favorable or
unfavorable to SQLite) would be appreciated."



SW: The DeviceSQL  performance advantage over SQLite has been demonstrated
by running a series of benchmarks with a variety of operations using  Linux
on PCs,  ARM, Freescale, and other processor platforms that are commonly
used in embedded applications. 

In all our benchmarking we attempt to present SQLite capabilities at their
best. So we "tweak" SQLite to use indexes, not scans, in all cases. We also
opt (for fairness) to compare the products using only paged storage with
B-trees like those of SQLite. In many cases our other indexing techniques
are far superior to this approach. We provide detailed benchmark reports as
well as the benchmark code to prospective customers.

 We have seen that SQLite performance, while poor on larger PCs, degrades
significantly on small processors compared with DeviceSQL. We believe that
this is due to the fact that SQLite uses a number of techniques that consume
large amounts of the available CPU capacity, and it is therefore unable to
operate at the flash or disk speed. While SQLite performance has improved in
some areas in the last few releases, we can still show that DeviceSQL is
2-10X faster in all of the interesting cases and 50X faster in one odd case. 

You do not see client listings on our site because our clients believe that
DeviceSQL is part of their competitive advantage and they do not like to
advertise to their competition what they are using. You will however see
outspoken users of DeviceSQL explain why they chose DeviceSQL over SQLite,
if they were not able to make SQLite satisfy their requirements, like
Gemstar-TV Guide.


RDH: "My understanding of DeviceSQL is:


* It is NOT transactional. There is no such thing as ROLLBACK."


SW: This is false. DeviceSQL DOES support transactions and ROLLBACK, just
not in the traditional, resource intensive manner of maintaining a
journaling log. Rather, we use a simple approach which maintains data
integrity, high performance, and small footprint without introducing the
possibility of corrupting the journal.



RDH:"* If you lose power during a write, your database is toast."


SW: Again, not true. DeviceSQL has supported transactions and rollback since
its very first release in 2003 and continues to do so today. Contrary to Mr.
Hipp's assertions, DeviceSQL ensures that writes complete successfully
(ensuring no power outage can cause corruption) before continuing after a
commit. In fact, because of DeviceSQL's novel (and very simple) commit
approach, it is possible to prove that application data is recoverable (this
is quite difficult to do with the logging approach used by SQLite and
important for devices that must handle critically important data). In
addition to fast updates, the DeviceSQL approach yields substantially
shorter boot times after failures. This is often important to consumer
devices where the end user will not tolerate long boot times.



RDH:"* If your database schema changes, you have to recompile your
application."


SW: This is true. DeviceSQL is targeted for embedded applications where
executables change rarely, so schema changes are a big deal, and the clients
do not want to make changes to the schema after production begins. We do
however, offer migration utilities and approaches for doing this if needed.


RDH:"* The database file format changes depending on the schema."


SW: Not sure what this statement is about, although all databases have this
to some extent.


RDH:"* DeviceSQL is not a general-purpose database engine. You compile SQL
statements into C code on a development workstation, then compile the C code
for your embedded device."


SW: Neither is SQLite by this standard. Both products are
application-resident database engines that live in the application's address
space. The question is whether the main use model is compiled SQL versus
interpreted SQL or C APIs. DeviceSQL also supports C query interfaces. This
is rarely an issue in small devices where the database manager is embedded
in the application, and where our compiled language can be used to implement 
application database logic.


RDH:"I can imagine circumstances where the DeviceSQL approach,