Re: [sqlite] The problem with index

2007-06-20 Thread Dan Kennedy
> Another question: Is it correct that virtual tables can be created using > Perl but not Tcl? I don't have a current need (with the possible > exception of FTS1/2, which are already accessible from Tcl), but the > situation seemed curious. Wondering whether there was an undocumented >

Re: [sqlite] Alternative index methods (hash key)

2007-06-20 Thread John Stanton
Andrew Finkenstadt wrote: On 6/20/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: "Scott Hess" <[EMAIL PROTECTED]> wrote: > On 6/20/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: > > How difficult do you think it would be to support an alternative method of > > indexing within SQLite

Re: [sqlite] Alternative index methods (hash key)

2007-06-20 Thread Andrew Finkenstadt
On 6/20/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: "Scott Hess" <[EMAIL PROTECTED]> wrote: > On 6/20/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: > > How difficult do you think it would be to support an alternative method of > > indexing within SQLite specifically to support O(1)

Re: [sqlite] Alternative index methods (hash key)

2007-06-20 Thread drh
"Scott Hess" <[EMAIL PROTECTED]> wrote: > On 6/20/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: > > How difficult do you think it would be to support an alternative method of > > indexing within SQLite specifically to support O(1) retrieval of the rowid > > for a table, and then potentially

Re: [sqlite] Data structure for versioned data

2007-06-20 Thread John Stanton
We perform some versioning by holding column material in XML and using RCS to maintain reverse deltas and versions. Samuel R. Neff wrote: Not specific to SQLite, but we're working on an app that needs to keep versioned data (i.e., the current values plus all previous values). The versioning

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread Martin Jenkins
Joe Wilson wrote: A non-volatile RAM drive is the way to go if you got the bucks. 16 Processor machine ~40Gb ram EMC storage suggests he does. ;) I worked on a project where the end client had Sun kit of this spec, and they claimed the systems cost 7 figures GBP back in 2005. Martin

Re: [sqlite] Alternative index methods (hash key)

2007-06-20 Thread Andrew Finkenstadt
On 6/20/07, Scott Hess <[EMAIL PROTECTED]> wrote: On 6/20/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: > How difficult do you think it would be to support an alternative method of > indexing within SQLite specifically to support O(1) retrieval of the rowid > for a table, and then

Re: [sqlite] Alternative index methods (hash key)

2007-06-20 Thread Scott Hess
On 6/20/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: How difficult do you think it would be to support an alternative method of indexing within SQLite specifically to support O(1) retrieval of the rowid for a table, and then potentially O(1) retrieval of the row data for a table, when

[sqlite] Alternative index methods (hash key)

2007-06-20 Thread Andrew Finkenstadt
How difficult do you think it would be to support an alternative method of indexing within SQLite specifically to support O(1) retrieval of the rowid for a table, and then potentially O(1) retrieval of the row data for a table, when in-order retrieval is undesired? My database design is highly

[sqlite] Data structure for versioned data

2007-06-20 Thread Samuel R. Neff
Not specific to SQLite, but we're working on an app that needs to keep versioned data (i.e., the current values plus all previous values). The versioning is integral to the app so it's more than just an audit trail or history. Can anyone share experiences with the database structure for this

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread Joe Wilson
I assumed he meant a volatile system RAM "drive", as opposed to a non-volatile external RAM drive by his wording. But no point speculating what he meant. A non-volatile RAM drive is the way to go if you got the bucks. --- Ken <[EMAIL PROTECTED]> wrote: > I think the performance of the ram drive

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread Joe Wilson
> mmm, I was thinking that I decrease the cache_size to like 20 when using the > ram drive since I dont need caching anymore then. > > I have inserted more timeing code and I am now convinced I have an IO > problem. When I coax a OS to fully cache my (smaller 40 rows) db file ( > which takes

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread Ken
I understand where you are heading, by putting the entire db on a ram drive. I think the performance of the ram drive (i'm guessing scsi based) will not be as good as physical system ram. But certainly better than the I/o speed of disk. Let us know how it turns out. pompomJuice <[EMAIL

Re: [sqlite] The problem with index

2007-06-20 Thread Gerry Snyder
[EMAIL PROTECTED] wrote: This gives a different answer because the EXCEPT operator makes the rows of the result set unique. So the result set will be: 1 2 111 where as formerly it was 1 1 2 2 111 111 Thank you for the clarification. That is not the behavior I would have expected. In my

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread pompomJuice
mmm, I was thinking that I decrease the cache_size to like 20 when using the ram drive since I dont need caching anymore then. I have inserted more timeing code and I am now convinced I have an IO problem. When I coax a OS to fully cache my (smaller 40 rows) db file ( which takes like 2-3

Re: Re[2]: [sqlite] The problem with index

2007-06-20 Thread Ken
Joe Wilson <[EMAIL PROTECTED]> wrote: --- "Sergey M. Brytsko" wrote: > But what about the following values: > > 1 1 2 2 100 100 100 100 100 100 100 100 100 100 100 111 111 I guess SQLite's query optimizer could take the cardinality of the column into account via its ANALYZE statistics for

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread Ken
The Ram drive is unlikely to work. It will still have the same cache invalidation. You need to get things logically working first. Ram drives are great to help improve performance where seeks are and rotational access requirements dictate. pompomJuice <[EMAIL PROTECTED]> wrote: AArrgh.

Re: [sqlite] The problem with index

2007-06-20 Thread drh
Gerry Snyder <[EMAIL PROTECTED]> wrote: > Igor Tandetnik wrote: > > Sergey M. Brytsko <[EMAIL PROTECTED]> > > wrote: > >> But what about the following values: > >> > >> 1 1 2 2 100 100 100 100 100 100 100 100 100 100 100 111 111 > > > > If you susptect your data is likely to look like this, you

Re: [sqlite] Re: Re[2]: The problem with index

2007-06-20 Thread Gerry Snyder
Igor Tandetnik wrote: Sergey M. Brytsko <[EMAIL PROTECTED]> wrote: But what about the following values: 1 1 2 2 100 100 100 100 100 100 100 100 100 100 100 111 111 If you susptect your data is likely to look like this, you may want to rewrite your query as SELECT BBB FROM XXX WHERE BBB <

Re: [sqlite] Step Query

2007-06-20 Thread Andrew Finkenstadt
On 6/20/07, Dennis Cote <[EMAIL PROTECTED]> wrote: Andrew Finkenstadt wrote: > > > I ended up writing a (multi-thread aware) C++ framework to keep me > out of trouble. In the SQLite namespace I have Is there any chance that your framework is freely licensed open source so others could use it

Re: [sqlite] How do I close the command line window

2007-06-20 Thread Dennis Cote
[EMAIL PROTECTED] wrote: I'm doing a bulk insert by calling sqlite3 -init BulkinsertItems.sql mydatabasefile with the BulkinsertItems.sql file containing: .separator \t .import BulkItems.txt items .quit The command window opens and the import works, but then it does not close again. How can

Re: [sqlite] Step Query

2007-06-20 Thread Dennis Cote
Andrew Finkenstadt wrote: I ended up writing a (multi-thread aware) C++ framework to keep me out of trouble. In the SQLite namespace I have class exception; class database; class connection; class statement; class execution; class query_result; where the ownership model is

Re: Re[2]: [sqlite] The problem with index

2007-06-20 Thread Joe Wilson
--- "Sergey M. Brytsko" wrote: > But what about the following values: > > 1 1 2 2 100 100 100 100 100 100 100 100 100 100 100 111 111 I guess SQLite's query optimizer could take the cardinality of the column into account via its ANALYZE statistics for inequality comparisons. It's just the small

Re: [sqlite] Recommend server for Windows?

2007-06-20 Thread John Stanton
Gilles Ganault wrote: At 20:47 19/06/2007 -0500, John Stanton wrote: Such a server can be made simpler then mine by making it single threaded. Is it publicly available from http://www.viacognis.com? Thanks G. No, but I can give you some code which might help your project. The components

[sqlite] Re: Re[2]: The problem with index

2007-06-20 Thread Igor Tandetnik
Sergey M. Brytsko <[EMAIL PROTECTED]> wrote: But what about the following values: 1 1 2 2 100 100 100 100 100 100 100 100 100 100 100 111 111 If you susptect your data is likely to look like this, you may want to rewrite your query as SELECT BBB FROM XXX WHERE BBB < 100 union all SELECT

Re[2]: [sqlite] The problem with index

2007-06-20 Thread Sergey M. Brytsko
But what about the following values: 1 1 2 2 100 100 100 100 100 100 100 100 100 100 100 111 111 Thanks. -- Sergey Wednesday, June 20, 2007, 3:21:25 PM, wrote: JW> Say you have the following values for BBB: JW> 1 2 3 10 20 30 50 70 80 80 90 100 101 110 110 120 120 150 190 200 JW> How is

Re: [sqlite] The problem with index

2007-06-20 Thread Joe Wilson
--- "Sergey M. Brytsko" wrote: > The problem is the index is NOT used for query: > SELECT BBB FROM XXX WHERE BBB <> 100; > > but in case of query > SELECT BBB FROM XXX WHERE BBB > 100; > all is ok ... > The indices are very important for me, how should I build these queries? Say you have the

[sqlite] Re: The problem with index

2007-06-20 Thread Igor Tandetnik
Sergey M. Brytsko <[EMAIL PROTECTED]> wrote: I have the following DB schema: CREATE TABLE XXX(AAA TEXT, BBB INTEGER); CREATE INDEX AAA_IDX ON XXX(AAA); CREATE INDEX BBB_IDX ON XXX(BBB); SQLite 3.4.0 The problem is the index is NOT used for query: SELECT BBB FROM XXX WHERE BBB <> 100; An

Re: [sqlite] Converting from 2.8.x to 3.x?

2007-06-20 Thread Christian Smith
Gilles Ganault uttered: Hello As we move from a 2.8.x file-based solution to a 3.x c/s solution, we'll have to convert databases from one format to the other. What's the easiest way to do this? sqlite olddb .dump | sqlite3 newdb Thank you G. Christian -- /"\ \ /ASCII

[sqlite] Converting from 2.8.x to 3.x?

2007-06-20 Thread Gilles Ganault
Hello As we move from a 2.8.x file-based solution to a 3.x c/s solution, we'll have to convert databases from one format to the other. What's the easiest way to do this? Thank you G. - To unsubscribe, send email to

RE: [sqlite] More SQLite Misuse, sorted i think

2007-06-20 Thread Dan Kennedy
> Hope that is more clear. Perfectly. I get it now. As you say in the other post, every sqlite call needs to be inside the critical section, including sqlite3_finalize(). Dan. - To unsubscribe, send email to [EMAIL

RE: [sqlite] More SQLite Misuse, sorted i think

2007-06-20 Thread Andre du Plessis
In my sample that I supplied I illustrated how two threads does the following: Lock (Global Critical Section) Queryobject.Prepare (Sqlite3_prepare) QueryObject.Step (Sqlite3_step) QueryObject.Reset (Sqlite3_reset) Unlock QueryObject.Free; (Sqlite3_reset (the missing piece of the puzzle)) In

RE: [sqlite] More SQLite Misuse, sorted i think

2007-06-20 Thread Andre du Plessis
Sorry if I created any confusion there were some code that seemed to have called Sqlite_reset simultaneously from more than one thread, even though the statements were unique for each thread the call to the library was not locked. I know assumptions are bad but I thought that reset on a unique

Re: [sqlite] Cache invalidation after insert statements.

2007-06-20 Thread pompomJuice
AArrgh. That is the one thing that I wont be able to do. It would require a complete system redesign. I can adapt my program easy but now to get it to work in the greater scheme of things would be a nightmare. My current efforts are being focussed into making a ram drive and putting the file in

Re: [sqlite] Proper way to transfer a live sqlite database

2007-06-20 Thread Dan Kennedy
On Tue, 2007-06-19 at 11:51 -0700, Gerry Snyder wrote: > Michael Hooker wrote: > > Many thanks for the explanation Dan. > Ditto the thanks. > > I suspected the purpose of ROLLBACK was as you say, but couldn't see > > why it was used here. You point out the "under the hood" difference > >

Re: [sqlite] Re: How to use pragmas from code?

2007-06-20 Thread Dan Kennedy
On Tue, 2007-06-19 at 15:39 -0400, Igor Tandetnik wrote: > Shane Harrelson > <[EMAIL PROTECTED]> wrote: > > To use pragmas from code, do I simply prepare them as a regular SQL > > statement and then execute them? > > Yes. Another thing to note: Some pragmas take effect during sqlite3_prepare(),