Re: [sqlite] Virtual table used to query big external database

2008-04-03 Thread Scott Hess
You can _generally_ work around this kind of problem using sub-selects. So instead of: SELECT x FROM fts_table WHERE fts_table MATCH "y" OR fts_table MATCH "z"; You might write: SELECT x FROM fts_table WHERE rowid IN (SELECT rowid FROM fts_table WHERE fts_table MATCH "y") OR rowid IN (SELECT

Re: [sqlite] Virtual table used to query big external database

2008-04-02 Thread Mike Owens
PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Ben Harper > > Sent: Tuesday, April 01, 2008 4:53 PM > > To: General Discussion of SQLite Database > > Subject: Re: [sqlite] Virtual table used to query big > > external database > > > > > > Th

Re: [sqlite] Virtual table used to query big external database

2008-04-02 Thread Evans, Mark (Tandem)
[EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Ben Harper > Sent: Tuesday, April 01, 2008 4:53 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Virtual table used to query big > external database > > The only limitation imposed by SQL that I c

Re: [sqlite] Virtual table used to query big external database

2008-04-02 Thread Mike Owens
t; Date: Tue, 1 Apr 2008 20:17:54 -0500> From: [EMAIL PROTECTED]> To: > sqlite-users@sqlite.org> Subject: Re: [sqlite] Virtual table used to query > big external database> > The main reason why SQLite's practical limit is in > the 10s of GBs as> opposed to TBs (theoretic

Re: [sqlite] Virtual table used to query big external database

2008-04-02 Thread Aladdin Lampé
Thank you. That's exactly the mistake I did :-)> Date: Wed, 2 Apr 2008 09:11:44 -0400> From: [EMAIL PROTECTED]> To: sqlite-users@sqlite.org> Subject: Re: [sqlite] Virtual table used to query big external database> > Hi, Aladin,> > Regarding: > "Do you know

Re: [sqlite] Virtual table used to query big external database

2008-04-02 Thread Griggs, Donald
Hi, Aladin, Regarding: "Do you know where I can get the files "fs.c" and "example.c" you mention? [ http://www.ddj.com/database/202802959 ] I've been on the FTP (available from www.ddj.com/code/) but I was not able to locate..." The code appears to be in the December 2007 (0712.zip) file,

Re: [sqlite] Virtual table used to query big external database

2008-04-02 Thread Aladdin Lampé
o: sqlite-users@sqlite.org> Subject: Re: [sqlite] Virtual table used to query big external database> > The main reason why SQLite's practical limit is in the 10s of GBs as> opposed to TBs (theoretical) is due to how it tracks dirty pages. This> is described in the "Appropriat

Re: [sqlite] Virtual table used to query big external database

2008-04-01 Thread Jay A. Kreibich
On Tue, Apr 01, 2008 at 08:17:54PM -0500, Mike Owens scratched on the wall: > The main reason why SQLite's practical limit is in the 10s of GBs as > opposed to TBs (theoretical) is due to how it tracks dirty pages. > [...] > SQLite tracks dirty pages with a bitmap which is > allocated before each

Re: [sqlite] Virtual table used to query big external database

2008-04-01 Thread Mike Owens
The main reason why SQLite's practical limit is in the 10s of GBs as opposed to TBs (theoretical) is due to how it tracks dirty pages. This is described in the "Appropriate Uses" page (http://www.sqlite.org/whentouse.html) but I'll rehash it here for convenience. SQLite tracks dirty pages with a

Re: [sqlite] Virtual table used to query big external database

2008-04-01 Thread Ben Harper
The only limitation imposed by SQL that I can think of would be inefficiencies in its query plan builder. That is the part that figures out how to use the indexes available in the database in order to execute the SQL query most efficiently. So it really depends on what type of SQL queries you are

[sqlite] Virtual table used to query big external database

2008-04-01 Thread Aladdin Lampé
Hi all! Very often, when people ask this list why they have trouble managing in sqlite a "big table" (50 million lines or more than 10 Go), they are told that sqlite is an embedded database and is not meant to be used for very big databases/tables. I'm currently in the process of designing a