Re: [sqlite] search time in FTS3 tables sometimes very long

2007-12-04 Thread Scott Hess
2007/12/4 Ingo Godau-Gellert <[EMAIL PROTECTED]>: > What is really strange is that FTS3 search phrases like > SELECT referenzcode FROM volltext where volltext match ('installation > manual') are performed really fast within some milliseconds, independent > to the search phrase. > But in general I a

Re: [sqlite] fts table insert performance

2007-12-04 Thread Scott Hess
Sorry for the delayed response. Was waiting for time to dig into this a little. Nov 18, 2007 2:05 AM Wang Yun <[EMAIL PROTECTED]>: > I insert rfc txt files into a full text search table, 4119 txt files are > 188MB totally. After insert, database file is 443MB. > Logic is below, it's not the real

Re: [sqlite] FTS Usage

2007-11-29 Thread Scott Hess
Also http://code.google.com/apis/gears/api_database.html#sqlite_fts might add some information. I have an item on my todo list which goes something like "Refactor the FTS docs on the wiki". Unfortunately, it's been on my todo list for a couple months, now, with little progress. -scott On Nov 2

Re: [sqlite] Re: building a custom DBD::SQLite

2007-11-24 Thread Scott Hess
On Nov 18, 2007 3:28 PM, P Kishor <[EMAIL PROTECTED]> wrote: > Following those pretty much to the t, I almost got everything working. > Except, I got the following during make > > ../sqlite-3.5.2/ext/fts1/fts1.c:7:2: error: #error fts1 has a design > flaw and has been deprecated. > make: *** [fts1.

Re: [sqlite] Request for help with the SQLite Website

2007-11-16 Thread Scott Hess
On Nov 15, 2007 8:20 AM, Dennis Cote <[EMAIL PROTECTED]> wrote: > When I saw the Developers link I thought it was a link to the bios and > pictures etc. for you and the other developers that had been requested > as an addition to the new website. I never even clicked it to see what > was there (whi

Re: [sqlite] Request for help with the SQLite Website

2007-11-14 Thread Scott Hess
te: > There seems to be no simple solution to that problem of fonts not > scaling precisely other than to use images for the captions. If someone > has one, let us know. > > > Scott Hess wrote: > > I notice that in Firefox on Linux with a maximized window on a > > 16

Re: [sqlite] BLOB data performance?

2007-11-14 Thread Scott Hess
Note that mod-11 will give you 11 items, 0..10! Squid (and Apache's mod_proxy) use a system something like: - hash the URL. - hex-encode or base64-encode the hash. - peel off 2 or 3 characters at a time for as many directories deep as you want. - the remaining characters are the filename in t

Re: [sqlite] Request for help with the SQLite Website

2007-11-14 Thread Scott Hess
I notice that in Firefox on Linux with a maximized window on a 1600x1200 screen, the "Support" link in the navbar wraps around. The navbar still looks nice, but since it's only half the width of my screen, it shouldn't need to wrap. It also happens with narrower browser windows, I'm guessing most

Re: [sqlite] Proposed sqlite3_initialize() interface

2007-10-30 Thread Scott Hess
On Oct 30, 2007 7:18 AM, <[EMAIL PROTECTED]> wrote: > This is still just an idea. If you think that adding a new > required sqlite3_initialize() interface would cause serious > hardship for your use of SQLite, please speak up now. I think this would cause some hardship for dynamically-loaded lib

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-18 Thread Scott Hess
On 10/17/07, Uma Krishnan <[EMAIL PROTECTED]> wrote: > Yes. Makes sense (not to cache query results for embedded apps). > So what is cached. Just dirty pages? or are raw tables cached when > queried? SQLite implements a tables and indices as btrees over a pager layer. The pager layer caches pages.

Re: [sqlite] HELP WITH SQLITE INTERNALS - VDBE and Virtual tables

2007-10-17 Thread Scott Hess
On 10/17/07, Trevor Talbot <[EMAIL PROTECTED]> wrote: > On 10/17/07, Uma Krishnan <[EMAIL PROTECTED]> wrote: > > One other question, when a query is issued, does SQLite cache the results, > > so that future queries can be processed off the cache (I think not) > > Like the "query cache" in some oth

Re: [sqlite] In-memory database: manually modifying ROWID value increases memory.

2007-10-17 Thread Scott Hess
Could it be that you're seeing the btree optimization around in-order insertion? From btree.c: #ifndef SQLITE_OMIT_QUICKBALANCE /* ** A special case: If a new entry has just been inserted into a ** table (that is, a btree with integer keys and all data at the leaves) ** and the new entry

Re: [sqlite] Accessing external applications from within SQLite triggers

2007-10-13 Thread Scott Hess
Your trigger can record the change notification in a separate table. So long as the system processing the change notifications cleans them up as it goes, this can be reasonably efficient. -scott On 10/13/07, Vladimir Stokic <[EMAIL PROTECTED]> wrote: > > Spot on! I could monitor the change of th

Re: [sqlite] Accessing external applications from within SQLite triggers

2007-10-12 Thread Scott Hess
Triggers won't help, either, because triggers run in the sqlite3 handle which, um, triggers. So if app A makes a change, the trigger runs in app A, but not in app B. You could have a custom function which the trigger invokes to send an IPC over to app B. -scott On 10/12/07, Vladimir Stokic <[E

Re: [sqlite] Making BEGIN IMMEDIATE the default.

2007-10-11 Thread Scott Hess
The attached patch seems to do it. The thinking-out-loud patch in my earlier email wasn't right (I'd kept an interim edit from the PRAGMA-based approach). I can't think of a reason to have EXCLUSIVE as the default. -scott On 10/10/07, Scott Hess <[EMAIL PROTECTED]>

Re: [sqlite] Making BEGIN IMMEDIATE the default.

2007-10-10 Thread Scott Hess
On 10/10/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > "Scott Hess" <[EMAIL PROTECTED]> wrote: > > We've just had a bit of discussion on the Google Gears team about some > > cases where failure of an UPDATE/DELETE/INSERT while within a > > tran

Re: [sqlite] Making BEGIN IMMEDIATE the default.

2007-10-10 Thread Scott Hess
. > > A pragma that could configure the default begin > "deffered/immediate/exclusive" would be nice :) > > Ken > > > Scott Hess <[EMAIL PROTECTED]> wrote: To clarify, this is for Google Gears, a > JavaScript library which > includes a Database compon

Re: [sqlite] Making BEGIN IMMEDIATE the default.

2007-10-10 Thread Scott Hess
Clarify^2: I'm suggesting for our use of SQLite in Google Gears. NOT for SQLite itself. Though Ken's suggestion of a PRAGMA might be interesting for SQLite core... -scott On 10/10/07, Scott Hess <[EMAIL PROTECTED]> wrote: > To clarify, this is for Google Gears, a JavaS

Re: [sqlite] Making BEGIN IMMEDIATE the default.

2007-10-10 Thread Scott Hess
n the appropriate places. This is a little bit of a departure from using SQLite in an embedded environment. -scott On 10/10/07, John Stanton <[EMAIL PROTECTED]> wrote: > If you are going to use BEGIN IMMEDIATE why not just enclose the > transaction in some form of lock like a mute

[sqlite] Making BEGIN IMMEDIATE the default.

2007-10-10 Thread Scott Hess
We've just had a bit of discussion on the Google Gears team about some cases where failure of an UPDATE/DELETE/INSERT while within a transaction is unexpected. Well, that and that when you're multi-threaded you can hit some hard-to-understand cases. One suggestion was to use BEGIN IMMEDIATE for e

Re: [sqlite] Feature request - Tcl variables as "value-list"s

2007-10-05 Thread Scott Hess
7, Andy Goth <[EMAIL PROTECTED]> wrote: > On Fri, 5 Oct 2007 09:41:27 -0700, Scott Hess wrote > > On 10/5/07, Andy Goth <[EMAIL PROTECTED]> wrote: > > > proc sql_expand {varname} { > > >upvar 1 $varname var > > >set result [list] > > >

Re: [sqlite] Feature request - Tcl variables as "value-list"s

2007-10-05 Thread Scott Hess
You really should be using an SQLite-specific quote function somewhere. But ... I don't see one in there (I'd have expected it to be something like [db quote $arg]). You could work around it by doing something like [db eval {select quote($arg)}], but that feels clunky. The quoting you're using w

Re: [sqlite] Opinions about per-row tokenizers for fts?

2007-09-19 Thread Scott Hess
On 9/19/07, Ralf Junker <[EMAIL PROTECTED]> wrote: > >Regarding per-row versus per-column tokenizers, your suggesting to > >have something like 'content_kr TOKENIZER icu_kr' for each variant is > >reasonable, but I'm not certain what gain it would have over simply > >having separate tables for each

Re: [sqlite] Opinions about per-row tokenizers for fts?

2007-09-18 Thread Scott Hess
but that seems like a project for some future quarter. I think doing anything automagic with query tokenization is a challenge, because queries tend much more towards sentence fragments. -scott On 9/18/07, Ralf Junker <[EMAIL PROTECTED]> wrote: > Hello Scott Hess, > > >In the in

[sqlite] Re: Opinions about per-row tokenizers for fts?

2007-09-18 Thread Scott Hess
t;DEFAULT" (or other suggestive word TBD), you're indicating that you want the ability to customize the tokenizer per row. -scott On 9/17/07, Scott Hess <[EMAIL PROTECTED]> wrote: > As part of doing internationalization work on Gears, it has been > determined that it is unlikely tha

[sqlite] Opinions about per-row tokenizers for fts?

2007-09-17 Thread Scott Hess
As part of doing internationalization work on Gears, it has been determined that it is unlikely that you can just define a global tokenizer that will work for everything. Instead, in some cases you may need to use a specific tokenizer, based on the content being tokenized, or the source of the con

Re: [sqlite] HELP!!!! LINKING AND LOADING FTS - on Linux/Ubuntu

2007-09-17 Thread Scott Hess
To help isolate if it's actually a library path problem, you might try using the full path. So, instead of: select load_extension('fts2'); do: select load_extension('/path/to/libfts2.so'); Modified for Windows as appropriate (sorry, I mostly use Linux and static linking). If that works,

Re: [sqlite] Adding additional operators to FTS3

2007-09-14 Thread Scott Hess
[Just some background, while I'm thinking about it.] Google has placeholder syntax, if you search for 'full * search', the first hit is a Wikipedia article on 'full text search'. It's not the same, but it is in a similar ballpark. For instance, you might have 'full */10 search' (Google doesn't,

[sqlite] Flipping fts1/2 to fts3.

2007-09-13 Thread Scott Hess
After being out of the office for a bit, I came back and thought "My to-do list is too long." So instead of working on the hardest bit, I decided to see how many pieces of low-hanging fruit I could knock off... So today I'm getting rid of all these to-do items around fts1/2 rowid breakage and fts

Re: [sqlite] Can I simulate a COMMIT?

2007-09-08 Thread Scott Hess
In general, you have to do something like: - for newly-created files, create them during the transaction. If the transaction rolls back, delete the file. If the transaction commits, leave them alone. - for deleted files, leave them in place. If the transaction commits, delete the files. If

Re: [sqlite] SQL approach to Import only new items, delete other items

2007-09-06 Thread Scott Hess
You could invert your solution. Create a temporary table which contains all of the existing keys, and every time you insert a new item, delete that item's key from the temporary table. At the end, do something like 'DELETE FROM main_table WHERE key IN (SELECT key FROM tmp_table)'. -scott On 9/6

Re: [sqlite] Regarding FTS1

2007-09-04 Thread Scott Hess
On 9/3/07, Babu, Lokesh <[EMAIL PROTECTED]> wrote: > One more question, As you said, Full text builds an index of data, so > I hope you have done some memory analysis too, could you please tell > me the memory usage based on your analysis. That's really something that you'll wan to do with your sp

Re: [sqlite] Regarding FTS1

2007-09-03 Thread Scott Hess
On 9/2/07, Babu, Lokesh <[EMAIL PROTECTED]> wrote: > Does anyone help me out, What is FTS1, How to use FTS1, If any sample > programs to use FTS1 and understand better. The "fts" modules are "fulltext search" modules for SQLite. "Fulltext search" meaning that it builds an index based on terms in

Re: [sqlite] rowid versus docid for fts3.

2007-08-31 Thread Scott Hess
Unfortunately, the reason fts2 couldn't be "fixed" was because you can't perform the necessary ALTER TABLE if the column you're adding is a primary key. Since the only alternative would be to build a new table and copy everything over, it seemed more reasonable to just let the app developer do tha

[sqlite] rowid versus docid for fts3.

2007-08-30 Thread Scott Hess
ext/fts3.c in the current code fixes the fts2-vs-vacuum problem by adding "docid INTEGER PRIMARY KEY" to the %_content table. This becomes an alias for rowid, and thus causes vacuum to not renumber rowids. It is safe to add that column because the other columns in %_content are constructed such t

Re: [sqlite] GUID/UUID in sqlite.

2007-08-29 Thread Scott Hess
In either case, if you use the UUID as a primary key in more than one table, you should consider having one table to convert the UUID to a 64-bit id, and use that as the primary keys on the other tables. If you have UUID as a primary key, your table will have 2 b-trees, one for the index of UUID t

Re: [sqlite] FTS2 suggestion

2007-08-29 Thread Scott Hess
a suspicion that this kind of index requires a materially different model than fts has been using, which might encourage it to be a completely different virtual table. -scott On 8/29/07, Scott Hess <[EMAIL PROTECTED]> wrote: > A primary constraint of the porter algorithm in fts is

Re: [sqlite] FTS2 suggestion

2007-08-29 Thread Scott Hess
fts2 experience, it will probably need 2 or 3 weeks beyond that to really settle into a usable state. -scott On 8/29/07, brian kruse <[EMAIL PROTECTED]> wrote: > On 8/24/07, Scott Hess <[EMAIL PROTECTED]> wrote: > > > > My current focus for the next generation is in

Re: [sqlite] FTS2 suggestion

2007-08-29 Thread Scott Hess
conjunction > to snowball/potter). Which would be a good n-gram algorithm to implement. > > Finally, what's the rationale in having sqlite's own search. Why not use > something like luceneC? > > Thanks in advance > > Uma > > Scott Hess <[EMAIL PROTECTED]&g

Re: [sqlite] FTS2 suggestion

2007-08-24 Thread Scott Hess
advance > > "Cesar D. Rodas" <[EMAIL PROTECTED]> wrote: > On 23/08/07, Scott Hess wrote: > > On 8/20/07, Cesar D. Rodas wrote: > > > As I know ( I can be wrong ) SQLite Full Text Search is only match with > > > hole > > > words right

Re: [sqlite] FTS2 suggestion

2007-08-23 Thread Scott Hess
focus. -scott On 8/23/07, Cesar D. Rodas <[EMAIL PROTECTED]> wrote: > On 23/08/07, Scott Hess <[EMAIL PROTECTED]> wrote: > > On 8/20/07, Cesar D. Rodas <[EMAIL PROTECTED]> wrote: > > > As I know ( I can be wrong ) SQLite Full Text Search is only match with &

Re: [sqlite] FTS2 suggestion

2007-08-23 Thread Scott Hess
On 8/20/07, Cesar D. Rodas <[EMAIL PROTECTED]> wrote: > As I know ( I can be wrong ) SQLite Full Text Search is only match with hole > words right? It could not be > And also no FT extension to db ( as far I know) is miss spell tolerant, Yes, fts is matching exactly. There is some primitive suppo

Re: [sqlite] A Question About Creating and Accessing a SQLite Database in a RAM Drive

2007-08-21 Thread Scott Hess
If you're already willing to use a ram drive, you should benchmark SQLite against a disk-backed file, with 'PRAGMA synchronous = off;'. For most modern operating systems, this shouldn't provide significantly different performance than using a RAM disk. Note that you said 'Data persistence is not r

Re: [sqlite] why doesn't this work? (fts rowids)

2007-08-15 Thread Scott Hess
fts1 and fts2 have a design flaw which assumes semantics for the standard SQLite rowid which aren't actually provided by SQLite across calls to VACUUM. fts3 will fix this, either making the rowid be a persistent idenfier across VACUUM, or by exposing a new id (or docid) column which operates as a

Re: [sqlite] why doesn't this work? (fts rowids)

2007-08-15 Thread Scott Hess
At this time I do not plan to provide a way to distinguish "INTEGER PRIMARY KEY AUTOINCREMENT" and "INTEGER PRIMARY KEY" for fts3. fts is in the business of doing a fulltext index, not enforcing constraints! If you require that level of operation, the appropriate solution would be to have a parall

Re: [sqlite] ALTER TABLE and INTEGER PRIMARY KEY.

2007-08-15 Thread Scott Hess
+1 for fts3 or fts2_1 :-) > > --- > We're Hiring! Seeking a passionate developer to join our team building > products. Position is in the Washington D.C. metro area. If interested > contact [EMAIL PROTECTED] > > -Original Message- > From: Scott Hess [

Re: [sqlite] PRAGMA writable_schema=ON;

2007-08-15 Thread Scott Hess
On 8/15/07, Joe Wilson <[EMAIL PROTECTED]> wrote: > If you find a way to get sqlite3 to re-parse the schema after your direct > sqlite_master change, please post it to the list. I don't think it can > be done without modifying the code or making a new connection. You could probably manage it by do

Re: [sqlite] ALTER TABLE and INTEGER PRIMARY KEY.

2007-08-14 Thread Scott Hess
On 8/14/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > "Scott Hess" <[EMAIL PROTECTED]> wrote: > > I was getting ready to checkin the rowid-versus-fts2 fix, and wanted > > to add one last bit, to upgrade older tables. > > > > Unfortunately, co

[sqlite] ALTER TABLE and INTEGER PRIMARY KEY.

2007-08-14 Thread Scott Hess
I was getting ready to checkin the rowid-versus-fts2 fix, and wanted to add one last bit, to upgrade older tables. Unfortunately, code of the form: ALTER TABLE x_segments ADD id INTEGER PRIMARY KEY; is documented as not supported. http://www.sqlite.org/lang_altertable.html . As far as I can

Re: [sqlite] shared cache/ test_server.c

2007-07-27 Thread Scott Hess
On 7/27/07, John Stanton <[EMAIL PROTECTED]> wrote: > Scott Hess wrote: > > On 7/26/07, Richard Klein <[EMAIL PROTECTED]> wrote: > >>According to the Mozilla article referenced above, it's even worse than > >>that: *All* cache pages, dirty or not, are

Re: [sqlite] shared cache/ test_server.c

2007-07-26 Thread Scott Hess
On 7/26/07, Richard Klein <[EMAIL PROTECTED]> wrote: > According to the Mozilla article referenced above, it's even worse than > that: *All* cache pages, dirty or not, are freed at the end of *every* > transaction, even if the transaction consisted of only read operations. I believe this is no lo

Re: [sqlite] Re: autoincrement and fts2?

2007-07-21 Thread Scott Hess
ncerned with a good fix than a fast fix. That and I have other demands on my time. Probably will be fixed this coming week. -scott On 7/21/07, Adam Megacz <[EMAIL PROTECTED]> wrote: "Scott Hess" <[EMAIL PROTECTED]> writes: > In fts tables all columns other than rowid are

Re: [sqlite] SQLite Version 3.4.1

2007-07-20 Thread Scott Hess
[Note that you may need to reload or shift-reload the page. I checked three times across the day for the new content, and was just about to send drh a notice that it wasn't there, when I tried a shift-reload!] On 7/20/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: SQLite Version 3.4.1 is now

Re: [sqlite] quickest way to duplicate table data?

2007-07-20 Thread Scott Hess
In past non-SQLite contexts, I've found that the following usually works very well, without needing snapshot capability: 1. copy the db file. 2. lock the db. 3. copy the db file. 4. unlock the db. The first copy hopefully pulls the entire file into memory buffers, making the next copy very effic

Re: [sqlite] Prepared Statement (select * from x where y in ());

2007-07-18 Thread Scott Hess
Long ago and far away, I build a database abstraction layer which used ?@ for this. So you'd say something like: stmt = prepare("select * from table where xyz in (?@)"); bind_array(stmt, 0, arrayRef); The library would take the array, quote each element, and separate them with commas. It w

Re: [sqlite] autoincrement and fts2?

2007-07-18 Thread Scott Hess
on is to add code to fts2 and fts1 to upgrade tables, and put a prominent disclaimer somewhere. -scott On 7/17/07, Scott Hess <[EMAIL PROTECTED]> wrote: [Forwarding gist of an offline conversation with Joe.] Looks about like what my patch looks like. Needs to additionally handle %_seg

Re: [sqlite] autoincrement and fts2?

2007-07-17 Thread Scott Hess
me something. Tell me what you think. --- Scott Hess <[EMAIL PROTECTED]> wrote: > I've updated the bug with an example of how this breaks fts tables > (fts1 or fts2). I'm thinking on the problem. > http://www.sqlite.org/cvstrac/tktview?tn=2510 > > Summary: In sqlite 3.4

Re: [sqlite] autoincrement and fts2?

2007-07-17 Thread Scott Hess
You can, and I'm working on a patch to do this to see how it might look. There's the question of how to handle existing tables. -scott On 7/17/07, Chris Wedgwood <[EMAIL PROTECTED]> wrote: On Tue, Jul 17, 2007 at 09:37:43AM -0700, Scott Hess wrote: > Summary: In sqlite

Re: [sqlite] autoincrement and fts2?

2007-07-17 Thread Scott Hess
7;ll try to add more constraints to the summary today, -scott On 7/17/07, Scott Hess <[EMAIL PROTECTED]> wrote: WTH! Wow, this is a very unexpected change. I must have not been paying attention at some point. -scott On 7/17/07, Ralf Junker <[EMAIL PROTECTED]> wrote: > &g

Re: [sqlite] autoincrement and fts2?

2007-07-17 Thread Scott Hess
WTH! Wow, this is a very unexpected change. I must have not been paying attention at some point. -scott On 7/17/07, Ralf Junker <[EMAIL PROTECTED]> wrote: >>The standard way to have non-TEXT information associated with rows in >>an fts table would be a separate table which joins with the ft

Re: [sqlite] autoincrement and fts2?

2007-07-17 Thread Scott Hess
On 7/16/07, Adam Megacz <[EMAIL PROTECTED]> wrote: Is there any way to use a INTEGER PRIMARY KEY AUTOINCREMENT on a table that has FTS2? Specifying it in the obvious manner looks like it works, but the column just ends up with nulls in it. In fts tables all columns other than rowid are of type

Re: [sqlite] Unexpected "no such table" error.

2007-07-11 Thread Scott Hess
On 7/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: "Scott Hess" <[EMAIL PROTECTED]> wrote: > do_test test-1.1 { > execsql {PRAGMA encoding} > > sqlite3 db2 test.db > execsql {CREATE TABLE t (id int)} db2 > db2 close > > #execsql {SEL

Re: [sqlite] Unexpected "no such table" error.

2007-07-11 Thread Scott Hess
Aha, yes, that does sound like exactly it. Thanks! On 7/11/07, Joe Wilson <[EMAIL PROTECTED]> wrote: --- Scott Hess <[EMAIL PROTECTED]> wrote: > do_test test-1.1 { > execsql {PRAGMA encoding} > > sqlite3 db2 test.db > execsql {CREATE TABLE t (id int)} db2 &

[sqlite] Unexpected "no such table" error.

2007-07-11 Thread Scott Hess
do_test test-1.1 { execsql {PRAGMA encoding} sqlite3 db2 test.db execsql {CREATE TABLE t (id int)} db2 db2 close #execsql {SELECT * FROM sqlite_master} catchsql {SELECT * FROM t} } {1 {no such table: t}} Looks like the schema info is obviously being cached. The first PRAGMA is to make su

Re: [sqlite] How to store 128 bit values

2007-07-11 Thread Scott Hess
On 7/11/07, Steve Krulewitz <[EMAIL PROTECTED]> wrote: In the application I am working on (Songbird), we have a simple two table schema representing the tracks in your music collection and the properties on those tracks. The keys used are all UUIDs (128 bit number) which we are currently storing

Re: [sqlite] FTS and upgrades

2007-07-10 Thread Scott Hess
On 7/10/07, Joe Wilson <[EMAIL PROTECTED]> wrote: --- [EMAIL PROTECTED] wrote: > "Scott Hess" <[EMAIL PROTECTED]> wrote: > > If you have an fts1 table f, you could drop f_term and f_content, but > > you won't be able to drop f itself. So you wo

Re: [sqlite] FTS and upgrades

2007-07-10 Thread Scott Hess
27;re Hiring! Seeking a passionate developer to join our team building products. Position is in the Washington D.C. metro area. If interested contact [EMAIL PROTECTED] On 7/9/07, Scott Hess <[EMAIL PROTECTED]> wrote: > > If you have not compiled in fts1, and try to drop an fts1 table,

Re: [sqlite] FTS and upgrades

2007-07-09 Thread Scott Hess
If you have not compiled in fts1, and try to drop an fts1 table, you'll get an error. I don't think you'll get a crash, but sqlite will simply not know how to deal with the table. I can think of two ways to deal with this. When builidng a new version, you could just leave the fts1 code in place

[sqlite] Global namespace versus fts2.c.

2007-06-27 Thread Scott Hess
In the misty mists of time, we made the decision to minimize namespace pollution by trying to keep everything in a single file. fts2.c is almost to 6000 lines, almost as big as btree.c. Does anyone have strong thoughts about the negatives of breaking things up? Offhand, the major bits of distin

Re: [sqlite] Full Text Search

2007-06-25 Thread Scott Hess
On 6/10/07, Mark Gilbert <[EMAIL PROTECTED]> wrote: I have just started learning about Full Text search in SQlite, and I have some questions Sorry for the delayed response, I just noticed this email waiting for an answer, and nobody has ... 1) With the Amalgamated Sqlite, I guess to enable F

[sqlite] Loose auto_vacuum.

2007-06-25 Thread Scott Hess
Recently, various incremental vacuum features have been added, so you can do: -- Turn on incremental vacuum. PRAGMA auto_vacuum = incremental; -- Release 100 pages from the freelist, or all if there are fewer. PRAGMA incremental_vacuum(100); -- Return the number of items on the freelist.

Re: [sqlite] Alternative index methods (hash key)

2007-06-20 Thread Scott Hess
On 6/20/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: How difficult do you think it would be to support an alternative method of indexing within SQLite specifically to support O(1) retrieval of the rowid for a table, and then potentially O(1) retrieval of the row data for a table, when in-ord

Re: [sqlite] Optimization of equality comparison when NULL involved

2007-06-18 Thread Scott Hess
On 6/18/07, Sean Cunningham <[EMAIL PROTECTED]> wrote: There was talk in the mailing list a while back about creating a new operator that would act as a superset of '==' which would treat NULL==NULL as True. I have seen this in some other database. Anybody know if this is on the roadmap? It wo

Re: [sqlite] Optimization of equality comparison when NULL involved

2007-06-15 Thread Scott Hess
You can use something like: select tableA.path, tableA.value from tableA,tableB where tableA.path=tableB.path and (tableA.value=tableB.value or (tableA.value IS NULL AND tableB.value IS NULL)); It's possible that won't use an index, either, due to the OR, in which case you could try a union betw

Re: [sqlite] Is it a bug?

2007-06-14 Thread Scott Hess
On 6/14/07, Joe Wilson <[EMAIL PROTECTED]> wrote: You can't infer a function's return type from its arguments. Take the hypothetical function FOO(x). If I pass it a number, it will return the number spelled out as TEXT, but if I pass it a BLOB it will return its length*PI as a FLOAT. Would it b

Re: [sqlite] Ranking in fts.

2007-06-10 Thread Scott Hess
On 6/10/07, Ralf Junker <[EMAIL PROTECTED]> wrote: Hello Scott Hess, >I've lined up some time to work on fts, again, which means fts3. One >thing I'd like to include would be to order doclists by some baked-in >ranking. The idea is to sort to most important items

[sqlite] Ranking in fts.

2007-06-08 Thread Scott Hess
I've lined up some time to work on fts, again, which means fts3. One thing I'd like to include would be to order doclists by some baked-in ranking. The idea is to sort to most important items to the front of the list, and then you can do queries which limit the number of hits and can thus be sig

Re: [sqlite] FTS-2

2007-06-07 Thread Scott Hess
On 6/7/07, John Stanton <[EMAIL PROTECTED]> wrote: I have just started to use FTS2 and it is working well but I would like to ask any other users if they have had good or bad experiences and why they would use FTS2 rather than FTS1. The software is new and I have not seen any feedback at this st

Re: [sqlite] how to change SQLite column definition

2007-05-28 Thread Scott Hess
In case it wasn't obvious, the "more complicated way" would probably be something like: BEGIN; ALTER TABLE RENAME MyTable TO MyTableOld; CREATE TABLE MyTable ( SameColumn INTEGER, NewColumn TEXT ); INSERT INTO MyTable SELECT SameColumn, OldColumn FROM MyTableOld; DROP TABLE MyTableOld; COMMIT;

Re: [sqlite] Prefix searching for fts2.

2007-05-01 Thread Scott Hess
On 5/1/07, Samuel R. Neff <[EMAIL PROTECTED]> wrote: One suggestion though, instead of (or in addition to) using '*' as the prefix operator perhaps '%' would be more appropriate in order to be closer to the LIKE operator. Hmm. I was mainly just doing what other groups appear to do (Lucene, MYS

[sqlite] Prefix searching for fts2.

2007-05-01 Thread Scott Hess
I just finished ( http://www.sqlite.org/cvstrac/chngview?cn=3893 ) checking in a string of changes to fts2.c to provide prefix search. This works like: CREATE VIRTUAL TABLE t USING fts2(c); INSERT INTO t (c) VALUES ('This is a test'); INSERT INTO t (c) VALUES ('That was a test'); INSERT INTO

Re: [sqlite] Insert order maintained?

2007-04-17 Thread Scott Hess
Additionally, note that if you use ORDER BY, and it _is_ in the indicated order already, then sqlite will optimize the ORDER BY away entirely. So use ORDER BY. -scott On 4/17/07, Andrew Finkenstadt <[EMAIL PROTECTED]> wrote: The order of the rows returned by a select that does not have an ORD

Re: [sqlite] Canonical way to get the CVS as of a particular release.

2007-04-13 Thread Scott Hess
On 4/13/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: "Scott Hess" <[EMAIL PROTECTED]> wrote: > My assumption would be that you'd use 'cvs log VERSION' to find the > datestamp where the VERSION file was updated, then check the tree out > using -D

[sqlite] Canonical way to get the CVS as of a particular release.

2007-04-13 Thread Scott Hess
My assumption would be that you'd use 'cvs log VERSION' to find the datestamp where the VERSION file was updated, then check the tree out using -D. Right? Thanks, scott - To unsubscribe, send email to [EMAIL PROTECTED]

Re: [sqlite] Re: FTS does not support REPLACE

2007-04-11 Thread Scott Hess
On 4/11/07, Dan Kennedy <[EMAIL PROTECTED]> wrote: On Tue, 2007-04-10 at 09:26 -0700, Scott Hess wrote: > On 4/10/07, Dan Kennedy <[EMAIL PROTECTED]> wrote: > > I checked the code and conflict handling mechanisms (OR ERROR, > > OR ABORT, OR REPLACE) do

Re: [sqlite] Re: FTS does not support REPLACE

2007-04-10 Thread Scott Hess
On 4/10/07, Dan Kennedy <[EMAIL PROTECTED]> wrote: I checked the code and conflict handling mechanisms (OR ERROR, OR ABORT, OR REPLACE) do not apply to virtual tables. Something to think about anyhow... Do we want conflict handling for FTS (and other virtual modules)? I think OR REPLACE woul

Re: [sqlite] Re: FTS does not support REPLACE

2007-04-09 Thread Scott Hess
Thanks for the concise report. I'm going to take a look at this today, to see if it's an fts1/2 problem. If it's _not_, I'll still look at it, but perhaps with less eventual success :-). -scott On 4/9/07, Paul Quinn <[EMAIL PROTECTED]> wrote: Very simple to replicate: CREATE VIRTUAL TABLE

Re: [sqlite] Index creation

2007-04-02 Thread Scott Hess
An interesting approach would be to use some sort of async I/O facility to implement read-ahead. Short of that, I have found that in some cases, on some operating systems, implementing explicit read-ahead buffering for fts2 segment merges improves performance when the disk caches are cold. Linux

Re: [sqlite] API enhancement

2007-03-19 Thread Scott Hess
I don't see how your modified version is any better than just putting the sqlite3_bind_int() inside the loop. You've superficially lifted some code out of the loop, but sqlite3_step() is going to have to go through and bind all of the "pointer bound" variables in your suggested API, so it won't s

Re: [sqlite] sqlite Performance

2007-03-15 Thread Scott Hess
Are you using explicit transactions at all? If not, as a quick test, put the _entire_ job in a transaction and see what happens. -scott On 3/15/07, Ken <[EMAIL PROTECTED]> wrote: To answer your question: Yes I can use a flat file at this stage, but eventually it needs to be imported into so

Re: [sqlite] MingW32 help wanted

2007-03-13 Thread Scott Hess
I took Makefile.linux-gcc, and made the obvious changes (there are mingw lines all over in there). EXE = .exe, SO =dll, SHPREFIX = , [that was nothing for that setting], and TCC, AR, and RANLIB set to the path to the appropriate commands from mingw. I should warn that I haven't actually built us

Re: [sqlite] FTS: index only, no text storage - Was: [sqlite] FTS: Custom Tokenizer / Stop Words

2007-03-13 Thread Scott Hess
On 3/13/07, Ralf Junker <[EMAIL PROTECTED]> wrote: Scott Hess wrote: >Keeping track of that information would probably double the >size of the index. With your estimate, the SQLite full text index (without document storage) would still take up only 50% of the documents' si

Re: [sqlite] FTS: index only, no text storage - Was: [sqlite] FTS: Custom Tokenizer / Stop Words

2007-03-13 Thread Scott Hess
On 3/13/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Ion Silvestru <[EMAIL PROTECTED]> wrote: > To Ralf: > >As a side effect, the offsets() and snippet() functions stopped working, > >as they seem to rely on the presence of the full document text in the > >current implementation. > > Did you

Re: [sqlite] journal - "Unable to open the database file"

2007-03-10 Thread Scott Hess
On 3/10/07, Joe Wilson <[EMAIL PROTECTED]> wrote: There's also this - CreateFileTransacted(): http://msdn2.microsoft.com/en-us/library/aa363859.aspx Doesn't this require Vista? -scott - To unsubscribe, send email to

Re: [sqlite] SQLite 2 to SQLite 3 - Varchar sizes

2007-03-09 Thread Scott Hess
On 3/9/07, Mitchell Vincent <[EMAIL PROTECTED]> wrote: I'm looking into ways of changing the schema type name when I do my SQLite2->SQLite3 conversion. Most likely I'll have to pipe the .dump from the SQLite2 db through a program to replace 'varchar' with 'text'... I'm working on that now! You

Re: [sqlite] Is there an inverse for .import?

2007-03-09 Thread Scott Hess
e of .dump would be .read. -scott On 3/9/07, Scott Hess <[EMAIL PROTECTED]> wrote: On 3/9/07, Gunnar Roth <[EMAIL PROTECTED]> wrote: > Anderson, James H (IT) schrieb: > > I need to "export" a table to a file in the same format as used by > > .import, but I do

Re: [sqlite] Is there an inverse for .import?

2007-03-09 Thread Scott Hess
On 3/9/07, Gunnar Roth <[EMAIL PROTECTED]> wrote: Anderson, James H (IT) schrieb: > I need to "export" a table to a file in the same format as used by > .import, but I don't see any such cmd. Am I missing something, or does > such a cmd just not exist? Maybe its dumb but its called .dump ;-) S

Re: [sqlite] Re: Re: how to get field names of empty tables ?

2007-02-23 Thread Scott Hess
I think only the dot commands are special (.help, etc). Everything else is fair game. Best reference for what you can feed a prepare or exec is http://www.sqlite.org/lang.html . -scott On 2/23/07, Igor Tandetnik <[EMAIL PROTECTED]> wrote: Stef Mientki <[EMAIL PROTECTED]> wrote: > Igor Tandet

[sqlite] Re: Bug using fts tables in attached databases.

2007-02-06 Thread Scott Hess
Fix is in http://www.sqlite.org/cvstrac/tktview?tn=2219 [This covers both fts1 and fts2.] -scott On 2/6/07, Scott Hess <[EMAIL PROTECTED]> wrote: http://www.sqlite.org/cvstrac/tktview?tn=2219 ATTACH DATABASE 'test2.db' AS two; CREATE VIRTUAL TABLE two.t2 USING fts2

[sqlite] Bug using fts tables in attached databases.

2007-02-06 Thread Scott Hess
http://www.sqlite.org/cvstrac/tktview?tn=2219 ATTACH DATABASE 'test2.db' AS two; CREATE VIRTUAL TABLE two.t2 USING fts2(content); will put t2_content, t2_segments, and t2_segdir in database 'main' rather than database 'two'. In many cases everything will appear to work, because the tables wi

<    1   2   3   4   5   >