Re: [sqlite] Near misses

2009-06-26 Thread Igor Tandetnik
Alberto Simoes wrote: > On Fri, Jun 26, 2009 at 3:00 PM, Igor Tandetnik > wrote: >> Alberto Simoes wrote: >>> SELECT DISTINCT(word) FROM dict WHERE word = "ar" OR word = "ca" OR >>> word LIKE "_car" OR word LIKE "c_r" OR word = "cr" OR word LIKE >>> "_ar" OR word LIKE "ca_r"

Re: [sqlite] (no subject)

2009-06-26 Thread Miroslav Zagorac
> Hello. > > I'm using SQLite with a VB wrapper (dhSQLite) for VB6. > > The following SQL string works fine for putting together a recordset where > the DATE field contains only the date of the last day of each month. > > > SQLString = "SELECT date(Date,'start of month','+1 month','-1

[sqlite] (no subject)

2009-06-26 Thread Rick Ratchford
Hello. I'm using SQLite with a VB wrapper (dhSQLite) for VB6. The following SQL string works fine for putting together a recordset where the DATE field contains only the date of the last day of each month. SQLString = "SELECT date(Date,'start of month','+1 month','-1 day') as Date, " & _

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Jay A. Kreibich
On Fri, Jun 26, 2009 at 05:07:16PM -0400, Greg Morehead scratched on the wall: > > If I close then reopen the database all my memory is recovered. > > Is this by design??? I was intending on keeping a connection open most of > time. Are you sure you're not looking at the page cache?

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Kees Nuyt
On Fri, 26 Jun 2009 17:07:16 -0400, "Greg Morehead" wrote: > >If I close then reopen the database all my memory is recovered. > >Is this by design??? Yes, what you see is probably the page cache. >I was intending on keeping a connection open most of time.

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Greg Morehead
If I close then reopen the database all my memory is recovered. Is this by design??? I was intending on keeping a connection open most of time. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Greg Morehead Sent: Friday,

Re: [sqlite] Near misses

2009-06-26 Thread Jean-Christophe Deschamps
At 13:25 26/06/2009, you wrote: ´¯¯¯ >I am trying to find words in a dictionary stored in sqlite, and trying >a near miss approach. >For that I tried an algorithm to create patterns corresponding to >Levenshtein distance of 1 (edit distance of 1). >That means, one adition, one remotion or one

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Greg Morehead
FYI. I replaced the sqlite3_exec call with sqlite3_prepare_v2, sqlite3_step, sqlite3_finalize. Same results. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org]on Behalf Of Greg Morehead Sent: Friday, June 26, 2009 4:21 PM To: General

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Greg Morehead
Based on the documentation in the link you sent I should only need to call free if there was an error message. But, since there is no harm in calling sqlite3_free on a null pointer I moved it out of the if statement. It had no impact, still leaking like faucet. -Original Message-

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Greg Morehead
I'm watching the heap from the eclipse IDE which connects to the remote debugging qconn service on the target. I've included a screen shot from the last run of the code I included. This was over 1 minute period where onlky 2782 records were written! The problem is much worse on the much

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Ribeiro, Glauber
I took only a quick look, but it seems to me that sqlite3_free is only being called if there is an error. See http://sqlite.org/c3ref/exec.html g -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Greg Morehead Sent: Friday,

Re: [sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread D. Richard Hipp
On Jun 26, 2009, at 3:49 PM, Greg Morehead wrote: > The extremely simple app below leaks. What am I doing wrong?? > Please help. How do you know it is leaking memory? How are you measuring? > > > #include > #include > #include "stdio.h" > #include "sqlite3.h" > #include > > #define

[sqlite] Memory leak with sqlite3_exec on qnx 6.4.1

2009-06-26 Thread Greg Morehead
The extremely simple app below leaks. What am I doing wrong?? Please help. #include #include #include "stdio.h" #include "sqlite3.h" #include #define TFQ_SQL_DB_NAME "/powerblock/datalog/TFQ-test.db" #define _TABLE_NAME "sqltest" int main(int argc, char *argv[]) { char sql[1024];

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Kosenko Max
Well, I understand idea in general and how it works. But as you have described in second part of your letter - this won't help. Even if you will create 100 tables that will save you just 1 step from 5-7 IO steps, but won't make Cache hit ratio significantly higher. And I'm pretty sure that even

Re: [sqlite] How to find the version of the database.

2009-06-26 Thread Martin Engelschalk
Hi, a database file does not have a version. You can access it with different versions of the library. AFAIK there is no way to determine what version of the library created it or which version wrote to it last. Martin Kalyani Phadke wrote: > Is there any way to find the version of SQlite3

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Jay A. Kreibich
On Fri, Jun 26, 2009 at 10:06:48AM -0700, Kosenko Max scratched on the wall: > > > Doug Fajardo wrote: > > No, I admit I haven't tried this under SQLITE. > > > > Whether this approach will help for the specific application will depend > > on data usage patterns, which we haven't delved into for

[sqlite] How to find the version of the database.

2009-06-26 Thread Kalyani Phadke
Is there any way to find the version of SQlite3 database. eg. I have test.DB file . I want to know which SQLite3 version its using ..eg 3.5.4 or 3.6.15? Thanks, -K ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Kosenko Max
Doug Fajardo wrote: > No, I admit I haven't tried this under SQLITE. > > Whether this approach will help for the specific application will depend > on data usage patterns, which we haven't delved into for this application. > Call me simple: since the main issue is degraded performance with

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Douglas E. Fajardo
No, I admit I haven't tried this under SQLITE. Whether this approach will help for the specific application will depend on data usage patterns, which we haven't delved into for this application. Call me simple: since the main issue is degraded performance with larger groupings of data, it

Re: [sqlite] Near misses

2009-06-26 Thread Alberto Simões
On Fri, Jun 26, 2009 at 3:43 PM, Simon Slavin wrote: > > On 26 Jun 2009, at 12:25pm, Alberto Simões wrote: > >> one adition, one remotion or one substitution > > I am always amazed at how well people use English.  For your word > 'remotion' you probably mean 'removal'

Re: [sqlite] Near misses

2009-06-26 Thread Alberto Simões
Hello On Fri, Jun 26, 2009 at 3:00 PM, Igor Tandetnik wrote: > Alberto Simoes wrote: >> For that, my script receives a word (say, 'car') and generated all >> possible additions and remotions, and substitutions: >> >> Additions: _car c_ar ca_r car_ >> Substitutions: _ar c_r

Re: [sqlite] Near misses

2009-06-26 Thread Simon Slavin
On 26 Jun 2009, at 12:25pm, Alberto Simões wrote: > one adition, one remotion or one substitution I am always amazed at how well people use English. For your word 'remotion' you probably mean 'removal' or 'omission'. You have joined the two possibilities together ! > Then, the script

Re: [sqlite] ROWID of 0

2009-06-26 Thread D . Richard Hipp
On Jun 26, 2009, at 12:22 PM, D. Richard Hipp wrote: > > On Jun 26, 2009, at 12:13 PM, Shaun Seckman (Firaxis) wrote: > >> I have a column ("ID") in a table that is the primary key integer >> so it >> should be an alias for ROWID. Is it safe to have a ROWID of 0? > > > A ROWID can have any

Re: [sqlite] ROWID of 0

2009-06-26 Thread Dan
On Jun 26, 2009, at 11:13 PM, Shaun Seckman (Firaxis) wrote: > I have a column ("ID") in a table that is the primary key integer so > it > should be an alias for ROWID. Is it safe to have a ROWID of 0? Yes. > > > > > -Shaun > > > > > > ___ >

Re: [sqlite] ROWID of 0

2009-06-26 Thread D. Richard Hipp
On Jun 26, 2009, at 12:13 PM, Shaun Seckman (Firaxis) wrote: > I have a column ("ID") in a table that is the primary key integer so > it > should be an alias for ROWID. Is it safe to have a ROWID of 0? A ROWID can have any value between -9223372036854775808 and 9223372036854775807,

[sqlite] ROWID of 0

2009-06-26 Thread Shaun Seckman (Firaxis)
I have a column ("ID") in a table that is the primary key integer so it should be an alias for ROWID. Is it safe to have a ROWID of 0? -Shaun ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Kosenko Max
John Stanton-3 wrote: > Why would it not work? It is just adding an extra top level to the > index. A tried and true method. It will work. But won't give performance benefit. And from my undestanding it will even slow down things. You can place parts of index in different DB and on different

[sqlite] How to update field with data in field in many rows?

2009-06-26 Thread Jan Martin
Hi all, my database has a table ExifPhoto with the fields GPSGeometry and Nearby. For each row I need to Update field Nearby with all coordinates (in the table) within a Distance of 1 degree of the coordinates in field GPSGeometry. The sqllite database has the spatial extension from SpatiaLite

Re: [sqlite] Near misses

2009-06-26 Thread Igor Tandetnik
Alberto Simoes wrote: > For that, my script receives a word (say, 'car') and generated all > possible additions and remotions, and substitutions: > > Additions: _car c_ar ca_r car_ > Substitutions: _ar c_r ca_ > remotions: ar cr ca > > Then, the script constructs an SQL query: > > SELECT

Re: [sqlite] SQLite error: cannot commit transaction - SQL statements in progress

2009-06-26 Thread Igor Tandetnik
Manasi Save wrote: > I am working on an application. In my code nowhere I am explicitly > setting AutoCommit = False after any statement. You set AutoCommit = False by executing BEGIN statement. You set it back to True by executing COMMIT, END or ROLLBACK. > But i am getting "SQLite error:

[sqlite] SQLite error: cannot commit transaction - SQL statements in progress

2009-06-26 Thread Manasi Save
Hi All, I am working on an application. In my code nowhere I am explicitly setting AutoCommit = False after any statement. But i am getting "SQLite error: cannot commit transaction - SQL statements in progress" this error. Can anyone provide any input on this case in which all senerio we get

Re: [sqlite] Cross-Compile and Installation of Sqlite

2009-06-26 Thread Ben Atkinson
On Thu, June 25, 2009 at 4:46 PM, Matthew L. Creech wrote: > cd /path/to/sqlite-3.6.15 > ./configure --prefix=/my/target/rootfs --host=arm-unknown-linux-gnueabi > make install > Thanks for the help. Matthew. That was exactly what I needed. With the configure --prefix option , the subsequent

Re: [sqlite] very large SQLite tables

2009-06-26 Thread John Stanton
Why would it not work? It is just adding an extra top level to the index. A tried and true method. Kosenko Max wrote: > Have you ever tested such proposal? > I believe that doesn't works. > > > Doug Fajardo wrote: > >> One approach might be to split the big, monolithic table into some

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Kosenko Max
I forgot to say about hash... My personal choice will be MurmurHash2 64 bit function http://murmurhash.googlepages.com/ http://en.wikipedia.org/wiki/MurmurHash2 - lots of implementations here It's fast (even in managed impls), have good characteristics and free. Don't use CRC64... P.S. You

Re: [sqlite] Near misses

2009-06-26 Thread Martin Pfeifle
Hi, I guess the speed could significantly be improved, if you leave out _car and _ar. The inverted index which is basically (term, blob_containing_all_document_ids_of_this_term), cannot skip any of the alphabetically ordered terms if the first character is variable. At least that's my

[sqlite] Near misses

2009-06-26 Thread Alberto Simões
Hello. I am trying to find words in a dictionary stored in sqlite, and trying a near miss approach. For that I tried an algorithm to create patterns corresponding to Levenshtein distance of 1 (edit distance of 1). That means, one adition, one remotion or one substitution. For that, my script

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Kosenko Max
Matthew O'Keefe wrote: > We wanted to post to the mailing list to see if there are any obvious, > first-order things we can try to improve performance for such a large > table. The problem with slow inserts generally speaking lies in the problem of cache miss. Imagine that each new insert in

[sqlite] Sqlite error in Multi-core systems (Sun T2000)

2009-06-26 Thread Nogueira, Jose (External)
Hello, I would like to inform you about a problem encountered upon writing in a sqlite database on a multi-core machine (Sun T2000). Indeed, on T2000 systems, during endurance tests that induce a lot of regular writings during several hours in a sqlite database (3.3 but also 3.6 sqlite

Re: [sqlite] very large SQLite tables

2009-06-26 Thread Kosenko Max
Have you ever tested such proposal? I believe that doesn't works. Doug Fajardo wrote: > > One approach might be to split the big, monolithic table into some number > of hash buckets, where each 'bucket' is separate table. When doing a > search, the program calculates the hash and accesses

Re: [sqlite] sqlite jdbc and rcp Eclipse

2009-06-26 Thread Juergen Schwitalla
Hi Enrico, the clean option is only of importance if you replace plugins within an already installed rcp application. Sorry for having confused you. cheers, Juergen On Thu, 25 Jun 2009, Enrico Piccinini wrote: Well, tomorrow I'm going to try if the option clean works. At this moment I can't