[sqlite] Compilation warning improvements with 3.3.0

2006-01-12 Thread Ken & Deb Allen

Wow! Am I impressed!

A while back I initiated the email concerning the large number of
compiler warnings that were being generated when I attempted to
compile the SQLite source code using either the Visual Studio 2005
compiler or using XCode 2.2 (gcc 4). This sparked off some
acrimonious comments, but it was noted that many of these issues
would be considered with the next release.

Well, I found time to download the 3.3.0 source code this evening and
compiled it with XCode -- and there are now a total of six (6)
compiler warnings, all concerning signed/unsigned differences!

The errors are in the following three files:
build.c @2363 and @3155
pager.c @462 and @473
vdbeaux.c @445 and @447

At least the objects appear to be the same size
and only differ in signed nature!

I also believe that the code compiled much faster, possibly as a
result of there being fewer warnings for the compiler to deal with.

This is a huge improvement in the code, in my opinion, as it
significantly increases the expectation that the code is behaving as
one would expect.



Re: [sqlite] SQLite as a Windows kernel tool

2005-10-29 Thread Ken & Deb Allen
This has been the whole point of my emails -- I am not sufficiently  
familiar with the actual code details for me to make that  
determination. There are no comments to indicate whether the implicit  
cast is 'safe' or not. I know that some casts are a simple fact of  
life, especially in UN!X based code, where the runtime or system API  
returns character data in an integer, but the conversion of a 64-bit  
to an 32-bit value is a different case.


If I could have determined the context to a reasonable degree of  
certainty, it would be one thing, but not being able to determine the  
actual intent raises some concerns.


-ken

On 29-Oct-05, at 12:56 PM, [EMAIL PROTECTED] wrote:


Ken & Deb Allen <[EMAIL PROTECTED]> wrote:



vdbeapi.c
e:\SQLITE\327\Source\vdbeapi.c(55) : warning C4244: 'return' :
conversion from 'i64' to 'int', possible loss of data
e:\SQLITE\327\Source\vdbeapi.c(195) : warning C4244: '=' : conversion
from 'double' to 'i64', possible loss of data
e:\SQLITE\327\Source\vdbeapi.c(232) : warning C4244: '=' : conversion
from 'double' to 'u64', possible loss of data




What about these three warnings do you think is a concern?
Have you actually looked at the code in question to see
if the possibility of data loss is real and unintentional?
What makes you think that these warnings are not just a case
of the compiler blowing smoke?

--
D. Richard Hipp <[EMAIL PROTECTED]>





[sqlite] SQLite as a Windows kernel tool

2005-10-28 Thread Ken & Deb Allen
I have been toying with the idea of checking whether or not I can use  
SQLite in a Windows device driver as its database. I am working on an  
existing product that is implemented as a series of Windows device  
drivers, filter drivers and even a file system driver. The file  
system driver manages a dedicated disk partition that contains an  
NTFS file system and a series of files (some flat files and some  
Btree files) that serve as the database. The code that manages this  
database is quite old and has been heavily modified several times,  
but the code is not well structured and contains a number of  
architectural features that act as hinderances to modifications and  
performance enhancements.


From the beginning it was clear to me that several issues would be  
greatly minimized or even eliminated if we could use some form of a  
relational database management package instead of the home-grown file  
management code, but there do not seem to be any database packages  
that are available to kernel code.


As part of another effort we looked at using SQLite, and performed a  
number of tests to that end. When I discovered that all of the source  
code was available, it seemed that we may be able to 'port' the code  
for use within out file system driver, either as a kernel DLL or  
directly compiled into the existing component. A number of features  
would have to be modified, such as the use of malloc() and free()  
calls, which would need to be replaced with the appropriate calls to  
kernel routines (if not the development of an extra set of routines  
for such dynamic memory usage), but we were hopeful.


I had used the Visual C++ 6 compiler (in C mode) to compile the  
existing SQLite source for use in user space, and that was not much  
of a problem, but the DDK (kernel) compiler was a modified version of  
the Visual C++ 7 compiler, and a number of compilation features are  
quite different when compiling code into the kernel as opposed as  
into user space. We had a number of problems with compiling the  
SQLite code as part of our kernel build, and so I decided to attempt  
to compile the latest source code for user space using the Visual C++  
7 compiler (Visual Studio .Net 2003). I have an HTML file that is the  
output from this attempt, which lists 193 compiler warnings (the code  
actually runs) based on the 3.2.7 version of the SQLite source code.  
This mail list does not permit me to attach the file (message  
rejected as too large), but if anyone wants to see it I can send it  
to you.


The vast majority of the warnings are related to either unsafe  
implicit data type conversions or signed and unsigned comparison  
operations. Some of these are not unreasonable, such as a complaint  
about conversion from "size_t" to "int", which should be the same  
size objects, but others definitely appear to be significant, such as  
an implicit conversion from "i64" or "u64" to int, which is a  
legitimate concern. Some of the signed/unsigned comparisons are also  
suspect (is an unsigned integer value of 0x81230001 greater than or  
less than a signed integer value of 0x0001? -- hint, as a signed  
value the former is a negative number, but as an unsigned it is not).


Some of these warnings are simply the compiler being extremely (and  
appropriately) cautious, but some of them truly do look like odd  
coding. Why should the code unconditionally and implicitly assign a  
64-bit integer value into a 32-bit integer variable? If the  
conditions under which this is done are such that there can be no  
loss of data, then there should be a comment explaining this, and the  
cast should be explicit to enforce that this is expected.  
Unfortunately, there are no such comments and not all of the implicit  
casts have a clear context when simply examining isolated lines of code.


I had a quick look at some of the code, but I am not certain whether  
all, or even most, of these warnings can be safely ignored or not. I  
tried modifying the code to add explicit casts to eliminate all of  
the warnings, which worked, but I do not know whether or not the  
resulting code contains runtime errors or not (specifically as a  
result of data loss at runtime or improper comparison logic).


Could someone have a look at some of these and provide some opinions?  
I know that the coding style used for SQLite is what I have heard  
referred to as "minimal C", but should the code not be commented  
concerning such conversions, and the conversion be made explicit?


-ken


Re: [sqlite] SQLite3 Explorer dates

2005-08-20 Thread Ken & Deb Allen
This would be more effective if it were also available for MacOS and  
Linux.


-ken

On 20-Aug-05, at 5:32 AM, Ralf Junker wrote:


Hello Clark Christensen,

SQLiteSpy executes the query "select date('2005-08-19');" with no  
problems. SQLiteSpy is a free SQLite3 database manager and is  
similar to EXP. To testdrive SQLiteSpy, download your copy from


  http://www.yunqa.de/delphi/sqlitespy/

Regards,

Ralf



If appropriate for a future rev of SQLite3 Explorer (EXP),
I'm requesting an update to the handling of date strings.

Using SQLite3 Explorer (EXP) v1.6 under WinXP, "select
date('2005-08-19')" gives a "not a valid FP number" error.






[sqlite] Problems using SQLITE .Net Data Provider (Windows)

2005-08-15 Thread Ken & Deb Allen
I recently downloaded the (Finisar) .Net Data Provider for SQLITE  
from SourceForge in order to use this in a C# applications under  
Windoes XP. I had a few small problems getting the database created  
at all via .Net, but that is working, and I was able to also define  
the schema for my database using CREATE TABLE, CREATE TRIGGER and  
CREATE INDEX statements.


Issue #1: I had an original problem where defining a table with the  
first column as "Record ID INTGER PRIMARY KEY AUTOINCREMENT NOT NULL"  
works fine when I execute it from the command line, but it chokes on  
the "AUTOINCREMENT" word when executed via .Net. It was suggested  
that I simply remove the word, which I did, and which permitted the  
command to execute without error, but it is my understanding that the  
behaviour of the table is different with and without this keyword.


Issue #2: Another problem is attempting to use UNICODE in the  
database. If, from the command line, the first command after I open  
the database is [PRAGMA encoding="UTF-16"], and then I create the  
schema, when I open the database later from the command line and  
execute the command [PRAGMA encoding;], the result is reported as  
"UTF-16le", as expected. When I do the same thing from my C# code,  
the command line reports the encoding as "UTF-8".


Issue #3: If I define a table as [CREATE TABLE Events (EventID INT  
NOT NULL, EventName VARCHAR(32) NOT NULL, EventLongName VARCHAR(128)  
NOT NULL, CreationDate DATETIME NOT NULL] via the C# code, then when  
I attempt to add records to the table I get the following exception.  
This same command executed from the SQLITE command line executes  
without an error and adds the record to the table.


Anyone have any thoughts on why these errors may be reported by the  
SQLITE .Net Data Provider?


By the way, I am using the SQLITE 3.2.2 DLL file for Windows  
downloaded from the SQLITE web site and (I believe, since I am home  
now) version 1.7 of the .Net Data Provider.


-ken


Database creation results;

MSDE database successfully created! (765.6397 milliseconds)
SQLITE database successfully created! (93.7518 milliseconds)
VistaDB database successfully created! (62.5012 milliseconds)
Firebird database successfully created! (1109.3963 milliseconds)

—> System.ApplicationException; 5QLITE; error inserting [Event]; no  
such column; current_timestamp


INSERT INTO Events(EventID; EventName; EventLongName; CreationDate)  
VALUE5(1; 'Event #1', 'Long Event Name #1'; current.


Finisar.5QLite.5QLiteException; no such column; current_timestamp

al:Finisar.SQLite.sqlite3.Throw()

al:Finisar,SQLite,sqlite3,CheckOK()

al: Finisar.SQLite.sqlite3.compile(String zSql)

al:Finisar.5QLite.One5QL5tatement.Compile()

al:Finisar.5QLite.5QLiteCommand.E?;ecul:eNonQuery()

al: DatabaseE valuation. 5qliteDatabase.Populal:eEventInformation 
(Int32 MaxEventID) in
f;\testing\databaseevaluation 
\databaseevaluation^sqlitedatabase.cs;linell6


— End of inner exception stack trace —

at DatabaseE valuation. 5qliteDatabase.PopulateEventInformation(Int32  
MaxEventID) in
f;\testing\databaseevaluation\databaseevaluation\sqlitedatabase.cs;  
line 130


at DatabaseE valuation.  
DatabaseEvaluationForm,buttonPrepareDatabases_Clicb(Object sender;  
EventArgs e) in
f;\testing\databaseevaluation\databaseevaluation 
\databaseevaluationform,cs; line 171


i ^...^.^ i




Re: [sqlite] Manifest Typing question

2005-06-21 Thread Ken & Deb Allen
Will declaring the column as VARCHAR not achieve the same thing? Does  
SQLITE not translate VARCHAR to TEXT, rather than numeric?


-ken

On 21-Jun-05, at 5:10 AM, D. Richard Hipp wrote:


On Tue, 2005-06-21 at 00:13 -0400, Tito Ciuro wrote:


Hello,

When I add text to the database, it's getting truncated because
SQLite is converting it to a number. For example, I enter "9.0", but
SQLite stores it as "9".

Is there a way to force the value to be inserted as string?




Make the declared datatype of the column TEXT.

Example:

CREATE TABLE t1(a, b INTEGER, c REAL, d TEXT);
INSERT INTO t1 VALUES(9.0,9.0,9.0,9.0);
SELECCT * FROM t1;

Results in:

9|9|9|9.0

--
D. Richard Hipp <[EMAIL PROTECTED]>





Re: [sqlite] updating records problem

2005-05-17 Thread Ken & Deb Allen
All database tables should be defined with a unique primary key. 
Ideally this should consist of one or more integer columns; tect 
columns can be used but they are universally less efficient. If the 
data being stored does not contain a natural unique identifier, then 
one should be added; you can simply name the column "CustomerID" or 
"PKey" and set it to be an integer. By ensuring that each record 
receives a unique identifier and including that in all queries, the 
main issue is resolved.

Of course one problem remains, and that is detecting whether some other 
user has changed the record since you saved it. To achieve this, you 
define another integer column and store another value in that field, 
but this value is updated each time the record is saved. This can be a 
simple sequential value that rolls around to zero (or one) again after 
reaching some maximum value. The name of the field should reflect its 
purpose, so you could name it something like "EditVersion" or 
"ChangeFlag". The client does not need to access this field, or the 
primary key field, but they can be used to ensure that the record being 
updated matches the current record (you could read the current record 
first, or use the fields to validate the update).

For example, to continue your example, the SELECT statement would be 
modified to include the "CustomerID" and "ChangeFlag" columns, and the 
RecordSet.Update() method would be modified to issue a SQL statement 
like "UPDATE Customers SET Name = xxx, Age = yyy WHERE CustomerID = 
 AND ChangeFlag = fff", and the code would check to ensure that 
exactly one record was modified.

-ken
On 17-May-05, at 4:46 PM, Will Leshner wrote:
I develop a database wrapper for SQLite and I have an interesting 
problem that I'm curious how other people solve. Basically, the users 
of my wrapper have the option of editing database records indirectly 
through the wrapper, rather than directly, using UPDATE. The database 
wrapper is a set of classes and one of those classes is a RecordSet. 
So, when the user asks for a RecordSet, with the intention of editing 
one or more records, she might do this (where rs is a RecordSet):

rs = db.SQLSelect("SELECT name, age FROM customers")
Now, the user can edit a record in the RecordSet like this:
rs.Edit
rs.Field("name") = "Frank"
rs.Field("age") = 10
rs.Update
What the wrapper does, when it sees the Update, is create SQL and feed 
it to SQLite:

UPDATE cusomers SET name='Frank', age=10 WHERE name= 
AND age=;

The problem is that the 'name' and 'age' fields are not sufficiently 
unique to identify the very row the user wanted to update. Instead 
every row that has matching names and ages are going to be updated.

To solve this problem, I've been telling users to explicitly add 
'rowid' as one of their columns when they issue SQL to select records 
to edit:

rs = db.SQLSelect("SELECT rowid, name, age FROM customers")
This works ok, but I'd really like to get rid of this limitation. I've 
considered ways of possibly inserting 'rowid' manually to the user's 
SQL, but the idea of modifying the user's SQL kind of leaves a bad 
taste in my mouth.

I know that PHP also uses SQLite and I'm wondering if it has the same 
problem and how it may have solved it.

Thanks for any help.



Re: [sqlite] timestamp how to ?

2005-04-23 Thread Ken & Deb Allen
Having used other databases extensively, and discovering that SQLITE 
does not have a native DATETIME data structure, I have elected to store 
the Date/Time value from the operating system (which is either a 32-bit 
or 64-bit value) directly into an INT field and then translate it into 
a string on retrieval.

I did some performance tests, and it seemed that converting a date/time 
string into the 32-bit or 64-bit value in my code and comparing integer 
values in a query were orders of magnitude faster than using the SQLITE 
functions for date/time comparisons on the fly. They are fine for 
formatting the data for display, but where I need to manipulate the 
data I still retrieve the 'raw' integer value and convert it as I 
require.

-ken
On 22-Apr-05, at 9:03 AM, [EMAIL PROTECTED] wrote:
"msaka msaka" <[EMAIL PROTECTED]> writes:
how can i use timestamp value in sqlite
http://www.sqlite.org/cvstrac/wiki?p=DateAndTimeFunctions



Re: [sqlite] Problem storing integers

2005-04-17 Thread Ken & Deb Allen
When can we expect a 3.2.2 release that includes this correction? I am 
working on a database that depends on 64-bit integer values.

-ken
On 15-Apr-05, at 7:35 AM, D. Richard Hipp wrote:
On Fri, 2005-04-15 at 10:53 +0100, Richard Boulton wrote:
I'm running the latest sqlite 3.2.1 command line tool on Windows
XP
I've just run some older versions of the command line tool and the 
last time
the value 281474976710655 was stored correctly was 3.0.8 I was 
missing a
couple of releases after 3.0.8 but saw the unexpected behaviour start 
in
3.1.2. Maybe it was introduced in 3.1?

The bug was introduced by check-in [2246] on 2005-Jan-20 just prior
to 3.1.0.  Any integer between 140737488355328 and 281474976710655
is stored incorrectly.  I'll put in a patch sometime today.
--
D. Richard Hipp <[EMAIL PROTECTED]>



Re: [sqlite] optimize table

2005-04-15 Thread Ken & Deb Allen
P.S. When experimenting with this, try naming this 'second' table first 
to reduce the work the database has to do in selecting/rejecting 
records. Ypu can experiment with this by creating this new table, 
populating it with a SELECT from the existing table (only the records 
without statistics), and then timing the SELECT/JOIN.

-ken
On 15-Apr-05, at 6:15 AM, Ken & Deb Allen wrote:
I cannot speak for SQLITE for certain, but I know with a number of 
commercial RDBMS packages the index is not used unless it contains a 
sufficient degree of distribution. Microsoft SQL Server, for example, 
is quite explicit in stating that if the distribution of values across 
the range of records in an index falls below a certain level, then the 
index is ignored, which is why you are used to put the most dynamic 
column first in a multi-column index. I do not know if SQLITE performs 
any of this analysis and makes decisions on whether to use the index 
or not, but it is possible, especially if a sort is involved, that the 
index is not helping much if all records have one of two values, and 
as the majority of he records have the same value, you are looking 
only for the minority records. I do not know if using a date/time for 
this files will speed it up or not, since your statistical inclusion 
query will be for records "WHERE statistic IS NULL" or "WHERE 
statistic = 0", and as the database grows in size this will be the 
minority set of records.

One 'trick' I have seen in the past was to create a second table that 
contained an entry for new records, also keyed by the same "id"; as 
each sale is created, an entry is also made in that table, possibly by 
a trigger. You select the records for statistics using a JOIN (inner), 
which will only return records that are in both tables. As the records 
are processed, the record is removed from the second table, and 
therefore the records will not be part of the JOIN in the future. In 
some databases this may be faster than using an index; I have not 
experimented with this in SQLITE, however.

-ken
On 15-Apr-05, at 4:30 AM, msaka msaka wrote:
i dont need working with dates and times..
i only set 1 records on which was done..
which index is better to create for this problem?
must i cut db files? or speed of statistic will be same now and after 
1000 000 records?


-Pôvodná správa-
Od: Gert Rijs [mailto:[EMAIL PROTECTED]
Odoslané: 15. apríla 2005 7:13
Komu: sqlite-users@sqlite.org
Predmet: Re: [sqlite] optimize table
msaka msaka wrote:
CREATE TABLE sale(
   id INTEGER NOT NULL UNIQUE PRIMARY KEY,
   bill_no integer,
   bill_item_no integer,
   item_name varchar(20),
   qty integer,
   amount DOUBLE,
   statistic integer
)

Why don't you replace "statistic" with a timestamp formatted like
mmdd, perhaps add the time as well. If you add an index on it you
can efficiently retrieve the rows for certain date periods without
needing to update the table.
Gert






Re: [sqlite] optimize table

2005-04-15 Thread Ken & Deb Allen
I cannot speak for SQLITE for certain, but I know with a number of 
commercial RDBMS packages the index is not used unless it contains a 
sufficient degree of distribution. Microsoft SQL Server, for example, 
is quite explicit in stating that if the distribution of values across 
the range of records in an index falls below a certain level, then the 
index is ignored, which is why you are used to put the most dynamic 
column first in a multi-column index. I do not know if SQLITE performs 
any of this analysis and makes decisions on whether to use the index or 
not, but it is possible, especially if a sort is involved, that the 
index is not helping much if all records have one of two values, and as 
the majority of he records have the same value, you are looking only 
for the minority records. I do not know if using a date/time for this 
files will speed it up or not, since your statistical inclusion query 
will be for records "WHERE statistic IS NULL" or "WHERE statistic = 0", 
and as the database grows in size this will be the minority set of 
records.

One 'trick' I have seen in the past was to create a second table that 
contained an entry for new records, also keyed by the same "id"; as 
each sale is created, an entry is also made in that table, possibly by 
a trigger. You select the records for statistics using a JOIN (inner), 
which will only return records that are in both tables. As the records 
are processed, the record is removed from the second table, and 
therefore the records will not be part of the JOIN in the future. In 
some databases this may be faster than using an index; I have not 
experimented with this in SQLITE, however.

-ken
On 15-Apr-05, at 4:30 AM, msaka msaka wrote:
i dont need working with dates and times..
i only set 1 records on which was done..
which index is better to create for this problem?
must i cut db files? or speed of statistic will be same now and after 
1000 000 records?


-Pôvodná správa-
Od: Gert Rijs [mailto:[EMAIL PROTECTED]
Odoslané: 15. apríla 2005 7:13
Komu: sqlite-users@sqlite.org
Predmet: Re: [sqlite] optimize table
msaka msaka wrote:
CREATE TABLE sale(
   id INTEGER NOT NULL UNIQUE PRIMARY KEY,
   bill_no integer,
   bill_item_no integer,
   item_name varchar(20),
   qty integer,
   amount DOUBLE,
   statistic integer
)

Why don't you replace "statistic" with a timestamp formatted like
mmdd, perhaps add the time as well. If you add an index on it you
can efficiently retrieve the rows for certain date periods without
needing to update the table.
Gert





[sqlite] Some Functional Questions

2005-04-13 Thread Ken & Deb Allen
I have been experimenting with SQLITE for a little over a week now, and 
I must say that I am fairly impressed with many of its capabilities. I 
have been experimenting with performance from several aspects, and the 
numbers are quite respectable. I have several years experience with 
Sybase, Access, Oracle, SQL Server and other relational databases, so I 
am fairly comfortable with evaluation features.

While I intend to write some more programs to test more facilities, I 
do have some questions that others may be able to answer, or provide 
comments, and thereby save me some considerable amount of time.

1. I notice that there are only four (4) data types in SQLITE, which is 
OK, but there is no TIMESTAMP type, which is an incredibly useful type. 
Using this type makes it very easy to test whether a specific record 
has been updated or not, since the database automatically updates the 
value to a unique value (at least within that table) each time a record 
is inserted or updated. I suspect that if I want this capability in 
SQLITE I shall have to resort to some form of trigger or manually 
control an incrementing field value. Are there any other options?

2. How does SQLITE handle the case where one program or thread (using 
its own open handle) attempts to read records that are being modified 
within a transaction from another program or thread? Will the rows read 
be the original values, the values from the transaction, or will the 
query fail?

3. One of the projects were I am considering using SQLITE is from 
within a Windows filter driver, to act as an intelligent repository for 
control information (of which there may be a significant amount). I 
know that the code as provided will not compile within the Windows 
kernel, as it depends on user runtime calls like 'malloc', 'free' and 
'FlushFileBuffers', but I am considering changing these calls to more 
internal forms (sqlite3_malloc, sqlite3_free and 
sqlite3_flushfilebuffers, for example), and then creating conditional 
compilation to define these to use either the user mode call or a 
kernel level call. Does anyone know if this has been attempted before, 
or if there are likely to be considerable problems with attempting 
this?

4. In some circumstances the information I want to store may represent 
a million or more records in each of a dozen or more tables. Is the 
organization of the data more efficient if I store each table in its 
own database file, or if I use a single file?

5. I also have cases where I may have a significant number of deleted 
records, which will produce a significant amount of free space that can 
be used for new records. I know that VACUUM can be used to dump the 
contents of the database to a clone, drop the database and rename it, 
but that takes a considerable amount of disk space, and in some cases 
that disk space may not be available. How 'expensive' is using 
auto-vacuum mode? I understand that this is going to attempt to reduce 
the size of the database file by releasing unused 'pages', but does 
that not require that data be moved around so that the empty pages are 
at the end of the database file?

Thanks in advance for any and all assistance.
-Ken


Re: [sqlite] create table question

2005-04-10 Thread Ken & Deb Allen
So, would this problem still exist if the code were to open two 
'connections' to the database, issue the SELECT on one open connection 
and then issue the CREATE TABLE via the other open connection?

Does this in any way prevent an application from opening a single 
connection, issuing a SELECT, and in the callback handling each of the 
rows from the SELECT have the code issue another SELECT and have a 
second callback handle the results from that query?

-ken
On 8-Apr-05, at 2:46 PM, Jay Sprenkle wrote:
select_stmt = db.execute("SELECT * FROM people")
# use some, but not all of the rows in select_stmt
create_stmt = db.execute("CREATE TABLE other (a,b)") # error:
database table is locked
Why does this happen?
Anyway around this?

You must finalize select_stmt before running again db.execute

Right.  I have an instance where I would like to keep the
select_stmt
_open_ (or not finalized) while I create a new table.  Is
this possible?
While you are reding the DB, you can't update it, sqlite support 
many
simultaneous readers but only one write; so you can't create
a new table
while your select statement is running.
but he has only one writer. A select is not a writer, the create 
statement is.
I couldn't get something like this to work either and ended up 
building a list
of updates in memory which I applied after the finalize of the select.




Re: [sqlite] Problem when asking for databases that don't exist

2005-04-09 Thread Ken & Deb Allen
To determine whether a specific database exists, check to see if the 
file exists before calling sqlite3_open(), using stat() or some other 
call.

To determine if a specific schema exists within an open database, use 
the query "SEELCT COUNT(*) FROM sqlite_master WHERE type = 'table' AND 
name IN ('Table12', 'Table2', ...)" and check that the resulting count 
is the expected value.

-ken
On 9-Apr-05, at 6:18 AM, Jochen Müller wrote:
Hello!
I am new to SQL and have a simple question concerning sqlite. I am 
using the functions sqlite3_open, sqlite3_get_table and 
sqlite3_free_table and sqlite3_close.

Because i do not yet know how to check whether a database exists or 
not, i always call the sqlite3_get_table function. If it returns an 
error, the database probably not exist. But if this function returns 
an error, all further calls to sqlite3 fail. In detail:

sqlite3_open(...)
sqlite3_get_table(...)
copy the columns to a string grid if sqlite3_get_table(...) succeeded
sqlite3_free_table(...)
sqlite3_close(...)
The first time sqlite3_get_table(...) fails, no further starts of this 
sequence work. I get no errors when executing the statements, but 
there's no table sqlite3_get_table copies to the given pointer 
variables.

There would be two solutions to my problem. I have to check first if a 
database exists or i have to make sqlite keep on working after 
sqlite3_get_table fails.

Sorry for my poor english.
Regards,
  Jochen




RE: [sqlite] www.sqlite.com has disappeared

2004-10-06 Thread Ken Cooper
I've had favorites linked to www.sqlite.com for some time now, working like
a charm. 

I'll update them, thanks.

-Original Message-
From: Eric Pankoke [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, October 05, 2004 11:42 PM
To: [EMAIL PROTECTED]
Subject: RE: [sqlite] www.sqlite.com has disappeared

The address is www.sqlite.org.

Eric Pankoke
Founder
Point Of Light Software
http://www.polsoftware.com/
 

-Original Message-
From: Ken Cooper [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, October 06, 2004 1:38 AM
To: [EMAIL PROTECTED]
Subject: [sqlite] www.sqlite.com has disappeared

Anyone else experiencing this?







[sqlite] www.sqlite.com has disappeared

2004-10-06 Thread Ken Cooper
Anyone else experiencing this?



[sqlite] Performance of using Intersect; sanity check for newbie

2004-09-23 Thread Ken Cooper
I'm implementing a simple full text search in a sqlite database, using an
additional table for the indexed words, associated record ids, and word
offsets. A typical query will have multiple keywords, plus some additional
fields specified.

 

CREATE TABLE words  (word TEXT, recordid INTEGER, word_offset INTEGER);

CREATE TABLE labels (label TEXT, recordid INTEGER);

 

I'm thinking of writing my query like this:

 

SELECT recordid FROM words WHERE word = "whitebeard"

INTERSECT

(SELECT recordid FROM words WHERE word = "wizard"

INTERSECT 

(SELECT recordid FROM labels WHERE label = "well-hidden"))

 

My question is, is this an efficient way to execute such a query? Are each
of the individual selects performed independently, or do they filter based
on the intersect? I will be able to determine which keywords are likely to
return more results, and therefore can order my selects. Right now I'm
operating under the assumption that they filter based on the intersect,
querying from left to right.

 

Thanks,

 

Ken

 



RE: [sqlite] Efficient select for multiple ids

2004-09-15 Thread Ken Cooper
The '.'s below should be ellipses.

-Original Message-
From: Ken Cooper [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, September 15, 2004 1:56 PM
To: [EMAIL PROTECTED]
Subject: [sqlite] Efficient select for multiple ids

I am implementing a virtualized listview on the result of a query by first
retrieving all unique ids for each row in the query result, then on demand
looking up additional information based on the ids in small chunks, say 20
at a time. I was thinking the best way to approach this second query was to
create and bind queries something like the following against the 20 ids I
have at the time:

 

SELECT id, . FROM additionalchunks WHERE id=? OR id=? OR . 

 

or

 

SELECT id, . FROM additionalchunks WHERE id in (? ? ? ? ? .)

 

Which of these is the more efficient way to approach the problem, or does
someone have a better suggestion? I remember seeing something referring to
this issue posted awhile back, but I'm having trouble locating it (googling
on 'where' and 'or' and 'in' is rather painful.).

 

Ken




[sqlite] Efficient select for multiple ids

2004-09-15 Thread Ken Cooper
I am implementing a virtualized listview on the result of a query by first
retrieving all unique ids for each row in the query result, then on demand
looking up additional information based on the ids in small chunks, say 20
at a time. I was thinking the best way to approach this second query was to
create and bind queries something like the following against the 20 ids I
have at the time:

 

SELECT id, . FROM additionalchunks WHERE id=? OR id=? OR . 

 

or

 

SELECT id, . FROM additionalchunks WHERE id in (? ? ? ? ? .)

 

Which of these is the more efficient way to approach the problem, or does
someone have a better suggestion? I remember seeing something referring to
this issue posted awhile back, but I'm having trouble locating it (googling
on 'where' and 'or' and 'in' is rather painful.).

 

Ken



[sqlite] Storing text in sqlite vs. external flat file

2004-09-10 Thread ken
I am looking into using sqlite for storing some data that will be ~100,000
records in size, where each record will contain text that has an average
size of 40k, but could be > 200k. I will likely need to encrypt (and
potentially compress) the database. 

 

My question is whether to store the text in the database, or to keep a
separate file for the text with seek pointers and lengths in the database.
My preference is for the former, since I wouldn't have to manage a separate
robust encryption/compression/deletion process, but I am concerned about the
perf and size of the insertion stress experiments I've been running, and
I've seen comments on this list suggesting the latter for BLOBs. Are there
specific tuning tweaks I can make to improve my results?

 

Thanks,

Ken Cooper



RE: [sqlite] long filenames on Mac OS

2004-04-12 Thread Williams, Kenneth (Ken) (TLR Corp)


> -Original Message-
> From: Ralph Wetzel [mailto:[EMAIL PROTECTED]
> Sent: Monday, April 12, 2004 11:52 AM
> To: Will Leshner
> Cc: Forum SQLite
> Subject: Re: [sqlite] long filenames on Mac OS
> 
> 
> Hi!
> 
> Just my 5 ct:
> 
> As only sqlite needs to know, how a journal is named, how about always
> truncating the original filename so that it fits (with the
> concetanated -j* ) into whatever length is ok for all 
> supported systems?

I'd prefer that it only did that on systems where filename length is a problem, i.e. 
that the max length is #define'd per-system.  No need to penalize people whose OSs 
aren't so draconian.

 -Ken


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] user id select

2004-04-12 Thread Williams, Kenneth (Ken) (TLR Corp)


> -Original Message-
> From: Lloyd thomas [mailto:[EMAIL PROTECTED]
> Sent: Sunday, April 11, 2004 1:18 PM
> To: [EMAIL PROTECTED]
> Subject: [sqlite] user id select
> 
> 
> I am trying to create a query which selects a user where it 
> is not part of a
> specified group. 

Perhaps you just want:

 SELECT user_id FROM users
 WHERE user_id NOT IN
  (SELECT user_id FROM grp_user WHERE group_id = 42)

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Diferent behavior in 2.8.12 and 2.8.13

2004-03-23 Thread Williams, Ken


> -Original Message-
> From: Rubens Jr. [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, March 23, 2004 5:00 PM
> 
> I read the OnLine Doc many times and I do not notice this :
> "SQLite expects the SQL string that you pass in to be 
> unchanged until the
> SQL has finished executing."
> 
> If this is in OnLine Doc I suggest it must be In a more 
> visible place ;)

That's probably a good idea, but if I were passing a char* to ANY C
function, not just the SQLite interface routines, I'd be pretty scared to
modify it in a callback before that function is finished running.

If the function doesn't expect to modify it, it can declare it "const".  If
it does expect to modify it, then you shouldn't ALSO modify it. =)

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Override conflict resolution behavior in triggers

2004-03-23 Thread Williams, Ken


> -Original Message-
> From: Banek, MB Matthew (8773) @ IS [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, March 23, 2004 1:19 PM
>  
> The problem I'm having is that the trigger does not use IGNORE for
conflict
> resolution, but uses REPLACE instead, since (as the documentation states)
> triggers use the conflict resolution method of statement that 
> causes them to fire (if one exists). 

Yeah, the documented behavior sounds very undesirable.  The conflict
resolution of a trigger should not have anything to do with the conflict
resolution of the statement that triggered it, in my opinion.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Precompiled queries forcing db syncs?

2004-03-22 Thread Williams, Ken


> -Original Message-
> From: Dan Thill [mailto:[EMAIL PROTECTED]
> Sent: Monday, March 22, 2004 10:34 AM
> To: [EMAIL PROTECTED]
> Subject: [sqlite] Precompiled queries forcing db syncs?
> 
> In short, when I switch to precompiled queries, according to gprof, it 
> looks like the DB is being synched after every call, even though I've 
> issued PRAGMA synchronous=OFF.  As a result, using 
> precompiled queries is an order of magnitude slower.

Make sure you're compiling the queries *inside* your transaction boundaries.
According to a message from DRH a few days ago, this is necessary to avoid
syncing:

On Mar 18, 2004, at 12:14 PM, D. Richard Hipp wrote:

> To run a query within a transaction, you also have to compile
> it within a transaction.  To run a query outside of a transaction
> it should be compiled outside of the transaction.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: [sqlite] performance question

2004-03-17 Thread Williams, Ken

> On my PC the following query requires about 53 seconds:
> select * from TABG a, TABB b where (a.S='3' or a.S='12 or...) and 
> b.G=a.G order by a.G asc;
> 
> (On Oracle with the same scheme and data it requires only 0.4 
> seconds.)

In my experience, even though SQLite has very low overhead and is pretty
lightweight, performance can get hurt pretty badly for complicated queries
(or even fairly simple ones like yours) when it chooses the wrong
optimization paths.

I've had situations where changing "SELECT ... FROM table1, table2 ..." to
"SELECT ... FROM table2, table1 ..." makes an enormous difference in
execution time, because when SQLite has more than one index to choose from,
it seems to choose randomly.  Sometimes it's wrong, and sometimes quite
badly so.

This is why many other DBMs put a lot of effort into developing things like
cost-based optimizers, so these kinds of issues can be dealt with nicely.
I'm not sure if SQLite plans to add such things, but I'm not sure it fits
with the stated goals of simplicity.

So I guess the moral is that when performance gets slow, you really have to
scrutinize the execution plans in SQLite more than in other databases I'm
used to, rather than just adding indexes you *think* should help and
trusting the database to do the "right" thing.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Multiuser experience under win32 anyone?

2004-03-15 Thread Williams, Ken


> -Original Message-
> From: Balthasar Indermuehle [mailto:[EMAIL PROTECTED]
> Sent: Monday, March 15, 2004 10:03 AM
> To: [EMAIL PROTECTED]
> Subject: [sqlite] Multiuser experience under win32 anyone?
> 
> 
> Hi all,
> 
> Has anyone been doing some testing on how sqlite performs in 
> a networked multiuser environment?

If you mean that multiple users will try to connect to the DB at the same
time, I'd say SQLite isn't going to perform well.  Traditional client-server
databases put a lot of work into supporting this model efficiently (by
supporting things like table-level or row-level locking, rather than
database-level like SQLite) and SQLite puts no work into it.

This assumes that multiple users will be reading *and* writing.  If they're
all just reading, SQLite might be fine.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] time calculations

2004-03-01 Thread Williams, Ken
Hi Donald,

> From: Griggs, Donald [mailto:[EMAIL PROTECTED]
> Sent: Monday, March 01, 2004 10:46 AM
> 
>I suppose that for a system to really care about a leap 
> second jump, it
> would have to be at least reasonably synched to the NIST clocks as a
> precondition -- otherwise the normal computer clock drift 
> would mean that the clock is off by multiple seconds routinely.

Not necessarily - you're assuming that the events stored in the database
correspond to events that took place on that computer, or at least that use
that computer's clock as a frame of reference.  But the events can take
place in any setting whatsoever.  For instance, if the events represent some
sort of timing for stock trades, or ..., then the database host's internal
clock has no relevance.

Imagine a (contrived) situation where you start with a datetime like "June
1, 1972 at 12:00:00" represented as a Unix epoch time (which should track
leap seconds correctly), then repeatedly add 60 seconds to it.  When you
pass the leap second at the end of 1972, you'll eventually get to "January
5, 1973 at 11:59:59".  By contrast, if you start with the same datetime in
SQLite and perform the same operation, you'll eventually get to "January 5,
1973 at 12:00:00", and the results won't agree.  (Here I'm assuming that
SQLite doesn't do leap seconds, but maybe Richard will reply that it does.)
This could potentially violate some application assumptions, and bugs could
ensue.

By contrast, if you repeatedly add one *minute* (which is not always the
same as adding 60 seconds), both systems should perform the same.

I'm not suggesting this error is catastrophic, merely that it's likely
present, and I'm not sure the members of this list necessarily have the
expertise (or desire) to implement the date/time functions in a really
correct way, though they may expect them to *work* in a correct way.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] time calculations

2004-03-01 Thread Williams, Ken


> -Original Message-
> From: D. Richard Hipp [mailto:[EMAIL PROTECTED]
> Sent: Saturday, February 28, 2004 6:57 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [sqlite] time calculations
> 
> 
> I also added the ability to put a time value in as the
> modifier and shift the date by that amount.  For example:
> 
>  SELECT datetime('2003-12-04 09:27:00', '00:01:21');
>  2003-12-04 09:28:21
> 

That sounds convenient but potentially dangerous - does it understand leap
years/seconds, for example?  And various other pitfalls of date/time math?

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Re: [inbox] [sqlite] About ORDER BY results...

2004-02-27 Thread Williams, Ken


> -Original Message-
> From: Jay Macaulay [mailto:[EMAIL PROTECTED]
> Sent: Friday, February 27, 2004 10:56 AM
> 
> Another solution that would allow using an index would be to 
> add another
> column, flatname, and insert the name into that column 
> without any accents
> while still inserting the real name with accents into a 
> different column,
> then do an order by on that flatname column.  This will allow 
> you to order
> it and still retain your indices, at the cost of an extra column.

Yes, that's perhaps what I would do in this situation, creating
INSERT/UPDATE triggers to make sure the integrity of the column is
maintained.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] help! cant unsubscribe

2004-02-17 Thread Williams, Ken


> -Original Message-
> From: zeb warrior [mailto:[EMAIL PROTECTED]

> I've been unable to carrry out your request: The address
> 
>[EMAIL PROTECTED]
> 
> was already on the sqlite-users mailing list when I received
> your request, and remains a subscriber.



It looks like it's trying to subscribe you, not unsubscribe you.
Double-check that you're sending it the right instructions.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[sqlite] Using multiple indexes?

2004-02-02 Thread Williams, Ken
Hi,

Is SQLite currently capable of using more than one index in a query?  I'm
adding indexes to a table, trying to optimize a query, and I only ever see
one index in the "EXPLAIN" output.

 -Ken


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Error on commit

2004-02-02 Thread Williams, Ken


> -Original Message-
> From: Matt Sergeant [mailto:[EMAIL PROTECTED]
> Sent: Monday, February 02, 2004 4:26 AM
> To: Williams, Ken
> Cc: SQLite-Users (E-mail)
> Subject: Re: [sqlite] Error on commit
> 
> 
> On 29 Jan 2004, at 18:23, Williams, Ken wrote:
> 
> >create_new_sqlite_database();
> >$dbh->do("BEGIN");
> >add_lots_of_rows_to_lots_of_tables();
> >$dbh->do("COMMIT");
> 
> Change to:
> 
>create_new_sqlite_database();
>$dbh->{AutoCommit} = 0;
>add_lots_of_rows_to_lots_of_tables();
>$dbh->commit;
># Optionally...
>$dbh->{AutoCommit} = 1;

Right, I've made that change (see a previous message) but it doesn't seem to
have any effect on this situation.

The real problem seemed to be that I was out of disk space, and the SQLite
error message to that effect (if any) was missing.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[sqlite] Error on commit

2004-01-29 Thread Williams, Ken
Hi,

I'm using DBD::SQLite compiled with SQLite version 2.8.11.  I'm getting the
following error, with RaiseError => 1:

   DBD::SQLite::db do failed:  at load_db.pl line 27, <$in_file> line
220663.

(Line 27 is simply $dbh->do("COMMIT"); )

So,

 A) Why isn't the error string being displayed as part of the error?

 B) What might be actually causing the commit to fail?  There is 1 active
transaction at this point in the code, and no previous transactions.  My
script boils down to the following pseudocode:

   create_new_sqlite_database();
   $dbh->do("BEGIN");
   add_lots_of_rows_to_lots_of_tables();
   $dbh->do("COMMIT");

Thanks,

 -Ken


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Functions & Keywords

2004-01-26 Thread Williams, Ken


> -Original Message-
> From: D. Richard Hipp [mailto:[EMAIL PROTECTED]
> Sent: Thursday, January 22, 2004 5:57 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [sqlite] Functions & Keywords
> 
> 
> Williams, Ken wrote:
> > 
> > Hmm - all of those functions seem to depend only on their 
> argument input,
> > making no external calls to the database.  How would I 
> implement, for
> > example, a stored procedure that performs several 
> inserts/selects on several
> > database tables?
> > 
> > I tried a little experiment with the Perl interface (which uses the
> > sqlite_create_function() call under the hood), and the 
> function I created
> > just hangs when it's called.  Is SQLite not re-entrant in 
> this way, or is
> > this a peculiarity of the Perl interface?  My code is below.
> > 
> 
> That capability was added on 2004-Jan-07.  See check-in [1166].
> 
> I have no idea whether or not Perl supports the new capability.

Hi,

I just downloaded sqlite-2.8.11, ran the DBD::SQLite getsqlite.pl script,
and tried my test again.  It worked.

Matt, it would be great if you would issue another release of DBD::SQLite
with an updated sqlite distribution.  These latest changes make it possible
to write some rather effective stored procedures in Perl.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Functions & Keywords

2004-01-22 Thread Williams, Ken


> -Original Message-
> From: Dennis Cote [mailto:[EMAIL PROTECTED]
> Sent: Thursday, January 22, 2004 5:23 PM
> To: Williams, Ken; [EMAIL PROTECTED]
> Subject: Re: [sqlite] Functions & Keywords
> 
> 
> Again, see the SQLite source, in particular the file func.c, 
> for examples of
> how to use these calls.

Hmm - all of those functions seem to depend only on their argument input,
making no external calls to the database.  How would I implement, for
example, a stored procedure that performs several inserts/selects on several
database tables?

I tried a little experiment with the Perl interface (which uses the
sqlite_create_function() call under the hood), and the function I created
just hangs when it's called.  Is SQLite not re-entrant in this way, or is
this a peculiarity of the Perl interface?  My code is below.

 -Ken


use strict;
use DBI;

my $dbh = DBI->connect('dbi:SQLite:dbname=test', undef, undef,
   {RaiseError => 1});

$dbh->do("CREATE TABLE foo (a, b)");
$dbh->do("INSERT INTO foo VALUES (1, 2)");
$dbh->do("INSERT INTO foo VALUES (2, 4)");

$dbh->func( 'get_b', 1,
sub {
  warn "Called the function";
  my ($x) = $dbh->selectrow_array
("SELECT b FROM foo WHERE a=$_[0]");
  return $x
}, 'create_function');

my $a = $dbh->selectall_arrayref
  ("SELECT a, get_b(a) FROM foo");

print "@$_\n" foreach @$a;

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Functions & Keywords

2004-01-22 Thread Williams, Ken


> -Original Message-
> From: Dennis Cote [mailto:[EMAIL PROTECTED]
> Sent: Thursday, January 22, 2004 1:58 PM
> To: Williams, Ken; 'Drew, Stephen'; [EMAIL PROTECTED]
> Subject: Re: [sqlite] Functions & Keywords
> 
> 
> > Please excuse the silly question, but how is that done?
> 
> It can't be done in SQL code.
> 
> You can do it quite easily using C language API calls. These 
> are documented
> in section 4 of the API documentation at
> http://www.hwaci.com/sw/sqlite/c_interface.html

Great, it looks fairly simple.  I was indeed looking for something I could
do using the C API.

So after I write & compile the code, how can I tell SQLite to load it?  Or
do I need to compile it into the sqlite libraries?

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Setting sqlite-shell preferences

2004-01-20 Thread Williams, Ken


> -Original Message-
> From: D. Richard Hipp [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, January 20, 2004 11:48 AM
> To: SQLite-Users (E-mail)
> Subject: Re: [sqlite] Setting sqlite-shell preferences
> 
> It is suppose to execute commands in ~/.sqliterc prior to giving you
> a prompt.  This works, mostly.  But I think there are some bugs
> outstanding against this.  The code there definately needs to be
> cleaned up.

Cool, that seems to work.  Thanks.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[sqlite] Setting sqlite-shell preferences

2004-01-20 Thread Williams, Ken
Hi,

Every time I use the sqlite shell, I'd like to be in column mode, and with
headers turned on.  Is there a way to autoexecute ".header on; .mode column"
every time I start up?  I think "sqlite -header -column" will do it, but is
there any way to put this in a stored preferences file or something?

 -Ken


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Optimizing a query

2004-01-14 Thread Williams, Ken

> -Original Message-
> From: D. Richard Hipp [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, January 14, 2004 9:22 AM
> To: Williams, Ken
> Cc: [EMAIL PROTECTED]
> Subject: Re: [sqlite] Optimizing a query
> 
> 
> Williams, Ken wrote:
> > 
> > So, no way to make it O(N)?  If the two indexes could be 
> iterated together,
> > as in the following pseudocode, it would seem to be an O(N) 
> operation.
> > 
> 
> Retrieving a single record from a BTree is an O(logN) operation.
> Doing so N times gives O(NlogN).

Oh, I thought it was also possible to step straight through an index, but I
guess I was mistaken.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Optimizing a query

2004-01-14 Thread Williams, Ken


> -Original Message-
> From: D. Richard Hipp [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, January 13, 2004 6:17 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [sqlite] Optimizing a query
> 
> 
> > Can anyone suggest a good way to optimize the following query?
> > 
> > SELECT count(*) FROM propositions p, output o
> >  WHERE p.verb_id=o.verb_id
> >AND p.tag=o.tag
> >AND (p.stop!=o.stop OR p.start!=o.start);
> > 
> 
> CREATE INDEX whatever ON output(verb_id,tag);
> 
> That will make it O(NlogN) instead of O(N**2).

So, no way to make it O(N)?  If the two indexes could be iterated together,
as in the following pseudocode, it would seem to be an O(N) operation.

  P_INDEX:
  while ($p_entry = p_index.next) {
while ($o_entry = o_index.next) {
  if ($o_entry == $p_entry) {
...do the rest of the query criteria...
  } elsif ($o_entry > $p_entry {
next P_INDEX;
  }
}
  }

 -Ken


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] locked - what am I doing wrong?

2004-01-09 Thread Williams, Ken
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> 
> The important thing to understand here, is that you're not stepping 
> through a set of results that you have in memory and that 
> were extracted 
> from the database. You are (at least for simple queries) 
> stepping through 
> data that is still in the database. Because you're still 
> actively reading 
> the database the writer thread can't get access until your query is 
> finished. If, instead, you want to load the whole results set 
> into memory 
> before stepping through them consider the sqlite_get_table_* 
> functions. 
> Alternatively you can collect the results into memory any way 
> you like 
> before stepping through the structures you've created yourself.

I ran into this yesterday too.  This is a pretty severe limitation compared
with other databases I've used, like Postgres, MySQL, and Oracle.  Are there
any plans/desires to fix this, or is it considered something that just comes
with the "lightweight" territory?

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] NULL popping up in INSERTed strings

2004-01-06 Thread Williams, Ken


> -Original Message-
> From: Nate Bargmann [mailto:[EMAIL PROTECTED]
> 
> While rare there are nasties like this:
> "81565","Why Not Reform the 
> ""Squeek-Box?""","Editorials","Young","Sumner B.","1CO",1917,9
> just how does a person handle this case?

With the Text::CSV_XS module, available on CPAN.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Update using multiple tables

2004-01-06 Thread Williams, Ken


> -Original Message-
> From: [EMAIL PROTECTED]
> 
> Maybe something along the lines of this could work.  There's 
> probably a more
> elegant method than this:
> 
> UPDATE output
>   SET stop=stop+1
>   WHERE ROWID =
> (SELECT output.ROWID
>FROM output, tokens,
>WHERE output.sentence=tokens.sentence
>  AND output.stop=tokens.position
>  AND output.type='prop'
>  AND tokens.postag='RP');

Aha, of course!  Thanks.  This ought to work.

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



RE: [sqlite] Update using multiple tables

2004-01-06 Thread Williams, Ken
Hi Derrell, thanks for the response.

> -Original Message-
> From: [EMAIL PROTECTED]
> 
> How about something like this:
> 
> UPDATE table1
>   SET field23 = (SELECT field42
>FROM table2
>WHERE condition);

Hmm, I'm not sure whether I can use that in this context.  My actual desired
query is:

sqlite> UPDATE output, tokens
   ...> SET output.stop=output.stop+1
   ...> WHERE output.sentence=tokens.sentence
   ...>   AND output.stop=tokens.position
   ...>   AND output.type='prop'
   ...>   AND tokens.postag='RP';

Can that be transformed into a valid statement using a sub-select as you
suggest?  If I do the obvious thing, the right-sides of the selection
criteria won't be properly "linked" to a single row, right? :

sqlite> UPDATE output
   ...> SET stop=stop+1
   ...> WHERE type='prop'
   ...>   AND sentence=(SELECT sentence FROM tokens
   ...> WHERE tokens.postag='RP')
   ...>   AND stop=(SELECT position FROM tokens)
   ...> WHERE tokens.postag='RP');

 -Ken

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



[sqlite] Update using multiple tables

2004-01-06 Thread Williams, Ken
Hi,

Is it possible in SQLite to update a table using information from another
table?  Something like the "UPDATE ... FROM" syntax in Postgres, or "UPDATE
table1, table2 ..." syntax in MySQL is what I'm looking for.

If not, can someone suggest a workaround?

 -Ken


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



<    1   2   3   4   5   6