Re: [sqlite] database image malformed

2007-01-09 Thread Dan Kennedy
On Tue, 2007-01-09 at 22:28 -0500, Mark Richards wrote:
> Using sqlite in our embedded device has offered tremendous capabilities 
> and very conveniently, and I thank the developers and enthusiasts who 
> continue to further this excellent project.
> 
> I've had one issue that I cannot explain and would ask for some input.
> 
> sqlite 3.1.3
> linux kernel 2.6.12
> cris-axis-linux-gnu
> 
> One field system began to issue "database disk image is malformed" for 
> reasons that I cannot yet explain.  I ran a PRAGMA integrity_check, 
> which told me:
> 
> sqlite> PRAGMA integrity_check;
> *** in database main ***
> Main freelist: 1 of 1 pages missing from overflow list starting at 0
> Page 46 is never used
> rowid 1355980 missing from index timestamp
> rowid 1356049 missing from index timestamp
> ...
> rowid 1356108 missing from index timestamp
> wrong # of entries in index timestamp
> wrong # of entries in index dataid
> sqlite>
> 
> VACUUM failed with the same "database disk image is malformed". 
> Attempts at deleting all records from the broken table failed.  In the 
> end, I was able to repair the database by bringing it down to my 
> workstation and loading it in the SQLite Administrator windows GUI and 
> executing Database: cleanup.
> 
> My database is built using the following PRAGMA statements:
> 
> pragmaPRAGMA auto_vacuum = 1;
> pragmaPRAGMA count_changes = 1;
> pragmaPRAGMA empty_result_callbacks = 1;
> pragmaPRAGMA legacy_file_format = OFF;
> pragmaPRAGMA synchronous = OFF;

With pragma synchronous set to "OFF", if the device lost power or 
the operating system crashed in the middle of a transaction database
corruption can occur. 

> 1) are there any tools available in sqlite3 that will help me find the 
> cause of this type of issue?
> 2) does anyone know what did SQLite Administrator do that VACUUM didn't?

The integrity check shows problems with index structures only - so maybe
SQLite Administrator issued queries that never used an index. Although
that doesn't explain why the VACUUM failed, I would of thought the same
reasoning would apply. Maybe it ran out of space in the file-system or
something?

> 3) since auto_vacuum is ON, I still need to do a VACUUM every so often. 
>   Any ideas why?

A vacuum recreates an entire database, more or less ensuring that
records are packed into database pages with very little wasted space.
By contrast, auto-vacuum mode automatically shrinks the file whenever
one or more pages are completely empty. So in an auto-vacuum database
file there are never empty pages but the packing of records may be
sub-optimal. 

Hence it is possible (even likely) that a VACUUM operation will
reduce the size of an auto-vacuum database a bit. The btree layer
tries to keep every page at least 2/3 full during regular balancing,
so I would guess a VACUUM could shrink an auto-vacuum database by
at most 33%. Almost certainly less.

Does this match up with what you're seeing?

Dan.



-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread Jay Sprenkle

Oplocks do not break things.  Oplocks will guarantee consistency.  They
are granted when only one client OS has a file open letting that client
OS perform locking and caching operations internally without consulting
the server each time.  If another client wants to open the file, then
that second open request is held up by the server, the first client
contacted and asked to flush any outstanding data, acquire locks, and
drop the oplock.  Once that has happened then the second client gets the
answer to its open request.


How is the first client 'contacted' and asked to respond?
I can't see how this is anything but useless. I can't imagine very many
programs honor this kind of request since I've never even heard of this
before last week. If the first client doesn't respond to the request
it would have to degenerate to a standard lock. Is this an OS hack
designed in for a specific microsoft application?



There are other forms of oplocks that allow for opening and closing the
file multiple times without consulting the server as well as some forms
of limiting sharing (dropped when clients start using byte range locking).

There have been some problems with Windows when smb signing is in use as
the design of smb signing assumes request response pairs whereas oplock
break notifications are asynchronous.  Other than degenerate cases,
current Windows versions have been patched.


Degenerate cases? This sounds like something only Microsoft could dream
up, so I guess degenerate applies... ;)

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jay Sprenkle wrote:
> I've heard this too. Windows networking has some issues with locking.
> You might research 'oplocks' or 'opportunistic locking' (or
> opportunistic caching)
> if you're interested in understanding what it's doing. I was reading
> it the other
> day and thought it might be the key to making it work correctly if you
> could
> turn oplocks off in windows.

Oplocks do not break things.  Oplocks will guarantee consistency.  They
are granted when only one client OS has a file open letting that client
OS perform locking and caching operations internally without consulting
the server each time.  If another client wants to open the file, then
that second open request is held up by the server, the first client
contacted and asked to flush any outstanding data, acquire locks, and
drop the oplock.  Once that has happened then the second client gets the
answer to its open request.

There are other forms of oplocks that allow for opening and closing the
file multiple times without consulting the server as well as some forms
of limiting sharing (dropped when clients start using byte range locking).

There have been some problems with Windows when smb signing is in use as
the design of smb signing assumes request response pairs whereas oplock
break notifications are asynchronous.  Other than degenerate cases,
current Windows versions have been patched.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFFpF+kmOOfHg372QQRAsUXAJ0WDnN/+L2EUO4Yp04eyo6XJU3a+QCg1Egd
81HnIn2KLf0QKyTlYsQIg3g=
=DccF
-END PGP SIGNATURE-

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



[sqlite] database image malformed

2007-01-09 Thread Mark Richards
Using sqlite in our embedded device has offered tremendous capabilities 
and very conveniently, and I thank the developers and enthusiasts who 
continue to further this excellent project.


I've had one issue that I cannot explain and would ask for some input.

sqlite 3.1.3
linux kernel 2.6.12
cris-axis-linux-gnu

One field system began to issue "database disk image is malformed" for 
reasons that I cannot yet explain.  I ran a PRAGMA integrity_check, 
which told me:


sqlite> PRAGMA integrity_check;
*** in database main ***
Main freelist: 1 of 1 pages missing from overflow list starting at 0
Page 46 is never used
rowid 1355980 missing from index timestamp
rowid 1356049 missing from index timestamp
...
rowid 1356108 missing from index timestamp
wrong # of entries in index timestamp
wrong # of entries in index dataid
sqlite>

VACUUM failed with the same "database disk image is malformed". 
Attempts at deleting all records from the broken table failed.  In the 
end, I was able to repair the database by bringing it down to my 
workstation and loading it in the SQLite Administrator windows GUI and 
executing Database: cleanup.


My database is built using the following PRAGMA statements:

pragmaPRAGMA auto_vacuum = 1;
pragmaPRAGMA count_changes = 1;
pragmaPRAGMA empty_result_callbacks = 1;
pragmaPRAGMA legacy_file_format = OFF;
pragmaPRAGMA synchronous = OFF;

1) are there any tools available in sqlite3 that will help me find the 
cause of this type of issue?

2) does anyone know what did SQLite Administrator do that VACUUM didn't?
3) since auto_vacuum is ON, I still need to do a VACUUM every so often. 
 Any ideas why?


/mark richards

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Shared cache mode issue

2007-01-09 Thread Dan Kennedy
On Tue, 2007-01-09 at 08:01 -0800, Peter James wrote:
> On 1/9/07, Dan Kennedy <[EMAIL PROTECTED]> wrote:
> But it looks to me like commit #3341 (August 2006) covers this
> up. #3341
> changes things so that the shared-schema is reset whenever any
> connection handle is closed, so it's not possible for the
> pointer in
> question to go stale.
> 
> 
> Hey Dan...
> 
> Thanks for confirming this, and I'll check out that patch.  Would you
> suggest that I file a bug on the issue I reported, or is resetting the
> shared schema like that on connection close here to stay?

I don't see why it's not here to stay. You could file a bug against
3.3.6 for reference purposes if you like.

Dan.

> 


-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread Jay Sprenkle

I've heard this too. Windows networking has some issues with locking.
You might research 'oplocks' or 'opportunistic locking' (or
opportunistic caching)
if you're interested in understanding what it's doing. I was reading
it the other
day and thought it might be the key to making it work correctly if you could
turn oplocks off in windows.

On 1/9/07, Daniel Önnerby <[EMAIL PROTECTED]> wrote:

I thought I read somewhere in the docs that this was not reliable (maybe
I dreamed it)???
This is great if this works, although I might still make the
socketserver for notifying when updates has been made.

Thank you for your replies.

John Stanton wrote:
> Why not just use the SMB file locks if you are using the SMB networking?



--
The PixAddixImage Collector suite:
http://groups-beta.google.com/group/pixaddix

SqliteImporter and SqliteReplicator: Command line utilities for Sqlite
http://www.reddawn.net/~jsprenkl/Sqlite

Cthulhu Bucks!
http://www.cthulhubucks.com

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread Daniel Önnerby
I thought I read somewhere in the docs that this was not reliable (maybe 
I dreamed it)???
This is great if this works, although I might still make the 
socketserver for notifying when updates has been made.


Thank you for your replies.

John Stanton wrote:

Why not just use the SMB file locks if you are using the SMB networking?

Daniel Önnerby wrote:
Well.. I do not mean that I will use the socketserver to run queries 
against it.
What I mean is that the database is opened by the applications from a 
windows share. The socketserver is only used to ask permission to 
write to the database and notifying the other applications that an 
update has been made.
My thought was that this will not require much implementation on the 
application. Just adding a call to the socketserver before every 
INSERT or UPDATE, and that call will wait until the permission has 
been granted by the socketserver. Once the INSERT/UPDATE has been 
made, another call is made to the socketserver to unlock the 
database. The socketserver will then notify the other clients that an 
update has been made.


Best regards
Daniel

John Stanton wrote:

That should work quite well.  We use such a strategy to implement 
remote, multi user access to Sqlite databases.  the user is 
unconcerned about locking or contentions.


In our case we made the server run on port 80 (HTTP) and use regular 
HTTP protocol so that it easily penetrates firewalls.  The server in 
our case can either be a CGI process on a regular WWW server or use 
a purpose developed multi-threaded daemon which gives better 
performance.


We make the data transport format XML for uniformity.  For example 
if the usage requirement were to become too intensive for sqlite we 
can switch the shared database to being PostgreSQL without affecting 
the clients.


Daniel Önnerby wrote:


Hi all!

At the company I work we have a windows application that use sqlite 
for the document format and this works great. We are now thinking 
about if it would be possible to have multiple users to access the 
db simultaneously from different computers (like a enterprise 
edition :) ). I have read everything about the multithreading 
issues  and I know that sqlite is not designed to work like this. 
But I have an idea on how I might solve this in our case and would 
like to ask the community if you think this is a god idea (or if it 
would work at all):
My idea is to create a small socketserver on the local network that 
the application holds an open connection to. When someone wants to 
write  (lock) to the  DB you always need to ask the socketserver if 
this is ok. The server will not keep any track of the database 
itself. The only purpose of the server is so that no one  tries to 
write simultaneously. The server will also notify the applications 
when a modification has been made (on unlock).


So.. could this work???

Best regards and thanks for the best (and smallest) SQL database 
ever made.

Daniel

- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 






- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 





- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 






- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 





-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread John Stanton

Why not just use the SMB file locks if you are using the SMB networking?

Daniel Önnerby wrote:
Well.. I do not mean that I will use the socketserver to run queries 
against it.
What I mean is that the database is opened by the applications from a 
windows share. The socketserver is only used to ask permission to write 
to the database and notifying the other applications that an update has 
been made.
My thought was that this will not require much implementation on the 
application. Just adding a call to the socketserver before every INSERT 
or UPDATE, and that call will wait until the permission has been granted 
by the socketserver. Once the INSERT/UPDATE has been made, another call 
is made to the socketserver to unlock the database. The socketserver 
will then notify the other clients that an update has been made.


Best regards
Daniel

John Stanton wrote:

That should work quite well.  We use such a strategy to implement 
remote, multi user access to Sqlite databases.  the user is 
unconcerned about locking or contentions.


In our case we made the server run on port 80 (HTTP) and use regular 
HTTP protocol so that it easily penetrates firewalls.  The server in 
our case can either be a CGI process on a regular WWW server or use a 
purpose developed multi-threaded daemon which gives better performance.


We make the data transport format XML for uniformity.  For example if 
the usage requirement were to become too intensive for sqlite we can 
switch the shared database to being PostgreSQL without affecting the 
clients.


Daniel Önnerby wrote:


Hi all!

At the company I work we have a windows application that use sqlite 
for the document format and this works great. We are now thinking 
about if it would be possible to have multiple users to access the db 
simultaneously from different computers (like a enterprise edition :) 
). I have read everything about the multithreading issues  and I know 
that sqlite is not designed to work like this. But I have an idea on 
how I might solve this in our case and would like to ask the 
community if you think this is a god idea (or if it would work at all):
My idea is to create a small socketserver on the local network that 
the application holds an open connection to. When someone wants to 
write  (lock) to the  DB you always need to ask the socketserver if 
this is ok. The server will not keep any track of the database 
itself. The only purpose of the server is so that no one  tries to 
write simultaneously. The server will also notify the applications 
when a modification has been made (on unlock).


So.. could this work???

Best regards and thanks for the best (and smallest) SQL database ever 
made.

Daniel

- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 






- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 





- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 






-
To unsubscribe, send email to [EMAIL PROTECTED]
-



[sqlite] Unable to close due to unfinalised statements

2007-01-09 Thread Ohad Eder-Pressman

i have an fts1 query that if i perform it and then close the database, i get
the above error.
i looked in and it seems like a query usually calls:

vm = compile(sql);
ret = _sqlite3_step(vm);


i noticed that the pointer to the vm is stored in the db struct as well.
in this particular query, the call to _sqlite3_step causes the db's pVdbe
pointer to change from that of the vm to something else.
that something else looks good and the query works just great, but when
trying to close the database later (after exhausting the resulting rowset) i
get the error.
after exhausting the rowset i call _sqlite3_finalize on the variable vm from
the code above, because that's what i was working on. after i do that,
db->pVbde continues to point to a vbde structure, and that's what causes the
'close' call to fail.

any idea what may be causing this ?


[sqlite] Any more bugs reports against 3.3.9?

2007-01-09 Thread drh
Last call for bug reports against version 3.3.9.
Unless something serious comes up, 3.3.10 goes out in
the morning.
--
D. Richard Hipp  <[EMAIL PROTECTED]>


-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread Daniel Önnerby
Well.. I do not mean that I will use the socketserver to run queries 
against it.
What I mean is that the database is opened by the applications from a 
windows share. The socketserver is only used to ask permission to write 
to the database and notifying the other applications that an update has 
been made.
My thought was that this will not require much implementation on the 
application. Just adding a call to the socketserver before every INSERT 
or UPDATE, and that call will wait until the permission has been granted 
by the socketserver. Once the INSERT/UPDATE has been made, another call 
is made to the socketserver to unlock the database. The socketserver 
will then notify the other clients that an update has been made.


Best regards
Daniel

John Stanton wrote:
That should work quite well.  We use such a strategy to implement 
remote, multi user access to Sqlite databases.  the user is 
unconcerned about locking or contentions.


In our case we made the server run on port 80 (HTTP) and use regular 
HTTP protocol so that it easily penetrates firewalls.  The server in 
our case can either be a CGI process on a regular WWW server or use a 
purpose developed multi-threaded daemon which gives better performance.


We make the data transport format XML for uniformity.  For example if 
the usage requirement were to become too intensive for sqlite we can 
switch the shared database to being PostgreSQL without affecting the 
clients.


Daniel Önnerby wrote:

Hi all!

At the company I work we have a windows application that use sqlite 
for the document format and this works great. We are now thinking 
about if it would be possible to have multiple users to access the db 
simultaneously from different computers (like a enterprise edition :) 
). I have read everything about the multithreading issues  and I know 
that sqlite is not designed to work like this. But I have an idea on 
how I might solve this in our case and would like to ask the 
community if you think this is a god idea (or if it would work at all):
My idea is to create a small socketserver on the local network that 
the application holds an open connection to. When someone wants to 
write  (lock) to the  DB you always need to ask the socketserver if 
this is ok. The server will not keep any track of the database 
itself. The only purpose of the server is so that no one  tries to 
write simultaneously. The server will also notify the applications 
when a modification has been made (on unlock).


So.. could this work???

Best regards and thanks for the best (and smallest) SQL database ever 
made.

Daniel

- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 






- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 





-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread John Stanton
We haven't released it for public distribution because of support 
constraints.  It needs documentation, a configure script etc.  If we get 
around to doing that we would publish it.  Currently it compiles and 
runs on Linux, Windows 9x/NT and AIX.


Having said that, tell me more about your application.  What level of 
SQL (short lookups or long jobs)?  How many users?  What is the server 
host?  We may be able to help.


Dusan Gibarac wrote:

Is it server available for the public or everybody has to find its own
solution? We need to support access in home networks that usually consist of
few PCs.

Dusan

-Original Message-
From: John Stanton [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, January 09, 2007 12:21 PM

To: sqlite-users@sqlite.org
Subject: Re: [sqlite] multiuser DB on network share

That should work quite well.  We use such a strategy to implement 
remote, multi user access to Sqlite databases.  the user is unconcerned 
about locking or contentions.


In our case we made the server run on port 80 (HTTP) and use regular 
HTTP protocol so that it easily penetrates firewalls.  The server in our 
case can either be a CGI process on a regular WWW server or use a 
purpose developed multi-threaded daemon which gives better performance.


We make the data transport format XML for uniformity.  For example if 
the usage requirement were to become too intensive for sqlite we can 
switch the shared database to being PostgreSQL without affecting the 
clients.


Daniel Önnerby wrote:


Hi all!

At the company I work we have a windows application that use sqlite for 
the document format and this works great. We are now thinking about if 
it would be possible to have multiple users to access the db 
simultaneously from different computers (like a enterprise edition :) ). 
I have read everything about the multithreading issues  and I know that 
sqlite is not designed to work like this. But I have an idea on how I 
might solve this in our case and would like to ask the community if you 
think this is a god idea (or if it would work at all):
My idea is to create a small socketserver on the local network that the 
application holds an open connection to. When someone wants to write  
(lock) to the  DB you always need to ask the socketserver if this is ok. 
The server will not keep any track of the database itself. The only 
purpose of the server is so that no one  tries to write simultaneously. 
The server will also notify the applications when a modification has 
been made (on unlock).


So.. could this work???

Best regards and thanks for the best (and smallest) SQL database ever


made.


Daniel





- 


To unsubscribe, send email to [EMAIL PROTECTED]




- 







-
To unsubscribe, send email to [EMAIL PROTECTED]

-




-
To unsubscribe, send email to [EMAIL PROTECTED]
-




-
To unsubscribe, send email to [EMAIL PROTECTED]
-



RE: [sqlite] multiuser DB on network share

2007-01-09 Thread Dusan Gibarac
Is it server available for the public or everybody has to find its own
solution? We need to support access in home networks that usually consist of
few PCs.

Dusan

-Original Message-
From: John Stanton [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, January 09, 2007 12:21 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] multiuser DB on network share

That should work quite well.  We use such a strategy to implement 
remote, multi user access to Sqlite databases.  the user is unconcerned 
about locking or contentions.

In our case we made the server run on port 80 (HTTP) and use regular 
HTTP protocol so that it easily penetrates firewalls.  The server in our 
case can either be a CGI process on a regular WWW server or use a 
purpose developed multi-threaded daemon which gives better performance.

We make the data transport format XML for uniformity.  For example if 
the usage requirement were to become too intensive for sqlite we can 
switch the shared database to being PostgreSQL without affecting the 
clients.

Daniel Önnerby wrote:
> Hi all!
> 
> At the company I work we have a windows application that use sqlite for 
> the document format and this works great. We are now thinking about if 
> it would be possible to have multiple users to access the db 
> simultaneously from different computers (like a enterprise edition :) ). 
> I have read everything about the multithreading issues  and I know that 
> sqlite is not designed to work like this. But I have an idea on how I 
> might solve this in our case and would like to ask the community if you 
> think this is a god idea (or if it would work at all):
> My idea is to create a small socketserver on the local network that the 
> application holds an open connection to. When someone wants to write  
> (lock) to the  DB you always need to ask the socketserver if this is ok. 
> The server will not keep any track of the database itself. The only 
> purpose of the server is so that no one  tries to write simultaneously. 
> The server will also notify the applications when a modification has 
> been made (on unlock).
> 
> So.. could this work???
> 
> Best regards and thanks for the best (and smallest) SQL database ever
made.
> Daniel
> 
>

- 
> 
> To unsubscribe, send email to [EMAIL PROTECTED]
>

- 
> 
> 



-
To unsubscribe, send email to [EMAIL PROTECTED]

-




-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] multiuser DB on network share

2007-01-09 Thread John Stanton
That should work quite well.  We use such a strategy to implement 
remote, multi user access to Sqlite databases.  the user is unconcerned 
about locking or contentions.


In our case we made the server run on port 80 (HTTP) and use regular 
HTTP protocol so that it easily penetrates firewalls.  The server in our 
case can either be a CGI process on a regular WWW server or use a 
purpose developed multi-threaded daemon which gives better performance.


We make the data transport format XML for uniformity.  For example if 
the usage requirement were to become too intensive for sqlite we can 
switch the shared database to being PostgreSQL without affecting the 
clients.


Daniel Önnerby wrote:

Hi all!

At the company I work we have a windows application that use sqlite for 
the document format and this works great. We are now thinking about if 
it would be possible to have multiple users to access the db 
simultaneously from different computers (like a enterprise edition :) ). 
I have read everything about the multithreading issues  and I know that 
sqlite is not designed to work like this. But I have an idea on how I 
might solve this in our case and would like to ask the community if you 
think this is a god idea (or if it would work at all):
My idea is to create a small socketserver on the local network that the 
application holds an open connection to. When someone wants to write  
(lock) to the  DB you always need to ask the socketserver if this is ok. 
The server will not keep any track of the database itself. The only 
purpose of the server is so that no one  tries to write simultaneously. 
The server will also notify the applications when a modification has 
been made (on unlock).


So.. could this work???

Best regards and thanks for the best (and smallest) SQL database ever made.
Daniel

- 


To unsubscribe, send email to [EMAIL PROTECTED]
- 






-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Sqlite design question

2007-01-09 Thread John Stanton
Consider having multiple databases.  One can have all the read only 
tables for example and will therefore always be available for reading. 
Dynamic tables would be in another.


You can increase the granularity by having more databases, perhaps as 
many as one per dynamic table, depending on the intricacy of your relations.


Even if you end up with four or five databases that is only three or 
four extra files with Sqlite, still far fewer files than if you were 
using a large scale database.


Florian Weimer wrote:

* Ken:



Would the reader be blocked by the writer?



Yes.



Would the writer be blocked by the reader?



Yes.

However, depending on the size of the transactions this may not be an
issue.



I guess I'm unclear what I can/cant do using sqlite and how to gain
as much performance as possible.



If your database isn't too large, and you aren't running on Windows,
you could make a copy of the database before updating it, so that
readers and the writer work on different databases.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-




-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Sqlite design question

2007-01-09 Thread Nicolas Williams
On Tue, Jan 09, 2007 at 01:28:21PM +0100, Florian Weimer wrote:
> If your database isn't too large, and you aren't running on Windows,
> you could make a copy of the database before updating it, so that
> readers and the writer work on different databases.

I wish ZFS would allow one to snapshot/clone individual files.
Technically it does for zvols, but IIRC there's only a CLI, not an API
for it.  If such APIs ever appear then it could prove useful in
improving SQLite concurrency (readers could run concurrently with a
single writer).

Nico
-- 

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Shared cache mode issue

2007-01-09 Thread Peter James

On 1/9/07, Dan Kennedy <[EMAIL PROTECTED]> wrote:


But it looks to me like commit #3341 (August 2006) covers this up. #3341
changes things so that the shared-schema is reset whenever any
connection handle is closed, so it's not possible for the pointer in
question to go stale.




Hey Dan...

Thanks for confirming this, and I'll check out that patch.  Would you
suggest that I file a bug on the issue I reported, or is resetting the
shared schema like that on connection close here to stay?

Thanks again!
Pete.


[sqlite] multiuser DB on network share

2007-01-09 Thread Daniel Önnerby

Hi all!

At the company I work we have a windows application that use sqlite for 
the document format and this works great. We are now thinking about if 
it would be possible to have multiple users to access the db 
simultaneously from different computers (like a enterprise edition :) ). 
I have read everything about the multithreading issues  and I know that 
sqlite is not designed to work like this. But I have an idea on how I 
might solve this in our case and would like to ask the community if you 
think this is a god idea (or if it would work at all):
My idea is to create a small socketserver on the local network that the 
application holds an open connection to. When someone wants to write  
(lock) to the  DB you always need to ask the socketserver if this is ok. 
The server will not keep any track of the database itself. The only 
purpose of the server is so that no one  tries to write simultaneously. 
The server will also notify the applications when a modification has 
been made (on unlock).


So.. could this work???

Best regards and thanks for the best (and smallest) SQL database ever made.
Daniel

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



[sqlite] Locking support for remote databases with Mac OSX

2007-01-09 Thread Marco Bambini
I think that starting from version 3.3.8 there is the code in os.c  
that add supports for locking remote databases with Mac OSX.
Inside os.c there are a couple of #defines  
(SQLITE_ENABLE_LOCKING_STYLE, SQLITE_FIXED_LOCKING_STYLE ) that I  
think should help with my request.


My question is: What I should do in order to compile an sqlite  
library that supports locking for remote databases with Mac OSX?


Thanks a lot,
---
Marco Bambini
http://www.sqlabs.net
http://www.sqlabs.net/blog/
http://www.sqlabs.net/realsqlserver/




-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Sqlite design question

2007-01-09 Thread Florian Weimer
* Ken:

>  Would the reader be blocked by the writer?

Yes.

> Would the writer be blocked by the reader?

Yes.

However, depending on the size of the transactions this may not be an
issue.

>  I guess I'm unclear what I can/cant do using sqlite and how to gain
>  as much performance as possible.

If your database isn't too large, and you aren't running on Windows,
you could make a copy of the database before updating it, so that
readers and the writer work on different databases.

-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Single-character pathnames in win2k

2007-01-09 Thread Ralf Junker

>> Can somebody who understands or regularly uses windows please 
>> look into it for me.
>
>It seems like changing 
>nByte = GetFullPathNameW(zWide, 0, 0, &zNotUsedW) + 1;
>to 
>nByte = GetFullPathNameW(zWide, 0, 0, &zNotUsedW) + 3;
>corrects the problem.

Not a solution to the problem, but a small optimization suggestion:

According to MSDN (http://msdn2.microsoft.com/en-us/library/aa364963.aspx) 
there is no need for zNotUsed and zNotUsedW. Both can be replaced by NULL.

Here is the relevant quote:

lpFilePart 
[out] A pointer to a buffer that receives the address (in lpBuffer) of the 
final file name component in the path. Specify NULL if you do not need to 
receive this information. 


-
To unsubscribe, send email to [EMAIL PROTECTED]
-



AW: [sqlite] Equivalent of OLE object da

2007-01-09 Thread Michael Ruck
An OLE object is persisted into a stream of bytes. You can store OLE objects
into SQLite as a BLOB, but you need to make your own (specialized)
implementation of one of the IPersistXXX interfaces (most likely
IPersistStream), which stores the object into an SQLite column/reads a
serialized object from an SQLite column.

Michael

-Ursprüngliche Nachricht-
Von: shivaranjani [mailto:[EMAIL PROTECTED] 
Gesendet: Dienstag, 9. Januar 2007 05:57
An: sqlite-users@sqlite.org
Betreff: [sqlite] Equivalent of OLE object da

Hi all,

 

 

Is there any equivalent of OLE object datatype of Access in SQlite dll.

 

 

Regards,

 

A. Shivaranjani



-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Shared cache mode issue

2007-01-09 Thread Dan Kennedy
On Mon, 2007-01-08 at 16:03 -0800, Peter James wrote:
> Hey folks...
> 
> The context of this message is sqlite library version 3.3.6, using the
> shared-cache mode, effectively following the test_server.c example.
> Immediately upon switching to shared-cache mode we started seeing errors
> like so when preparing statements:
> 
> [ERROR] (lib/sqlite/src/build.c:1220) no such collation sequence:  garbage>
> 
> Drilling down, this is what I'm understanding to be the case...  Collators
> are attached to the individual sqlite* handles, remaining valid only while
> the connection to which the handle refers is valid.  On the other hand, it
> appears that indexes are stored inside of the schema, and use a lookup
> string ("BINARY", "NOCASE") to find the contained column collators.  This
> lookup string is actually in memory allocated as part of the collator, and
> is freed when the connection is closed, leaving a dangling pointer in the
> index.

This only happens with the default collation sequence. In build.c, a
pointer may be copied from sqlite3.pDfltColl->zName into the schema.
This is a bug, for the reasons identified above. (in cvs sources: 
line 2479 of build.c).

But it looks to me like commit #3341 (August 2006) covers this up. #3341
changes things so that the shared-schema is reset whenever any
connection handle is closed, so it's not possible for the pointer in
question to go stale.

So if you upgrade to 3.3.7 or newer you should be Ok. Or if you can't
upgrade for your own reasons, maybe add something similar to #3341 
(only one line).

Dan.



-
To unsubscribe, send email to [EMAIL PROTECTED]
-



Re: [sqlite] Single-character pathnames in win2k

2007-01-09 Thread BardzoTajneKonto
> Can somebody who understands or regularly uses windows please 
> look into it for me.

It seems like changing 
nByte = GetFullPathNameW(zWide, 0, 0, &zNotUsedW) + 1;
to 
nByte = GetFullPathNameW(zWide, 0, 0, &zNotUsedW) + 3;
corrects the problem.

According to documentation even that + 1 isn't necessary, and actually without 
it and with longer file name the call does't return an error and everything 
work as expected. Error isn't returned even with 1-letter file, but +3 is 
required in my computer to make it work.

Wiktor Adamski


--
Lewa kasa! Zobacz >> http://link.interia.pl/f19e2


-
To unsubscribe, send email to [EMAIL PROTECTED]
-