[sqlite] Multiple instances of the same program accessing the same db file

2015-05-05 Thread Keith Medcalf
> > There is no difference between a file opened over the network and one > opened locally, except that in the network open case the network > filesystem does not maintain consistency between the client view of the > file and the server view of that same file, because doing so takes time > and

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-05 Thread Simon Slavin
On 5 May 2015, at 1:57pm, Keith Medcalf wrote: > There is no difference between a file opened over the network and one opened > locally, except that in the network open case the network filesystem does not > maintain consistency between the client view of the file and the server view > of

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-05 Thread Simon Slavin
On 5 May 2015, at 11:44am, James K. Lowden wrote: > Simon Slavin wrote: > >> In contrast to NFS both SMB and AFP are designed to support networks >> properly, with caches only on the computer which hosts the file and >> locking correctly implemented. > > Are you saying that SMB clients

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-05 Thread Keith Medcalf
> SMB keeps its cache on the computer which hosts the file. So if a file is > opened locally the cache is on the only computer concerned. If computer A > opens a file on computer B, the file-system cache is on computer B, where > all file requests pass through it. > > Of course a badly written

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-05 Thread James K. Lowden
On Mon, 4 May 2015 02:05:54 +0100 Simon Slavin wrote: > On 4 May 2015, at 1:30am, James K. Lowden > wrote: > > > That is the way most remote filesystems are designed and implemented > > and documented. Cf. http://www.ietf.org/rfc/rfc1813.txt: > > > > 4.11 Caching policies > > > > The

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-04 Thread Simon Slavin
On 4 May 2015, at 8:51pm, Scott Doctor wrote: > The issue of locking a file during concurrent access seems to be a major > issue, mostly due to the schizophrenic abilities of network file systems. Has > the SQLite development team considered embedding its own file locking system > logic,

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-04 Thread Richard Hipp
On 5/4/15, Scott Doctor wrote: > > The issue of locking a file during concurrent access seems to be > a major issue, mostly due to the schizophrenic abilities of > network file systems. Has the SQLite development team considered > embedding its own file locking system logic, such as creating >

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-04 Thread Scott Doctor
The issue of locking a file during concurrent access seems to be a major issue, mostly due to the schizophrenic abilities of network file systems. Has the SQLite development team considered embedding its own file locking system logic, such as creating its own lock file to prevent damage from

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-04 Thread Simon Slavin
On 4 May 2015, at 1:30am, James K. Lowden wrote: > That is the way most remote filesystems are designed and implemented > and documented. Cf. http://www.ietf.org/rfc/rfc1813.txt: > > 4.11 Caching policies > > The NFS version 3 protocol does not define a policy for > caching on the

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-04 Thread Stephen Chrzanowski
TL;DR Get SQLite away from networks entirely when you start throwing multiple machines at the problem. Your last paragraph is probably the best bet if you're not willing to get into a real client/server setup, and yes, you're going to run into aggravated users, but, then, which is more important

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Simon Slavin
On 3 May 2015, at 9:09pm, Scott Doctor wrote: > Hmm, one for doing my own locking, one against it. As this seems to be an > obvious issue in any network, I wonder why the network developers have not > appropriately addressed this issue. They've tried. With modern operating systems locking

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread James K. Lowden
On Sun, 03 May 2015 13:09:43 -0700 Scott Doctor wrote: > I wonder why the network developers have not appropriately addressed > this issue. Looks like I need to research this problem more before > implementing. >From one perspective -- a common use case -- they *have* addressed this issue.

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Scott Doctor
I googled file locking issues. Sheesh, The horror stories of programs like Quickbooks corrupting data files. Files getting locked when nobody is using them (A windows 7 bug that supposedly was fixed) and endless information about what not to do, or what does not work. Very little about what

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread James K. Lowden
On Sat, 02 May 2015 19:24:19 -0700 Scott Doctor wrote: > Each computer has its own copy of the program, but may > share the database file which may be located on any of the computers. > So I am wondering whether I should implement my own locking logic in > my program. Yes, you need to arrange

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Keith Medcalf
> Hmm, one for doing my own locking, one against it. As this seems to be > an obvious issue in any network, I wonder why the network developers > have not appropriately addressed this issue. They have. In the early 80's when network filesystems were invented they were incredibly slow. So

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Scott Doctor
Hmm, one for doing my own locking, one against it. As this seems to be an obvious issue in any network, I wonder why the network developers have not appropriately addressed this issue. Looks like I need to research this problem more before implementing. I dislike probability games of designs

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Simon Slavin
On 3 May 2015, at 3:24am, Scott Doctor wrote: > My program runs on windows and mac with a local area network connecting the > computers. Each computer has its own copy of the program, but may share the > database file which may be located on any of the computers. So I am wondering > whether

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Simon Slavin
On 3 May 2015, at 2:03am, Scott Doctor wrote: > To review, after opening the database, issue the PRAGMA busy_timeout = x, > with x being however long I want to wait before aborting. Yes. Since you have two copies of one program, I don't need to say this for you, but for other people, the

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-03 Thread Simon Slavin
On 3 May 2015, at 12:55am, J Decker wrote: > Yes, it really requires only a little additional work on application side. > The native open will open it in read/write share allow, and handle > interlocking. > > if you get a result of SQLITE_BUSY you need to retry the operation after a > short

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread Richard Hipp
On 5/2/15, Scott Doctor wrote: > > Is the PRAGMA value the retry interval, or the timeout where it > aborts and reports a failure? > The timeout. -- D. Richard Hipp drh at sqlite.org

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread Scott Doctor
hmm, I am using sqlite as a project file that keeps track of a variety of information (usually a couple dozen megabytes in size per project). My initial post assumed a single user with a couple windows open. The file might be accessed by another user on a local area network. Usually no more

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread Scott Robison
On Sat, May 2, 2015 at 7:03 PM, Scott Doctor wrote: > > To review, after opening the database, issue the PRAGMA busy_timeout = x, > with x being however long I want to wait before aborting. I can keep both > database handles open at the same time, but need to make sure I finalize > the

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread Scott Doctor
To review, after opening the database, issue the PRAGMA busy_timeout = x, with x being however long I want to wait before aborting. I can keep both database handles open at the same time, but need to make sure I finalize the operation before the timeout happens (assuming the other program is

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread Scott Doctor
Is the PRAGMA value the retry interval, or the timeout where it aborts and reports a failure? Scott Doctor scott at scottdoctor.com -- On 5/2/2015 5:08 PM, Simon Slavin wrote: > On 3 May 2015, at 12:55am, J Decker wrote: > >> Yes, it really requires only a little

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread J Decker
Yes, it really requires only a little additional work on application side. The native open will open it in read/write share allow, and handle interlocking. if you get a result of SQLITE_BUSY you need to retry the operation after a short time. On Sat, May 2, 2015 at 4:52 PM, Scott Doctor wrote:

[sqlite] Multiple instances of the same program accessing the same db file

2015-05-02 Thread Scott Doctor
I am somewhat new to sqlite and am trying to decide an issue with the program I am writing (cross platform, written in C/C++). After reading through the sqlite documentation, I am still unsure about the issue how to implement multiple instances of the same program. Consider a program that may