Re: [sqlite] help needed for major SQLite problem

[Stephen Chrzanowski]

Information on how to open SQLite files:
http://www.sqlite.org/c3ref/open.html
How the locking mechanisms work: http://sqlite.org/lockingv3.html

Specifically, it'll depend on the language or wrapper you're using to
access the database.  In my case (Delphi) there is an option in the open
function that will allow me to specify exactly HOW I want to open a file.
If you use an exclusive lock, no other application will be able to do
anything with it.  DropBox will complain about it, but it won't be able to
do anything about it.  However, the question remains if or how the WAL file
will get written if your main database is open exclusively.

As I said before, I've used DB for a long while, been the EXCLUSIVE user on
this particular database, I'm never writing to the database at anytime
other than the machine I happen to be sitting at.  I've only ever
exclusively locked a file to one program because there COULD be a chance
that I'd accidentally run multiple versions of the program in production
mode, and I'd rather the error out on the second app rather than duplicate
writes. If I realize that I have a lock on a file in my version control
system at home, and I'm at work, I'll log into my machine at home, check in
the changes, then check out at work, and continue on.  However, if I'm at
home and realize what I want to work on is at work... err.. well I'll
go blow up some Kerbals (Kerbal Space Program) or drive manically into the
back end of the competition in Gran Tourismo 5 and take the day off.  ;)



On Wed, Feb 12, 2014 at 12:59 PM, C M  wrote:

>
>
> > I'd suggest looking into opening the database with an exclusive lock,
>
>
> Can you say more about that?  I'm not familiar with that option.
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Mon, Feb 10, 2014 at 4:40 PM, Stephen Chrzanowski wrote:

> Personally, I don't buy that DropBox is the culprit as I've done this kind
> of thing a few times in a few applications of my own, however, I'm the
> single user that works on that single account, and any app that uses DB is
> usually under development and "closed" on any other geographical site.
>

That is the same situation with me; I only develop and use this
application, and thus this database, on one computer only.  I haven't tried
the Dropbox syncing in at least two years, though it used to work.

But I'm getting the sense from others here responding to this post that
Dropbox does put a lock on the database file in order to back it up, and it
has to do that whenever the data is changed, and so one might wind up
having Dropbox lock the file when you need to write to it.  Oddly, I make
changes to the database hundreds of times a day and only get this lock
conflict--if that is indeed what is happening--every now and then, like
every few weeks or months, particularly when the RAM is filled up with a
memory leaky Firefox and other open applications...possibly because the
lock is held longer during those times?


> I'd suggest looking into opening the database with an exclusive lock,


Can you say more about that?  I'm not familiar with that option.



> or look into using the Backup API that SQLite uses.
>
Using the SQLite Backup API, when your program starts, do a flat-file
> standard file copy from DropBox to a different location (%USERPATH% or
> something similar) and wait for the file to finish to copy.  Then open that
> backed up file and work on it.  When your user saves, or closes your
> application, use the backup API to put the file back to DropBox directory.
> This isolates WAL file writes to the local system and not to DB.
>

That sounds similar to what James Lowden is suggesting in this thread.
I'll look into that, and will probably have questions for the list later.

Thanks,
Che
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[James K. Lowden]

On Mon, 10 Feb 2014 16:20:40 -0500
C M  wrote:

> But this must be a fairly commonly sought need.  The solution you
> propose where I occasionally export a copy of the db to Dropbox is
> great *for backup purposes*  but seems to exclude the possibility of
> syncing across multiple computers.  So what would you recommend?

Rethink the problem.  :-)

The problem is not, actually, how to sync a SQLite database across
multiple computers.  The problem is, how to make data in a SQLite
database available at multiple locations, right?  

In fact, the problem may be stated more generally, without reference to
SQLite.  

If you look at it that way, one solution might be to leave the database
in one place, and route queries to that place.  Another might be to use
a DBMS with a "replication" feature that maintains synchronized copies
of the database.  Or build such a feature yourself.  

It's my understanding that Firefox uses SQLite for its bookmarks, and
supports keeping bookmarks in the cloud.  

Hacks borrow and artists steal.  

HTH.  

--jkl
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Simon Slavin]

On 10 Feb 2014, at 9:20pm, C M  wrote:

> I purposefully put the SQlite database file in the Dropbox folder because
> it was my intention, with this app, to allow a user to use the app on more
> than one computer and "sync" the database via Dropbox.  E.g., s/he could
> make changes to the db at home and then also from his/her office computer,
> get home, and the database would be synced.  I tried this out on two
> computers at home and it seemed to be sort of working, but I occasionally
> got conflicted copies and yet never pursued the right way to do it.
> 
> But this must be a fairly commonly sought need.  The solution you propose
> where I occasionally export a copy of the db to Dropbox is great *for
> backup purposes*  but seems to exclude the possibility of syncing across
> multiple computers.  So what would you recommend?

Dropbox needs to lock the files it is writing until it has finished writing 
them.  This is a perfectly reasonably request to make of an application which 
plays with other application's data files.

This is not something that can be fixed inside SQLite.  To do so would prevent 
all multi-access activity for SQLite databases.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[RSmith]

On 2014/02/10 23:40, Stephen Chrzanowski wrote:

Personally, I don't buy that DropBox is the culprit as I've done this kind
of thing a few times in a few applications of my own, however, I'm the
single user that works on that single account, and any app that uses DB is
usually under development and "closed" on any other geographical site.
However, with the chance that the WAL file makes it down the wire, I could
see that as being a problem if your application spits out small bits of
data periodically and is being run in multiple places.  DropBox, for me at
least, realizes that another application has a handle on the database file
and won't touch it, or overwrite anything.  But the WAL file might make it
through.


Yes, Dropbox realises when YOU are locking the file, the problem comes when you try to open the file while IT is locking it - 
something that will only really happen on an update (upload) or back-sync (download when it was changed elsewhere).


Heh, just try to open the DB on another computer it is sync'ed with while you have it open on your computer, then change stuff in 
both systems and close the apps (to release any SQLite locks)... You will quickly realise what a culprit Dropbox can be. Now imagine 
it for many users...


Further to this, imagine the WAL file (or any other of the temp files) gets synced without the DB file... or vice-versa, or you get 
another user's newer WAL file overwriting your own - a very probable situation.


[Note: This is not drop-box's fault, it has to do what it needs to do to ensure 
sharing tactics that suits most syncing needs]





___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[RSmith]

On 2014/02/10 23:20, C M wrote:

On Mon, Feb 10, 2014 at 2:54 PM, RSmith  wrote:


How to go from the error codes to the diagnosis? I think the logic is as
follows:
[lots of snipping]

Thanks for this insight.

I purposefully put the SQlite database file in the Dropbox folder because
it was my intention, with this app, to allow a user to use the app on more
than one computer and "sync" the database via Dropbox.  E.g., s/he could
make changes to the db at home and then also from his/her office computer,
get home, and the database would be synced.  I tried this out on two
computers at home and it seemed to be sort of working, but I occasionally
got conflicted copies and yet never pursued the right way to do it.

But this must be a fairly commonly sought need.  The solution you propose
where I occasionally export a copy of the db to Dropbox is great *for
backup purposes*  but seems to exclude the possibility of syncing across
multiple computers.  So what would you recommend?


This is something all the systems we make do all the time, and I am sure is a common thing with most people on this list, so we 
definitely understand the need to sync. Problem is Dropbox (or any of the other ones) is a file-syncing solution and not suitable 
for direct DB syncs for reasons we covered already.


With most database syncing we just use actual cloud servers and cloud syncs with an actual database, i.e we have several cloud 
servers running typically Apache, PHP, MySQL, SQLite, etc. to which all systems connect and either upload or download additions, 
changes and the like. Simple versioning within the tables keep track of who updated most recently on a per-record basis, sometimes 
per-field for really critical apps, but this is uncommon.


What you essentially trying to do here is allow another system to take care of the syncing for you with the very real benefit that 
you don't need to support a cloud server. The downside is that you have to play by the rules of whatever service is used.


I will be honest here, for every suggestion I can come up with, I can find a scenario where it would break in a cloud-sync 
environment. We actually have one system that uses cloud syncing through external service (Dropbox, Skydrive, Google drive, whatever 
the user uses), but the we use custom data files that play nice with the services, the SQLite (or any other RDBMS) files simply 
won't work. This means some extra work. Basically we create a computer-based key (hash), write all changes from the DB to a file 
with that key as part of the name in the sync folder (MyDocs/Dropbox/ or whatever the user sets up). Any other system with which it 
is shared sees that file appear in its syncing folder, and as soon as it becomes available (checking every few seconds or so), opens 
it, adds the updates contained within it if those updates are newer than it's own last changes for the specified records/keys.  In 
turn, it will write a file containing its own newest changes etc.


SO any system connected to the cloud sync, sees all files created by all other computers, there may be many, but they are typically 
very small. Every system incorporates updates from every other system's file if it is newer, but only ever creates its own update 
file for its own changes and only really does that on a timely basis when the file is not locked. Every time updates are exported 
only the newest updates are included and most important, what actually makes it all work, is that every such file has only 1 writer, 
nobody else changes it - so you won't ever get the file trying to sync-back to your own folder or other related problems.


A headache we have is that one of the participant systems might go down for several days, user on holiday, whatever, and then it is 
so far out of date that the update files it sees no longer contains some of the stuff it missed, in which case the user is prompted 
to get a full DB backup from another user, and they can email it or dropbox it as another file or whatever.


ALL of these shenanigans simply because of the way the cloud syncs work - a rather cumbersome work-around, but hey, the 
cloud-syncing is not on us, so a win.


I tried to be scant with details and give basic ideas because different approaches might be better depending on the exact nature of 
the data and frequency of changes and importance of updates etc. etc. but feel free to ask off-list if needing more detail lest we 
bore the others.



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Stephen Chrzanowski]

Personally, I don't buy that DropBox is the culprit as I've done this kind
of thing a few times in a few applications of my own, however, I'm the
single user that works on that single account, and any app that uses DB is
usually under development and "closed" on any other geographical site.
However, with the chance that the WAL file makes it down the wire, I could
see that as being a problem if your application spits out small bits of
data periodically and is being run in multiple places.  DropBox, for me at
least, realizes that another application has a handle on the database file
and won't touch it, or overwrite anything.  But the WAL file might make it
through.

I'd suggest looking into opening the database with an exclusive lock, or
look into using the Backup API that SQLite uses.

Using the SQLite Backup API, when your program starts, do a flat-file
standard file copy from DropBox to a different location (%USERPATH% or
something similar) and wait for the file to finish to copy.  Then open that
backed up file and work on it.  When your user saves, or closes your
application, use the backup API to put the file back to DropBox directory.
This isolates WAL file writes to the local system and not to DB.


On Mon, Feb 10, 2014 at 4:20 PM, C M  wrote:

> On Mon, Feb 10, 2014 at 2:54 PM, RSmith  wrote:
>
> > How to go from the error codes to the diagnosis? I think the logic is as
> > follows:
> >
> > We can see an error occurs when trying to access the file, or more
> > specifically, trying to obtain a shared lock on it. This means it is
> locked
> > by another application (as opposed to another SQLite thread). Now the
> > question remains which other application? We would usually simply suggest
> > to look in your system, but you already provided a log of the error, and
> it
> > is clear from the error that a file you are trying to access is in
> > "Documents\My Dropbox\myapp\" which, as everyone knows, is a dropbox
> > folder, which means likely you have dropbox installed. Secondly, Dropbox
> is
> > a known culprit in this regard, because it syncs files with the cloud (it
> > is not the only one, Skydrive, Google drive etc all do this), which means
> > it will have to lock a file while either uploading or download-syncing it
> > for consistency and concurrency reasons.  Put these three pieces of
> > evidence together, and the answer is inevitable - you probably have
> dropbox
> > problems.
> >
> > The remedy is not easy - same as when dealing with Excel exports or some
> > other system that will lock files of it's own volition if it is opened
> with
> > that system - simply making a byte-copy of the file, changing it and
> > replacing it afterwards, with a possible replace-queue facility which
> will
> > wait till a lock is released. Problem is, what if the other app made
> > changes that you actually mean to keep?
> >
> > To put this into your perspective, what if the file was dropboxed,
> altered
> > on another machine of the user's, or by another user (through a dropbox
> > share), and is now updated in the cloud and due to sync back?  Whatever
> > solution, versioning-control or other system you come up with to handle
> > this, it has to be full of user-informative messages and you can never
> keep
> > an editable file where locking might be a problem inside a dropbox (or
> > other locking+syncing) folder.
> >
> > It is better to have a DB file (meaning a file that gets small
> incremental
> > changes over time as opposed to a load-once, save-once methodology) in a
> > place that is not affected by locks, and sometimes exporting (using the
> > SQLIte backup facility maybe) to the dropbox or shared folder so it gets
> > propagated to the cloud...  Using it within that folder is just not
> > feasible.
> >
>
> Thanks for this insight.
>
> I purposefully put the SQlite database file in the Dropbox folder because
> it was my intention, with this app, to allow a user to use the app on more
> than one computer and "sync" the database via Dropbox.  E.g., s/he could
> make changes to the db at home and then also from his/her office computer,
> get home, and the database would be synced.  I tried this out on two
> computers at home and it seemed to be sort of working, but I occasionally
> got conflicted copies and yet never pursued the right way to do it.
>
> But this must be a fairly commonly sought need.  The solution you propose
> where I occasionally export a copy of the db to Dropbox is great *for
> backup purposes*  but seems to exclude the possibility of syncing across
> multiple computers.  So what would you recommend?
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Mon, Feb 10, 2014 at 2:54 PM, RSmith  wrote:

> How to go from the error codes to the diagnosis? I think the logic is as
> follows:
>
> We can see an error occurs when trying to access the file, or more
> specifically, trying to obtain a shared lock on it. This means it is locked
> by another application (as opposed to another SQLite thread). Now the
> question remains which other application? We would usually simply suggest
> to look in your system, but you already provided a log of the error, and it
> is clear from the error that a file you are trying to access is in
> "Documents\My Dropbox\myapp\" which, as everyone knows, is a dropbox
> folder, which means likely you have dropbox installed. Secondly, Dropbox is
> a known culprit in this regard, because it syncs files with the cloud (it
> is not the only one, Skydrive, Google drive etc all do this), which means
> it will have to lock a file while either uploading or download-syncing it
> for consistency and concurrency reasons.  Put these three pieces of
> evidence together, and the answer is inevitable - you probably have dropbox
> problems.
>
> The remedy is not easy - same as when dealing with Excel exports or some
> other system that will lock files of it's own volition if it is opened with
> that system - simply making a byte-copy of the file, changing it and
> replacing it afterwards, with a possible replace-queue facility which will
> wait till a lock is released. Problem is, what if the other app made
> changes that you actually mean to keep?
>
> To put this into your perspective, what if the file was dropboxed, altered
> on another machine of the user's, or by another user (through a dropbox
> share), and is now updated in the cloud and due to sync back?  Whatever
> solution, versioning-control or other system you come up with to handle
> this, it has to be full of user-informative messages and you can never keep
> an editable file where locking might be a problem inside a dropbox (or
> other locking+syncing) folder.
>
> It is better to have a DB file (meaning a file that gets small incremental
> changes over time as opposed to a load-once, save-once methodology) in a
> place that is not affected by locks, and sometimes exporting (using the
> SQLIte backup facility maybe) to the dropbox or shared folder so it gets
> propagated to the cloud...  Using it within that folder is just not
> feasible.
>

Thanks for this insight.

I purposefully put the SQlite database file in the Dropbox folder because
it was my intention, with this app, to allow a user to use the app on more
than one computer and "sync" the database via Dropbox.  E.g., s/he could
make changes to the db at home and then also from his/her office computer,
get home, and the database would be synced.  I tried this out on two
computers at home and it seemed to be sort of working, but I occasionally
got conflicted copies and yet never pursued the right way to do it.

But this must be a fairly commonly sought need.  The solution you propose
where I occasionally export a copy of the db to Dropbox is great *for
backup purposes*  but seems to exclude the possibility of syncing across
multiple computers.  So what would you recommend?
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[RSmith]

On 2014/02/10 21:18, C M wrote:

Documents\My Dropbox\myapp\gorp.db-journal) - Access is denied. (3338)
SQLITE_IOERR

SQLITE_LOG: statement aborts at 16: [SELECT resumes, start FROM
WHERE start='2014-02-07 14:24:14.064000' AND value='activity'] disk I/O
error (3338) SQLITE_IOERR

Looks like GetFileAttributesEx() might be throwing an ERROR_ACCESS_DENIED
exception. Maybe a virus scanner or some other background process had
temporarily locked the database file.

Dan.

I agree, and I think Dropbox is the culprit here.

May I ask either Dan or Kees, or anyone here, how to go from the error
codes to that diagnosis?

Kees, why do you think Dropbox is the culprit?

I may want to deploy this app to users who would also backup their database
by having it in the Dropbox folder.  What would people suggest I do about
this?


How to go from the error codes to the diagnosis? I think the logic is as 
follows:

We can see an error occurs when trying to access the file, or more specifically, trying to obtain a shared lock on it. This means it 
is locked by another application (as opposed to another SQLite thread). Now the question remains which other application? We would 
usually simply suggest to look in your system, but you already provided a log of the error, and it is clear from the error that a 
file you are trying to access is in "Documents\My Dropbox\myapp\" which, as everyone knows, is a dropbox folder, which means likely 
you have dropbox installed. Secondly, Dropbox is a known culprit in this regard, because it syncs files with the cloud (it is not 
the only one, Skydrive, Google drive etc all do this), which means it will have to lock a file while either uploading or 
download-syncing it for consistency and concurrency reasons.  Put these three pieces of evidence together, and the answer is 
inevitable - you probably have dropbox problems.


The remedy is not easy - same as when dealing with Excel exports or some other system that will lock files of it's own volition if 
it is opened with that system - simply making a byte-copy of the file, changing it and replacing it afterwards, with a possible 
replace-queue facility which will wait till a lock is released. Problem is, what if the other app made changes that you actually 
mean to keep?


To put this into your perspective, what if the file was dropboxed, altered on another machine of the user's, or by another user 
(through a dropbox share), and is now updated in the cloud and due to sync back?  Whatever solution, versioning-control or other 
system you come up with to handle this, it has to be full of user-informative messages and you can never keep an editable file where 
locking might be a problem inside a dropbox (or other locking+syncing) folder.


It is better to have a DB file (meaning a file that gets small incremental changes over time as opposed to a load-once, save-once 
methodology) in a place that is not affected by locks, and sometimes exporting (using the SQLIte backup facility maybe) to the 
dropbox or shared folder so it gets propagated to the cloud...  Using it within that folder is just not feasible.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Kees Nuyt]

On Mon, 10 Feb 2014 14:18:18 -0500, C M 
wrote:

>On Sat, Feb 8, 2014 at 4:28 AM, Kees Nuyt  wrote:
>>
>> On Sat, 08 Feb 2014 12:06:01 +0700, Dan Kennedy
>>  wrote:
>
>
>
>> >> SQLITE_LOG: delayed 1375ms for lock/sharing conflict (10) SQLITE_IOERR
>> >>
>> >> SQLITE_LOG: os_win.c:35129: (5) winAccess(C:\Documents and
>Settings\user\My
>> >> Documents\My Dropbox\myapp\gorp.db-journal) - Access is denied. (3338)
>> >> SQLITE_IOERR
>> >>
>> >> SQLITE_LOG: statement aborts at 16: [SELECT resumes, start FROM
>Durations
>> >> WHERE start='2014-02-07 14:24:14.064000' AND value='activity'] disk I/O
>> >> error (3338) SQLITE_IOERR
>
>
>> >Looks like GetFileAttributesEx() might be throwing an ERROR_ACCESS_DENIED
>> >exception. Maybe a virus scanner or some other background process had
>> >temporarily locked the database file.
>> >
>> >Dan.
>>
>> I agree, and I think Dropbox is the culprit here.
>
>May I ask either Dan or Kees, or anyone here, how to go from the error
>codes to that diagnosis?
>
>Kees, why do you think Dropbox is the culprit?

Because the log indicates that the database and its journal are
located in a dropbox directory:

"SQLITE_LOG: os_win.c:35129: (5) winAccess(C:\Documents and
Settings\user\My Documents\My Dropbox\myapp\gorp.db-journal) -
Access is denied. (3338) SQLITE_IOERR"

Given that name, I just assumed you are using dropbox. Dropbox
will try to synchronise files in that directory with its copy in
the cloud whenever its contents have changed.
In an effort to make a consistent copy, dropbox will probably
lock the file during the copy operation.

Both the database and its journal will change frequently when
you are using it, and dropbox will try yo keep up.

You may want to temporarily disable dropbox to avoid this
unwanted cpncurrent access, or perhaps restrict dropbox to time
slots when you don't use the database.

>I may want to deploy this app to users who would also backup their database
>by having it in the Dropbox folder.  What would people suggest I do about
>this?

Perhaps you could put the database somewhere else and
periodically use the sqlite3_backup() interface to save a copy
to the dropbox folder.


>Thanks again to all.  (btw, I now view my original subject line as a bit
>much; I was just frustrated by it happening so randomly and without the
>ability to fix it)

-- 
Groet, Cordialement, Pozdrawiam, Regards,

Kees Nuyt

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Simon Slavin]

On 10 Feb 2014, at 7:18pm, C M  wrote:

> I may want to deploy this app to users who would also backup their database
> by having it in the Dropbox folder.  What would people suggest I do about
> this?

Don't run the app while Dropbox is messing with its datafile.

The problem is with Dropbox, which can copy stuff between folder at any time.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sat, Feb 8, 2014 at 4:28 AM, Kees Nuyt  wrote:
>
> On Sat, 08 Feb 2014 12:06:01 +0700, Dan Kennedy
>  wrote:



> >> SQLITE_LOG: delayed 1375ms for lock/sharing conflict (10) SQLITE_IOERR
> >>
> >> SQLITE_LOG: os_win.c:35129: (5) winAccess(C:\Documents and
Settings\user\My
> >> Documents\My Dropbox\myapp\gorp.db-journal) - Access is denied. (3338)
> >> SQLITE_IOERR
> >>
> >> SQLITE_LOG: statement aborts at 16: [SELECT resumes, start FROM
Durations
> >> WHERE start='2014-02-07 14:24:14.064000' AND value='activity'] disk I/O
> >> error (3338) SQLITE_IOERR


> >Looks like GetFileAttributesEx() might be throwing an ERROR_ACCESS_DENIED
> >exception. Maybe a virus scanner or some other background process had
> >temporarily locked the database file.
> >
> >Dan.
>
> I agree, and I think Dropbox is the culprit here.

May I ask either Dan or Kees, or anyone here, how to go from the error
codes to that diagnosis?

Kees, why do you think Dropbox is the culprit?

I may want to deploy this app to users who would also backup their database
by having it in the Dropbox folder.  What would people suggest I do about
this?

Thanks again to all.  (btw, I now view my original subject line as a bit
much; I was just frustrated by it happening so randomly and without the
ability to fix it)
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Kees Nuyt]

On Sat, 08 Feb 2014 12:06:01 +0700, Dan Kennedy
 wrote:

>On 02/08/2014 03:00 AM, C M wrote:
>> This is a follow-up to a question I asked on this list on Sep 1st, 2013,
>> about an error that I was randomly getting with disk-based SQLite database
>> in a Python desktop application.  I now have more info to provide about the
>> error...such as what was asked for at that time:
>>
>> On Sun, Sep 1, 2013 at 6:12 PM, Richard Hipp  wrote:
>>
>>> Does Python have an interface to the error and warning log mechanism of
>>> SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?  It
>>> will probably give more details about what it happening.
>>>
>> I wasn't able to do this at first, but thanks to switching from the
>> standard sqlite3 module in Python (otherwise known as pysqslite) to Roger
>> Binns's APSW module, and then also using an experimental module,
>> apswdbapi2, from Edzard Pasma (thank you both), I was able to set things up
>> to return the warning log...I think.  Today, after not seeing the error in
>> a very long time, I hit the error, and this was printed to sys.stdout:
>>
>> SQLITE_LOG: delayed 1375ms for lock/sharing conflict (10) SQLITE_IOERR
>>
>> SQLITE_LOG: os_win.c:35129: (5) winAccess(C:\Documents and Settings\user\My
>> Documents\My Dropbox\myapp\gorp.db-journal) - Access is denied. (3338)
>> SQLITE_IOERR
>>
>> SQLITE_LOG: statement aborts at 16: [SELECT resumes, start FROM Durations
>> WHERE start='2014-02-07 14:24:14.064000' AND value='activity'] disk I/O
>> error (3338) SQLITE_IOERR
>>
>> Does that give anyone a better idea of what could be happening and how I
>> can fix this problem?
>
>Looks like GetFileAttributesEx() might be throwing an ERROR_ACCESS_DENIED
>exception. Maybe a virus scanner or some other background process had
>temporarily locked the database file.
>
>Dan.

I agree, and I think Dropbox is the culprit here.


-- 
Groet, Cordialement, Pozdrawiam, Regards,

Kees Nuyt

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Dan Kennedy]

On 02/08/2014 03:00 AM, C M wrote:

This is a follow-up to a question I asked on this list on Sep 1st, 2013,
about an error that I was randomly getting with disk-based SQLite database
in a Python desktop application.  I now have more info to provide about the
error...such as what was asked for at that time:

On Sun, Sep 1, 2013 at 6:12 PM, Richard Hipp  wrote:


Does Python have an interface to the error and warning log mechanism of
SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?  It
will probably give more details about what it happening.


I wasn't able to do this at first, but thanks to switching from the
standard sqlite3 module in Python (otherwise known as pysqslite) to Roger
Binns's APSW module, and then also using an experimental module,
apswdbapi2, from Edzard Pasma (thank you both), I was able to set things up
to return the warning log...I think.  Today, after not seeing the error in
a very long time, I hit the error, and this was printed to sys.stdout:

SQLITE_LOG: delayed 1375ms for lock/sharing conflict (10) SQLITE_IOERR

SQLITE_LOG: os_win.c:35129: (5) winAccess(C:\Documents and Settings\user\My
Documents\My Dropbox\myapp\gorp.db-journal) - Access is denied. (3338)
SQLITE_IOERR

SQLITE_LOG: statement aborts at 16: [SELECT resumes, start FROM Durations
WHERE start='2014-02-07 14:24:14.064000' AND value='activity'] disk I/O
error (3338) SQLITE_IOERR

Does that give anyone a better idea of what could be happening and how I
can fix this problem?


Looks like GetFileAttributesEx() might be throwing an ERROR_ACCESS_DENIED
exception. Maybe a virus scanner or some other background process had
temporarily locked the database file.

Dan.


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

This is a follow-up to a question I asked on this list on Sep 1st, 2013,
about an error that I was randomly getting with disk-based SQLite database
in a Python desktop application.  I now have more info to provide about the
error...such as what was asked for at that time:

On Sun, Sep 1, 2013 at 6:12 PM, Richard Hipp  wrote:

> Does Python have an interface to the error and warning log mechanism of
> SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?  It
> will probably give more details about what it happening.
>

I wasn't able to do this at first, but thanks to switching from the
standard sqlite3 module in Python (otherwise known as pysqslite) to Roger
Binns's APSW module, and then also using an experimental module,
apswdbapi2, from Edzard Pasma (thank you both), I was able to set things up
to return the warning log...I think.  Today, after not seeing the error in
a very long time, I hit the error, and this was printed to sys.stdout:

SQLITE_LOG: delayed 1375ms for lock/sharing conflict (10) SQLITE_IOERR

SQLITE_LOG: os_win.c:35129: (5) winAccess(C:\Documents and Settings\user\My
Documents\My Dropbox\myapp\gorp.db-journal) - Access is denied. (3338)
SQLITE_IOERR

SQLITE_LOG: statement aborts at 16: [SELECT resumes, start FROM Durations
WHERE start='2014-02-07 14:24:14.064000' AND value='activity'] disk I/O
error (3338) SQLITE_IOERR

Does that give anyone a better idea of what could be happening and how I
can fix this problem?

Thanks to all again,
Che M
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Mon, Sep 2, 2013 at 3:28 PM, E.Pasma  wrote:

> Op 2 sep 2013, om 17:58 heeft C M het volgende geschreven:
>
>> ...
>>
>> Is setting up APSW and making the transition from pysqlite2's wrapper
>> fairly straightforward, or would I have to re-do all the database queries?
>> (Even so, I do them all in one utilities library, so perhaps it would be
>> not too much work).
>> ...
>>
> The transition from Pythons builtin SQLite wrapper to APSW is not too
> hard. But you must know that APSW assumes isolation_level = None (pysqlite
> connect parameter).
>
> I wrote an experimental module, apswdbapi2, that lets you try APSW in a
> pysqlite compatible mode including the isolation_level.
>
> http://pythonhosted.org/**sqmediumlite/src/apswdbapi2.**py.html


Thanks, that looks quite interesting.  I will be in touch with Roger
regarding APSW and if and once I get set up with that, I will give this a
try as well.  I may have a few questions for you if I try it.

-Che
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/09/13 08:58, C M wrote:
> If you think APSW would provide more details about just what went
> wrong other than "SQLite logic error or missing database" (which, I
> feel, doesn't tell me much at all), then I could potentially try it.

I'm in the process of adding error log support and should have it in a few
days for you to try.  You are welcome to email me directly or use the
python-sqlite mailing list until the Python side of this is nailed down.

"SQLite logic error or missing database" is the string associated with
SQLITE_ERROR code which is used in a lot of places.

SQLite does have extended codes that give more detail.  pysqlite doesn't
provide them but APSW does:

  http://apidoc.apsw.googlecode.com/hg/exceptions.html

> Is setting up APSW and making the transition from pysqlite2's wrapper 
> fairly straightforward, or would I have to re-do all the database
> queries? (Even so, I do them all in one utilities library, so perhaps
> it would be not too much work).

The actual querying etc is the same.  (pysqlite borrowed how some of it is
done from APSW :-)  Note that pysqlite parses your SQL and tries to second
guess what is going with transactions, and starts them behind your back.
That is possibly causing problems for you too.

Roger


-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIk8J8ACgkQmOOfHg372QTYCQCgi79D27GRMc6Tgjc8a+lI0Gt2
7IUAoOHGHlTKTJDmle82l5d98nWozmJ8
=PA/r
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[E.Pasma]

Op 2 sep 2013, om 17:58 heeft C M het volgende geschreven:

...
Is setting up APSW and making the transition from pysqlite2's wrapper
fairly straightforward, or would I have to re-do all the database  
queries?
(Even so, I do them all in one utilities library, so perhaps it  
would be

not too much work).
...
The transition from Pythons builtin SQLite wrapper to APSW is not too  
hard. But you must know that APSW assumes isolation_level = None  
(pysqlite connect parameter).


I wrote an experimental module, apswdbapi2, that lets you try APSW in  
a pysqlite compatible mode including the isolation_level.


http://pythonhosted.org/sqmediumlite/src/apswdbapi2.py.html


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sun, Sep 1, 2013 at 6:12 PM, Richard Hipp  wrote:

> Does Python have an interface to the error and warning log mechanism of
> SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?  It
> will probably give more details about what it happening.
>

Looks like, no, though maybe Roger Binn's AWSW does allow that; I'm looking
into it.  Thanks for your help!
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sun, Sep 1, 2013 at 7:51 PM, Roger Binns  wrote:

>
> He needs to give the exception traceback which will show what is happening
> at the time.
>
> A common mistake with newish Python programmers is to catch all
> exceptions, and then keep going which also hides the exception tracebacks.
>

I have taken out the try/except catch and will hope this error occurs soon
and report back with the offending line.  I don't think it will tell me
much, though, other than the "SQlite logic error or missing database" and
the SELECT or INSERT INTO query that triggered that.  Keep in mind, these
exact same queries *do work* if I just wait a few seconds and try again, so
there is nothing wrong with them per se.



>
> On 01/09/13 15:12, Richard Hipp wrote:
> > Does Python have an interface to the error and warning log mechanism
> > of SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?
> > It will probably give more details about what it happening.
>
> pysqlite is the standard Python sqlite3 module and doesn't expose that
> functionality.  (It also hasn't had a new release in over two years.)  It
> tries to make SQLite look like the standard Python DBAPI and maps the
> SQLite errors into those fewer DBAPI exceptions.
>
> APSW (disclosure: I am the author) does provide the SQLite errors in a 1:1
> mapping to exceptions.  The error log could be used in theory, but is
> impractical because it has to be setup before any other SQLite call.
> There doesn't appear to be any reason why it can't be changed after the
> library is initialised.
>
> I'll see if I can add something more useful to the next APSW release.
>

If you think APSW would provide more details about just what went wrong
other than "SQLite logic error or missing database" (which, I feel, doesn't
tell me much at all), then I could potentially try it.

Is setting up APSW and making the transition from pysqlite2's wrapper
fairly straightforward, or would I have to re-do all the database queries?
(Even so, I do them all in one utilities library, so perhaps it would be
not too much work).

That's an impressive project, btw!  I'm surprised this is the first I've
heard of it.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Mon, Sep 2, 2013 at 2:33 AM, Markus Schaber wrote:

> Hi, C M,
>
>
> Just to be safe: Can you roule out any antivirus or other security
> software locking the files temporarily?
>

Good idea, but I don't think that is what is happening in my case.  I
actually have a way to catch the "Database is locked" exception and
instruct the user appropriately (it is easy to lock a database by making a
change with SQLite Database Browser but not saving it).  Also, I don't use
security software on this PC other than web browsing security.

That is something to keep in mind for other users, though.  Thanks.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sun, Sep 1, 2013 at 6:53 PM, Simon Slavin  wrote:

>
> On 1 Sep 2013, at 10:34pm, C M  wrote:
>
> > Do you know how I can do that with Python?  For example, I tried this:
> >
> > status = cursor.execute("some SQL statement here")
> > print "The status is: ", status
> >
> > But it prints the cursor object:
> >
> >> The status is 
>
> Does the cursor object have properties ?  Can you either look through them
> in a debugger or find documentation somewhere that tells you if one of them
> is something like 'last error' ?
>
> There should be a way to do it: the ability to read the code returned by
> API calls is essential to using SQLite properly.
>

I don't know.  I am looking into this, and may consider trying APSW as the
wrapper instead of pysqlite2 (see Roger Binns's post) to see if I can get
that information.

With regard to your other posts about this problem, my guess is that you
> have some underlying error causing this that isn't part of SQLite.  The
> expected cause of this error is if someone deletes your database file (or a
> journal) while you have the database open.  There are other errors which
> can make the file handle invalid.  But I suspect that just as you wrote
> about your RAM, you actually have a hardware or OS problem which is the
> real culprit.
>

That very well may be, but I want to be able to respond appropriately in
the application if this happens to a user, who may also have a similar
hardware problem (in that PCs that run slow or are bogged down with RAM use
from Firefox, e.g., are incredibly common).

One "hypothesis"  (this is totally naive, so may be really off base) I have
is that if the CPU use / RAM use is high, then the various processes are
not getting their chance to write to the hard drive, and so if SQLite tries
to write it is put in a queue and is getting "timed out".  Then, if I try
again in a few seconds, now the other processes have had a chance to write
and SQLite gets write access right away.  If that is the case, that seems
like something that can be worked around...I'm just not sure how to
determine if it is the case and how to work around it if it is.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Markus Schaber]

Hi, C M,


Im Auftrag von C M
> On Sun, Sep 1, 2013 at 4:28 PM, jose isaias cabrera
> wrote:
> > "C M" wrote...
> >  Keeping it simple:
> >>
> >> I have a Python application that uses SQLite, and I randomly get this
> >> error:
> >>
> >> "SQL logic error or missing database"
> >>
> >>  Is the database in network drive or not in the same machine that is
> > running the app?
> >
> 
> The database is a file on the same hard drive that the app is on.  It's all
> on a one laptop.

Just to be safe: Can you roule out any antivirus or other security software 
locking the files temporarily?



Best regards

Markus Schaber

CODESYS(r) a trademark of 3S-Smart Software Solutions GmbH

Inspiring Automation Solutions

3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. +49-831-54031-979 | Fax +49-831-54031-50

E-Mail: m.scha...@codesys.com | Web: http://www.codesys.com | CODESYS store: 
http://store.codesys.com
CODESYS forum: http://forum.codesys.com

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade 
register: Kempten HRB 6186 | Tax ID No.: DE 167014915

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

He needs to give the exception traceback which will show what is happening
at the time.

A common mistake with newish Python programmers is to catch all
exceptions, and then keep going which also hides the exception tracebacks.

On 01/09/13 15:12, Richard Hipp wrote:
> Does Python have an interface to the error and warning log mechanism
> of SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?
> It will probably give more details about what it happening.

pysqlite is the standard Python sqlite3 module and doesn't expose that
functionality.  (It also hasn't had a new release in over two years.)  It
tries to make SQLite look like the standard Python DBAPI and maps the
SQLite errors into those fewer DBAPI exceptions.

APSW (disclosure: I am the author) does provide the SQLite errors in a 1:1
mapping to exceptions.  The error log could be used in theory, but is
impractical because it has to be setup before any other SQLite call.
There doesn't appear to be any reason why it can't be changed after the
library is initialised.

I'll see if I can add something more useful to the next APSW release.

Note that the APSW API is similar to pysqlite, but not the same.  It
behaves the SQLite way rather than the DBAPI way.

  http://apidoc.apsw.googlecode.com/hg/pysqlite.html

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIj0vYACgkQmOOfHg372QSYrwCcCuJyMqsUiS5SYVFpeYll72d+
Z0oAoJSoWZNanNAxoBMUzjXHMfyEFmj1
=eyCs
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/13 14:34, C M wrote:
> Do you know how I can do that with Python?  For example, I tried this:
> 
> status = cursor.execute("some SQL statement here") print "The status
> is: ", status
> 
> But it prints the cursor object:
> 
>> The status is 

cursor.execute returns an iterator of the results which is the cursor itself.

What you need to provide is the traceback which will show what calls were
active.

The exception is generally caused by opening a file that isn't a database,
but can be caused by other rare sequences of api calls.

Roger

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlIj0AIACgkQmOOfHg372QTU5QCfeGcle8r3m8nDs3dDgI7zb6BK
qWQAoLWkeKarpubX4S1SzcBxaT5LT1yr
=sjZY
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Simon Slavin]

On 1 Sep 2013, at 10:34pm, C M  wrote:

> Do you know how I can do that with Python?  For example, I tried this:
> 
> status = cursor.execute("some SQL statement here")
> print "The status is: ", status
> 
> But it prints the cursor object:
> 
>> The status is 

Does the cursor object have properties ?  Can you either look through them in a 
debugger or find documentation somewhere that tells you if one of them is 
something like 'last error' ?

There should be a way to do it: the ability to read the code returned by API 
calls is essential to using SQLite properly.

With regard to your other posts about this problem, my guess is that you have 
some underlying error causing this that isn't part of SQLite.  The expected 
cause of this error is if someone deletes your database file (or a journal) 
while you have the database open.  There are other errors which can make the 
file handle invalid.  But I suspect that just as you wrote about your RAM, you 
actually have a hardware or OS problem which is the real culprit.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Richard Hipp]

Does Python have an interface to the error and warning log mechanism of
SQLite?  (http://www.sqlite.org/errlog.html)  Can you turn that on?  It
will probably give more details about what it happening.


On Sun, Sep 1, 2013 at 5:32 PM, C M  wrote:

> On Sun, Sep 1, 2013 at 1:38 PM, Richard Hipp  wrote:
>
> > On Sun, Sep 1, 2013 at 1:12 PM, C M  wrote:
> >
> > >
> > > I have a Python application that uses SQLite, and I randomly get this
> > > error:
> > >
> > > "SQL logic error or missing database"
> > >
> > > I have no idea how to figure out what's wrong,
> > >
> >
> > A good starting place might be to tell us what the program is doing when
> > the error comes back.
> >
>
> I'm not 100% sure, because there are several SQLite calls that it could be
> tripping up on at that part of the program's use.  But all of them are
> basic SQL, and at the point in the code where I see the error, it is either
> SELECT or INSERT INTO statements.  (If it becomes necessary, I can try to
> put in a way to know right where it happens, though I'm not sure how to do
> that...see my response to Simon Slavin).
>
> Importantly, the exact same part of the code will work on one occasion, and
> on another occasion with the same "state" of the application, will not
> work.  That is, I can run my application, try something basic from a fresh
> start, this error will occur, close the application, try the same exact
> thing again, and that time it will work fine.  It occurs rather
> infrequently, but the point is I want to make sure it never occurs.
>
> The only thing I've noticed is that *maybe* it is more likely to happen
> when my laptop's RAM is pretty filled up with other processes (such as when
> Firefox runs high RAM and lots of other applications are open).  Could that
> put us on the trail of what could be going wrong?
>
> Thanks!
> Che
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sun, Sep 1, 2013 at 4:28 PM, jose isaias cabrera
wrote:

>
> "C M" wrote...
>
>
>  Keeping it simple:
>>
>> I have a Python application that uses SQLite, and I randomly get this
>> error:
>>
>> "SQL logic error or missing database"
>>
>>  Is the database in network drive or not in the same machine that is
> running the app?
>

The database is a file on the same hard drive that the app is on.  It's all
on a one laptop.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sun, Sep 1, 2013 at 2:23 PM, Simon Slavin  wrote:

>
> On 1 Sep 2013, at 6:38pm, Richard Hipp  wrote:
>
> > A good starting place might be to tell us what the program is doing when
> > the error comes back.
>
> As well as telling us the call that trieggers the error, please put logic
> into your program so that it not only checks the result code of the call
> that generates the error but also checks to see that all earlier SQLite
> calls return SQLITE_OK when you expect them to.  Often the call that
> returns the error is after the one that caused the problem.
>

Do you know how I can do that with Python?  For example, I tried this:

status = cursor.execute("some SQL statement here")
print "The status is: ", status

But it prints the cursor object:

> The status is 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[C M]

On Sun, Sep 1, 2013 at 1:38 PM, Richard Hipp  wrote:

> On Sun, Sep 1, 2013 at 1:12 PM, C M  wrote:
>
> >
> > I have a Python application that uses SQLite, and I randomly get this
> > error:
> >
> > "SQL logic error or missing database"
> >
> > I have no idea how to figure out what's wrong,
> >
>
> A good starting place might be to tell us what the program is doing when
> the error comes back.
>

I'm not 100% sure, because there are several SQLite calls that it could be
tripping up on at that part of the program's use.  But all of them are
basic SQL, and at the point in the code where I see the error, it is either
SELECT or INSERT INTO statements.  (If it becomes necessary, I can try to
put in a way to know right where it happens, though I'm not sure how to do
that...see my response to Simon Slavin).

Importantly, the exact same part of the code will work on one occasion, and
on another occasion with the same "state" of the application, will not
work.  That is, I can run my application, try something basic from a fresh
start, this error will occur, close the application, try the same exact
thing again, and that time it will work fine.  It occurs rather
infrequently, but the point is I want to make sure it never occurs.

The only thing I've noticed is that *maybe* it is more likely to happen
when my laptop's RAM is pretty filled up with other processes (such as when
Firefox runs high RAM and lots of other applications are open).  Could that
put us on the trail of what could be going wrong?

Thanks!
Che
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[jose isaias cabrera]

"C M" wrote...


Keeping it simple:

I have a Python application that uses SQLite, and I randomly get this 
error:


"SQL logic error or missing database"

Is the database in network drive or not in the same machine that is running 
the app? 


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Simon Slavin]

On 1 Sep 2013, at 6:38pm, Richard Hipp  wrote:

> A good starting place might be to tell us what the program is doing when
> the error comes back.

As well as telling us the call that trieggers the error, please put logic into 
your program so that it not only checks the result code of the call that 
generates the error but also checks to see that all earlier SQLite calls return 
SQLITE_OK when you expect them to.  Often the call that returns the error is 
after the one that caused the problem.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] help needed for major SQLite problem

[Richard Hipp]

On Sun, Sep 1, 2013 at 1:12 PM, C M  wrote:

>
> I have a Python application that uses SQLite, and I randomly get this
> error:
>
> "SQL logic error or missing database"
>
> I have no idea how to figure out what's wrong,
>

A good starting place might be to tell us what the program is doing when
the error comes back.


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] help needed for major SQLite problem

[C M]

Keeping it simple:

I have a Python application that uses SQLite, and I randomly get this error:

"SQL logic error or missing database"

I have no idea how to figure out what's wrong, and if I can't figure it
out, it leaves a huge deal-breaking bug in my application, such that I'd
have to abandon SQlite and use another RDMS.  I'd hate that, as I love
SQLite and it is integrated deeply with my application.  But I can't have a
database application that occasionally just doesn't work, and I've put
years of work into this project and really need it to be robust.

Any help on this could potentially make a huge positive difference in my
efforts.  Thank you!
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users