Re: [sqlite] How to create connection life-time object?

[Max Vlasov]

On Fri, Jul 26, 2013 at 9:56 PM, Dušan Paulovič  wrote:

> Thanks for suggestion, but:
> 1.) one object is not linked to one connection
>

If you have your own memory management, it's not a problem since the scheme
I described is basically just a storage of pointers. To free or not to free
(if the pointer points to a disposable entity) is your decision at your
chosen time. But if you want automatic reference counting (so when a
pointer not referenced anymore, it would be automatically deallocated),  I
agree, the proposal is not good. Probably using your own global structure
not related to sqlite is less pain.


> 2.) object is not destroyed together with connection
>

The same, you're free to use the table just as pointers storage

Max
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Any way to debug database is locked?

[David de Regt]

They're all being accessed from multiple threads within a single process app, 
with no external sources/connections.  This happens on both OSX and Win32.  
Always on a local disk.

I'm using the release amalg straight off the website (3.7.17), with 
SQLITE_ENABLE_FTS3/FTS3_PARENTHESIS as preprocessor options.

The database is fine when you restart the app -- it just seems to hit the 
locked error once in a blue moon, which I have set up to assert out right now 
to help me track it down.

-David

-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of Simon Slavin
Sent: Friday, July 26, 2013 5:53 PM
To: General Discussion of SQLite Database
Subject: Re: [sqlite] Any way to debug database is locked?


On 27 Jul 2013, at 1:20am, David de Regt  wrote:

> Is there any chance that the OS is still flushing the WAL changes to disk 
> post-closing the database in another connection, when another connection 
> tries to get an exclusive lock?

How are your various processes contacting the database file ?  Are they all 
running on the computer the database is kept on ?  Or are they using network 
access ?  If so, which NFS ?

Which pragmas have you used ?

Does your database pass the integrity check ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Any way to debug database is locked?

[Simon Slavin]

On 27 Jul 2013, at 1:20am, David de Regt  wrote:

> Is there any chance that the OS is still flushing the WAL changes to disk 
> post-closing the database in another connection, when another connection 
> tries to get an exclusive lock?

How are your various processes contacting the database file ?  Are they all 
running on the computer the database is kept on ?  Or are they using network 
access ?  If so, which NFS ?

Which pragmas have you used ?

Does your database pass the integrity check ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Any way to debug database is locked?

[David de Regt]

We have a fairly complicated system of OS mutexes and using exclusive DB 
transactions to attempt to avoid database locking issues with SQLite.  It works 
great most of the time, but every few days one of our testers randomly runs 
into a database is locked error.  Every time it's been in a debugger, and we've 
checked all threads, none of which have any other open database commands.  So, 
while it's potentially an error in our locking logic, all of the locks are 
stack auto-construct/destructed, that I've been over a bunch of times, so it's 
fairly unlikely it's part of that.  We're using a simple busy handler that just 
returns 1, and we may want to add a sleep(0) into it, but either way, some sort 
of lock issue is occurring.

So, in an attempt to narrow down what's going on, is there any internal sqlite 
logic I can check or query against to see what other thread or even connection 
object currently has the lock on the database, so that I can start building 
some instrumentation to track it down when the issue happens?

Is there any chance that the OS is still flushing the WAL changes to disk 
post-closing the database in another connection, when another connection tries 
to get an exclusive lock?

Any other ideas for me to check?

Thanks for any ideas!
-David
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to create connection life-time object?

[Dušan Paulovič]

Thanks for suggestion, but:
1.) one object is not linked to one connection
2.) object is not destroyed together with connection

Dusan


2013/7/26 Max Vlasov 

> Hi, Dušan
>
>
> On Thu, Jul 25, 2013 at 2:39 PM, Dušan Paulovič 
> wrote:
>
> > Hello, is there a way to somehow set a connection life-time object?
> > ...
> >
> >
> > It would be fine to have something like:
> > int sqlite3_set_lifetime_object(
> >   sqlite3 *db,  /*db connection*/
> >   const char *zObjectName,  /*utf8 name of object*/
> >   void *pObject,/*if NULL, object is removed*/
> >   void(*xDestroy)(void*)/*destructor*/
> > );
> >
> > void * sqlite3_get_lifetime_object(
> >   sqlite3 *db,  /*db connection*/
> >   const char *zObjectName   /*utf8 name of object*/
> > );
> >
>
>
>
> How about temporary memory table just for the task of storing your objects.
>
> You initialization code for particular connection
>   Attach ':memory:' as MyObjectStorage
>   Create table MyObjectStorage.[objects] (Name Text, Ptr Text)
>
> Your code for inserting an object
>   Insert into MyObjectStorage.[objects] (Name, Ptr) VALUES ('obj1',
> '0x12345678')
>
> This code will query the pointer
>   select Ptr from MyObjectStorage.[objects] where Name='obj1'
>
> The only convention rule here will be the name of the attached db so no
> other databases (or instances of the same storage) should use this name.
>
> Max
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLITE to build database from tab delimited file

[Simon Slavin]

On 26 Jul 2013, at 9:25am, Rob Slater  wrote:

> I have an iPad app that uses several (4) different SQLITE databases and I
> currently build these databases using Mozilla Firefox SQLite Manager utility
> (on a Mac).  The databases are simple (single table) with a number of rows,
> and are created from a text file with tab delimited fields.
> 
> This all works but is clunky.  I would prefer to create the SQLITE databases
> on my Windows PC (where the text files are created).
> 
> My question is whether the SQLITE can do this job and how I go about it.  I
> suspect that I should be able to set up a file of SQL commands to build the
> new database(s), to automate the process somewhat

I recommend you use the sqlite3 shell tool, which you can find here



precompiled for Windows.  Documentation is here:



The simplest way to pre-make a database is to make a text file with the SQL 
commands which you would need, separated by semi-colons.  You can then use the 
shell tool to process all those commands in sequence using the '.read' command.

>From your description it might be simplest to write a pre-processor which 
>turns your existing 'text file with tab delimited fields' into SQL commands 
>and write those commands to the end of a text file of SQL commands which 
>creates the database file with blank tables and indexes.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLITE to build database from tab delimited file

[fnoyanisi]

You can write your sql commands in a txt file (say mycmd.txt), and then 

sqlite3 your.databasefile < mycmd.txt

Or 

I have a simple API that lets you import / export csv (not necessarly CSV 
inderd, any kind of delimited file) files into SQLite, so you can do it via 
your own C/C++ application as well.

https://github.com/fnoyanisi/sqlite3_capi_extensions

On 26/07/2013, at 5:55 PM, "Rob Slater"  wrote:

> I have an iPad app that uses several (4) different SQLITE databases and I
> currently build these databases using Mozilla Firefox SQLite Manager utility
> (on a Mac).  The databases are simple (single table) with a number of rows,
> and are created from a text file with tab delimited fields.
> 
> This all works but is clunky.  I would prefer to create the SQLITE databases
> on my Windows PC (where the text files are created).
> 
> My question is whether the SQLITE can do this job and how I go about it.  I
> suspect that I should be able to set up a file of SQL commands to build the
> new database(s), to automate the process somewhat
> 
> Any help much appreciated.
> 
> Rob Slater
> 
> 
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] SQL logic error or missing database with Python sqlite

[C M]

This appears to have been asked many times online, but none of the
threads I've seen help me fix my issue.

I am using SQLite through Python 2.5 or 2.7, which is the sqlite3
module.  In a desktop application, every now and then, and in a fairly
irreproducible way, when committing to the database I get this error:

sqlite3.OperationalError: SQL logic error or missing database

I thought this was a PySqlite generated error, but now I see the same
error is seen with Ruby, PHP, C++ and other languages, so now it seems
it is generated by SQLite itself...but I really don't know.

If I try additional times in that same instance of my app being open,
it gives me the same error every time.  If I close the app and re-open
it, it probably will not give me this error, with the same or very
similar data being written in the same routines.  So I "know" that the
code as written is correct (a significant--greater than 90%?--of the
time I don't see this error).

In terms of what is causing this, I don't know, but I've noticed that
on the occasions that this has happened my computer's RAM was pretty
bogged down.  I am testing all this on a 2004 laptop with a very
fragmented HD with 1 GB of RAM, and I had the following processes
running with this much RAM consumed:

- Firefox.exe: 297,000 K (and climbing...This is FF 17)
- plugin-container.exe:  51,260K.
- Dropbox.exe: 30,212 K.
- explorer.exe:  22,836 K
- pythonw.exe: 18,432 K
- svchost.exe: 16,992 K
- notepad.exe: 5,532 K
- and others...

When there is this much RAM being used, particularly when FF is
hogging it like that, my computer gets quite slow to respond to any
action, such as saving a file (sometimes taking more than 10 seconds
to return responsiveness).

Is it possible that the "SQL logic error or missing database" error is
due to the issue of low available RAM?  Or the fragmented HD?  Or
something like that?  (I get the impression from reading online that
this is a very general error that could be due to just about anything,
and as such gives just about no information).  The problem is, I am
currently sitting with FF over 400,000 K and I just successfully used
my app, so it truly is intermittent.

This error might go away if I used a newer/cleaner/more RAM computer,
but I want to "stress test" my application for those who may be using
similarly clunky computers--I want to try to avoid it even for older
model computers.

Any advice appreciated.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Using SQLITE to build database from tab delimited file

[Rob Slater]

I have an iPad app that uses several (4) different SQLITE databases and I
currently build these databases using Mozilla Firefox SQLite Manager utility
(on a Mac).  The databases are simple (single table) with a number of rows,
and are created from a text file with tab delimited fields.

This all works but is clunky.  I would prefer to create the SQLITE databases
on my Windows PC (where the text files are created).

My question is whether the SQLITE can do this job and how I go about it.  I
suspect that I should be able to set up a file of SQL commands to build the
new database(s), to automate the process somewhat

Any help much appreciated.

Rob Slater



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to create connection life-time object?

[Max Vlasov]

Hi, Dušan


On Thu, Jul 25, 2013 at 2:39 PM, Dušan Paulovič  wrote:

> Hello, is there a way to somehow set a connection life-time object?
> ...
>
>
> It would be fine to have something like:
> int sqlite3_set_lifetime_object(
>   sqlite3 *db,  /*db connection*/
>   const char *zObjectName,  /*utf8 name of object*/
>   void *pObject,/*if NULL, object is removed*/
>   void(*xDestroy)(void*)/*destructor*/
> );
>
> void * sqlite3_get_lifetime_object(
>   sqlite3 *db,  /*db connection*/
>   const char *zObjectName   /*utf8 name of object*/
> );
>



How about temporary memory table just for the task of storing your objects.

You initialization code for particular connection
  Attach ':memory:' as MyObjectStorage
  Create table MyObjectStorage.[objects] (Name Text, Ptr Text)

Your code for inserting an object
  Insert into MyObjectStorage.[objects] (Name, Ptr) VALUES ('obj1',
'0x12345678')

This code will query the pointer
  select Ptr from MyObjectStorage.[objects] where Name='obj1'

The only convention rule here will be the name of the attached db so no
other databases (or instances of the same storage) should use this name.

Max
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Final preparations for the release of System.Data.SQLite v1.0.88.0 have begun...

[Joe Mistachkin]

If you have any issues with the current code, please report them via this
mailing
list (and/or by creating a ticket on "https://system.data.sqlite.org/;)
prior to
next Friday, August 2nd.

Thanks.

--
Joe Mistachkin

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users