Re: [sqlite] Sqlite in dead lock state when deleting records from the same table from different threads

[Simon Slavin]

> On 1 Jul 2014, at 10:36pm, Igor Tandetnik  wrote:
> 
> On 7/1/2014 5:20 PM, Igor Tandetnik wrote:
>> On 7/1/2014 4:55 PM, Clemens Ladisch wrote:
>>> To prevent deadlocks, transactions that will modify the database should
>>> be started with BEGIN IMMEDIATE.  (This kind of lock is not available
>>> in shared cache mode.)
>> 
>> Are you sure? Nothing in the documentation appears to suggest that.
> 
> Upon rereading, this sounds ambiguous. I meant - are you sure BEGIN IMMEDIATE 
> doesn't work in shared cache mode?

My guess is that the OP tested the originally quoted sequence

On 7/1/2014 4:40 PM, Srikanth Bemineni wrote:
> 10:00.234 Thread 1 BEGIN
> 10:00.235 Thread 1 select * from 
> 10:00.234 Thread 1 select * from 
> 10:00.456 Thread 1 delete from 
> 
> 10:00.456 Thread 2 BEGIN
> 10:00.456 Thread 2 select * from 
> 10:00.906 Thread 2 select * from 
> 10:01.156 Thread 2 delete from  

using BEGIN IMMEDIATE instead of BEGIN.  Of course, in this case the second 
BEGIN IMMEDIATE does its job and thread 2 can't get started.  OP saw the second 
BEGIN return an error and concluded that it wasn't working.  Because it's hard 
to see why a SELECT should lock a database until you know what you explained in 
your previous post: within a transaction SELECT must be repeatable.

The solution is to use BEGIN IMMEDIATE as you wrote, but also to set a 
realistic and acceptable value for timeout using the API call



or the PRAGMA



Just a reminder to others that a realistic value for timeout is 2 minutes 
rather than 2 seconds.  Long enough that you are sure that your app has crashed 
rather than is just handling an unusual but reasonable amount of work.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Sqlite in dead lock state when deleting records from the same table from different threads

[Igor Tandetnik]

On 7/1/2014 5:20 PM, Igor Tandetnik wrote:

On 7/1/2014 4:55 PM, Clemens Ladisch wrote:

To prevent deadlocks, transactions that will modify the database should
be started with BEGIN IMMEDIATE.  (This kind of lock is not available
in shared cache mode.)


Are you sure? Nothing in the documentation appears to suggest that.


Upon rereading, this sounds ambiguous. I meant - are you sure BEGIN 
IMMEDIATE doesn't work in shared cache mode?


I didn't mean to cast doubt on the first part - the fact that 
transactions that start as readers and later try to upgrade to writers 
are deadlock-prone.

--
Igor Tandetnik

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Sqlite in dead lock state when deleting records from the same table from different threads

[Igor Tandetnik]

On 7/1/2014 4:55 PM, Clemens Ladisch wrote:

To prevent deadlocks, transactions that will modify the database should
be started with BEGIN IMMEDIATE.  (This kind of lock is not available
in shared cache mode.)


Are you sure? Nothing in the documentation appears to suggest that. But 
I must admit I haven't tried it myself.


If all else fails, one could try and simulate BEGIN IMMEDIATE by running 
a dummy modifying statement right after BEGIN - e.g.


delete from table1 where 0;

--
Igor Tandetnik

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Sqlite in dead lock state when deleting records from the same table from different threads

[Igor Tandetnik]

On 7/1/2014 4:40 PM, Srikanth Bemineni wrote:

10:00.234 Thread 1 BEGIN
10:00.235 Thread 1 select * from 
10:00.234 Thread 1 select * from 
10:00.456 Thread 1 delete from 

10:00.456 Thread 2 BEGIN
10:00.456 Thread 2 select * from 
10:00.906 Thread 2 select * from 
10:01.156 Thread 2 delete from  

Thread 1 SQLITE_LOCKED(6) Error  is locked
Thread 2 SQLITE_LOCKED(6) Error database table is locked

Thread 1 which is the first one to enter BEGIN and do modifications to the
table, and it should have gained the WRITE lock on the table.


It can't - Thread 2 is holding a read lock on that table.


Even If we
consider Thread 2 was performing the select on the same table at the same
time, then Thread 2 should have locked table in its delete call.


It can't - Thread 1 is holding a read lock on that table.


In this
case none of the threads are getting lock on the table and are waiting for
ever.


To be precise, both threads have acquired a read lock on the same table; 
both are now trying and failing to promote it to a write lock.



In each of the thread we are waiting for a random amount of time
re-executing(sqlite3_step) the same prepared statement after calling reset
on the prepared statement.


Won't help: once acquired, all locks are held until the end of 
transaction. You may choose to re-read the same table later in the 
transaction, and SQLite must guarantee that you see the same data. The 
only way to resolve a deadlock is for one thread to roll back its 
transaction.



In any of this case one thread should be the winner in getting the lock on
this table.


Which part of the documentation makes you believe that?


why is this happening for delete when concurrent update
and inserts are happening properly ?


With all due respect, I find this difficult to believe. All other things 
equal, the same problem should manifest if DELETE statement is replaced 
with INSERT or UPDATE. There must be something else different about the 
scenarios where those operations work.

--
Igor Tandetnik

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Sqlite in dead lock state when deleting records from the same table from different threads

[Clemens Ladisch]

Srikanth Bemineni wrote:
> Lately we are seeing a dead lock kind of state while deleting records
> from a table.
>
> All threads open their own shared connection to the database.

Why are you using shared cache mode?

To prevent deadlocks, transactions that will modify the database should
be started with BEGIN IMMEDIATE.  (This kind of lock is not available
in shared cache mode.)


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Sqlite in dead lock state when deleting records from the same table from different threads

[Srikanth Bemineni]

Hi,

We are using the sqlite 3.7.14.1 code in our application.  Lately we are
seeing a dead lock kind of state while deleting records from a table. The
deletion is done two different threads and acting upon the same table.

Sqlite is configured in WAL mode. All threads open their own shared
connection to the database. The application is complied with
SQLITE_THREADSAFE=1 and SQLITE_ENABLE_MEMORY_MANAGEMENT.

m_init = sqlite3_open_v2(
m_dbfilename.toUtf8().data(),
/* Database filename (UTF-8) */
_dbHandler,
/* OUT: SQLite db handle */
SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE |
SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_SHAREDCACHE,/* Flags */
NULL
  /* Name of VFS module to use */
);


All update,insert and  deletion of records in multiple threads happen in
transactions. The update/insert statement works absolutely fine with out
any dead lock. But when two or more threads try to remove an  entry in the
same table, then they go into a dead lock state where sqlite3_step keeps on
returning SQLITE_LOCKED. When we did debug the code


10:00.234 Thread 1 BEGIN
10:00.235 Thread 1 select * from 
10:00.234 Thread 1 select * from 
10:00.456 Thread 1 delete from 

10:00.456 Thread 2 BEGIN
10:00.456 Thread 2 select * from 
10:00.906 Thread 2 select * from 
10:01.156 Thread 2 delete from  

Thread 1 SQLITE_LOCKED(6) Error  is locked
Thread 2 SQLITE_LOCKED(6) Error database table is locked

Thread 1 which is the first one to enter BEGIN and do modifications to the
table, and it should have gained the WRITE lock on the table. Even If we
consider Thread 2 was performing the select on the same table at the same
time, then Thread 2 should have locked table in its delete call. In this
case none of the threads are getting lock on the table and are waiting for
ever. In each of the thread we are waiting for a random amount of time
re-executing(sqlite3_step) the same prepared statement after calling reset
on the prepared statement.

In any of this case one thread should be the winner in getting the lock on
this table.

My question is how can I find who is locking the table. Is there any way I
can get this info ? why is this happening for delete when concurrent update
and inserts are happening properly ?

Srikanth Bemineni
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] 64-bit precompiled command-line shell binary

[RSmith]

Apologies this reply went to the wrong mailbox - rectified here.

On 2014/07/01 20:19, Gert Van Assche wrote:

Thanks for your replay.

I'm not a developer, and I don't know how to compile the source code.
The reason why I think I will need a 64-bit exe is that the 32-bit one is 
limited to 2GB RAM in memory.
As I'll have to read/wrtite a lot to a huge db, I think a 64-bit might help me.

gert



By "help me" I assume you mean setting a cache-size that is sufficiently big to 
speed up database processes of very large size DBs?

The RAM usage is an OS limitation, not one of the program. SQLite (or any other program) can only ask for a memory allocation, which 
may or may not fail. Allocating more than 2GB of memory by just one application (such as sqlite3.exe) is plausible, however I don't 
know if sqlite3.exe or the sqlite code in general is able to manage more than 2GB of memory or more than 32bit addressing (even when 
compiled in 64 bit guise). This is a separate issue to the OS's ability to address more than 2GB memory (It might be able to read 
16GB of memory and still only supply <2GB chunks of it to 32bit apps).


Even then, I think there is a hard upper limit to the cache size that can be 
specified, but cannot find it now - For information see:
http://www.sqlite.org/pragma.html#pragma_cache_size
http://www.sqlite.org/malloc.html


I seem to remember a similar question having surfaced some time ago but cannot find the reference now - so let me ask the great 
forum devs again: WIll the 64-bit version of sqlite3.exe be able to use >2GB of memory for itself, is this different to the 32bit 
version, and if so, does anyone have a pre-compiled 64bit sqlite3.exe version please?



Thanks!
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] 64-bit precompiled command-line shell binary

[RSmith]

On 2014/07/01 18:52, Gert Van Assche wrote:

Maybe a stupid question: how should I use this DLL? just put in the same folder 
as the sqlite3.exe?



Not a stupid question at all - in fact my mistake, the previous request was for the 64-bit DLL and of course any time you compile 
SQLite into your code or compile the existing amalgamation for sqlite3.exe, it will compile whatever your compiler wishes.
The DLL is for linking to applications in a 64-bit Windows OS, but that is NOT sqlite3.exe specifically, which in turn has to be 
compiled to 64bit bit by itself.


Windows 8 and 8.1 should (and does) run the 32bit sqlite3.exe as is and will have negligible effects when changing to 64 bit... is 
there a reason you need it to be 64-bit? That usually doesn't matter at all - which is why it isn't generally available.  Are you 
perhaps meaning windows RT? In which case there is a whole other level of things to be considered apart from just compiling it to 64 
bit.


Still, if you absolutely need a 64 bit version, you can compile it so (the same site has the amalgamation and recently even added 
the detailed steps to take in compiling your own version).


It isn't common though, but I am sure someone on here has done a 64 bit version - generally they only do it to allow/disable 
different compiler switches to the standard version, so they might be able to send you a version, but I doubt there is a standard 
place to get one from - especially not one that is updated in step with the sqlite.org distribution. (I might be wrong though!)






2014-07-01 16:39 GMT+02:00 RSmith >:


On 2014/07/01 14:29, Gert Van Assche wrote:

All,

where could I download a 64-bit sqlite3.exe for running on Windows 8?


From an earlier reply by Richard Hipp:

A 64-bit Windows DLL is now available athttp://www.sqlite.org/download.html 




___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] 64-bit precompiled command-line shell binary

[Gert Van Assche]

Maybe a stupid question: how should I use this DLL? just put in the same
folder as the sqlite3.exe?




2014-07-01 16:39 GMT+02:00 RSmith :

>
> On 2014/07/01 14:29, Gert Van Assche wrote:
>
>> All,
>>
>> where could I download a 64-bit sqlite3.exe for running on Windows 8?
>>
>
> From an earlier reply by Richard Hipp:
>
> A 64-bit Windows DLL is now available athttp://www.sqlite.org/
> download.html
>
>
>
>
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Multiple SQLiteDataReader objects against a single connection

[Joseph L. Casale]

> By doing exactly what you have described.
>
> What is the problem?

Hey Clemens,
Sorry I should have updated the thread, I was receiving an "There is already an 
open
DataReader associated with this Command which must be closed first." exception
which was simply from a lack of paying attention.

I am still reading and trialing approaches as my only previous experience with 
SQLite
was through Python which exposes a different interface. That being said, I 
think I
have a reasonable grasp now of how to apply previously learned Python approaches
in C#.

Thanks for the follow up,
jlc

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Importing a lot of data from many databases

[Gert Van Assche]

Thanks Simon, this is exactly what I needed to know.

gert


2014-07-01 16:48 GMT+02:00 Simon Slavin :

>
> > On 1 Jul 2014, at 12:26pm, Gert Van Assche  wrote:
> >
> > 1 - Open the BIG db, attach all small files, ten by 10, and copy the
> tables
> > from the attached databases to the big table. I can speed up the import
> by
> > putting the INSERT in a transaction.
> >
> > 2 - Export one by one the table from all databases to a CSV file; import
> > all CSV files into the BIG db; I don't have to worry about grouping the
> > files by 10, but I cannot use transaction.
> >
> > What would be the best way to do this? One of these 2 ways, or is there a
> > third way to do this?
>
> First, are you doing this just once, or does it have to become part of a
> workflow for regular use ?
>
> The results of the two should be exactly equivalent.
>
> (1) will be faster.  You can do things like "INSERT INTO tableB SELECT *
> FROM tableA".  This automatically executes the whole thing as one
> transaction.
>
> (2) will be easier to debug.  SQLite doesn't have a built-in function to
> express a row as an INSERT command, but this is part of the function of the
> SQLite shell tool and shell tool scripts can incorporate "BEGIN" and "END"
> commands.
>
> If I was doing it just once I'd probably write a script for the shell tool
> that did (2).  If I had to implement it as part of a frequently used
> workflow I'd probably write code which did (1).
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Multiple SQLiteDataReader objects against a single connection

[Clemens Ladisch]

Joseph L. Casale wrote:
> How does one accomplish this in the case where I iterate over a long result 
> set
> with the first reader open, then open a new reader against a prepared 
> statement
> and pass in a value derived from the first reader.

By doing exactly what you have described.

What is the problem?


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: signal 6 during sqlite_step in WAL mode

[Simon Slavin]

On 1 Jul 2014, at 2:50pm, Mattan Shalev  wrote:

> Valgrind only shows "
> 
> Warning: set address range perms: large range [0x7dc88040, 0x8f5d2058)
> (undefined)", which is a debug msg for developers.
> 
> 
> Also, forgot to mention that the reader threads are initiated via gSoap,
> though the signal always occur from sqlite.

It still looks like a corrupted heap.  The assumption is that something in your 
app is writing over SQLite's memory space between SQLite API calls.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Importing a lot of data from many databases

[Simon Slavin]

> On 1 Jul 2014, at 12:26pm, Gert Van Assche  wrote:
> 
> 1 - Open the BIG db, attach all small files, ten by 10, and copy the tables
> from the attached databases to the big table. I can speed up the import by
> putting the INSERT in a transaction.
> 
> 2 - Export one by one the table from all databases to a CSV file; import
> all CSV files into the BIG db; I don't have to worry about grouping the
> files by 10, but I cannot use transaction.
> 
> What would be the best way to do this? One of these 2 ways, or is there a
> third way to do this?

First, are you doing this just once, or does it have to become part of a 
workflow for regular use ?

The results of the two should be exactly equivalent.

(1) will be faster.  You can do things like "INSERT INTO tableB SELECT * FROM 
tableA".  This automatically executes the whole thing as one transaction.

(2) will be easier to debug.  SQLite doesn't have a built-in function to 
express a row as an INSERT command, but this is part of the function of the 
SQLite shell tool and shell tool scripts can incorporate "BEGIN" and "END" 
commands.

If I was doing it just once I'd probably write a script for the shell tool that 
did (2).  If I had to implement it as part of a frequently used workflow I'd 
probably write code which did (1).

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] 64-bit precompiled command-line shell binary

[RSmith]

On 2014/07/01 14:29, Gert Van Assche wrote:

All,

where could I download a 64-bit sqlite3.exe for running on Windows 8?


From an earlier reply by Richard Hipp:

A 64-bit Windows DLL is now available athttp://www.sqlite.org/download.html







___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] think I need better error-handling guidance in the C API

[Simon Slavin]

On 1 Jul 2014, at 2:22am, Keith Medcalf  wrote:

> IF ResultCode == A_OK YipeeKiAiii else OhShitItDidntWork;
> 
> Seems pretty straightforward to me.  You handle the result codes you know 
> what to do with, and everything that remains means your program should 
> explode immediately and very very loudly dumping ALL relevant information 
> (ie, error code, statement, inout data, output data, etc.)
> 
> In other words, there is no point checking for error conditions which you 
> cannot handle. Instead you just die.

Okay, right.  Now, where in the documentation does it tell us "what to do with" 
each error ?  Or alternatively, how should my app die ?

Suppose _step() gets a result of SQLITE_PROTOCOL.  Can I just execute _step() 
again and assume I didn't miss a row ?  Or will the next _step() always just 
return SQLITE_DONE ?  Should I actually _reset() and start again ?  Or is my 
statement so messed up I can only _finalize() it ?  Is it necessary to 
_finalize() it ?  Or is my whole database connection now invalid and I should 
quit without having to even _close() ?

It's this sort of documentation which is not present.  And it's that sort of 
thing that I think Eric is asking about.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] think I need better error-handling guidance in the C API

[Eric Rubin-Smith]

Keith Medcalf wrote: 

> IF ResultCode == A_OK YipeeKiAiii else OhShitItDidntWork; 
> 
> Seems pretty straightforward to me.  You handle the result codes you 
> know what to do with, 

There is a difference between things I don't personally know how to 
handle and things that the SQLite authors would consider "permanent" 
errors (where "permanent" is appropriately scoped e.g.  to the level of 
the function call, the sqlite3_stmt*, the sqlite3*, or so on).  

My ignorance of the API details is not a justification for writing a 
crappy client.  I would like to cure my ignorance, and the docs are 
insufficient in that regard AFAICT.  Hence my question to this forum.  

> and everything that remains means your program should explode 
> immediately and very very loudly dumping ALL relevant information (ie, 
> error code, statement, inout data, output data, etc.)  

SQLite is not the center of my universe.  The layer using it is not at 
the center of the program in which it resides.  The program I am writing 
is a high-availability, mission-critical system.  Barfing because of an 
SQLite error is unacceptable in my case, especially if the only reason 
the program is barfing is because the programmer who wrote it was a 
dummy (or at least more of a dummy than he should have been:-).  

And whether it barfs is sort of beside the point.  If I restart my 
program and give SQLite the same series of commands, should I expect 
it to fail in the exact same way, or should I expect its behavior 
to change?  Depends of course on the state of the system and on the 
particular series of calls.  If the documentation were sufficient, then 
it would allow me to reach the appropriate conclusion.  

I would first like to know exactly what the *authors* expect to be 
returned in various cases, so that I can code to that expectation.  

> In other words, there is no point checking for error conditions 
> which you cannot handle.  Instead you just die.  Quickly, noisily, 
> and immediately.  There is no point checking for the error message 
> SQLITE3_CPU_REGISTER_HIT_BY_COSMIC_RAY if there is nothing that you can 
> do about it.  Every error that you cannot handle is a fatal error, and you 
> should die accordingly.  

I can handle every kind of error.  It's just a question of how.  I 
don't know how in many cases, because I don't understand the semantics 
of the error codes, in turn because (I think) the documentation is 
insufficient.  It is not healthy to bake such ignorance into the 
application if I can help it -- again, I would rather just cure my 
ignorance (preferably not by digging through the code and reaching
my own flawed conclusions about it).  

-- 
Eric A. Rubin-Smith

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: signal 6 during sqlite_step in WAL mode

[Mattan Shalev]

Valgrind only shows "

Warning: set address range perms: large range [0x7dc88040, 0x8f5d2058)
(undefined)", which is a debug msg for developers.


Also, forgot to mention that the reader threads are initiated via gSoap,
though the signal always occur from sqlite.


On Tue, Jul 1, 2014 at 11:58 AM, Dan Kennedy  wrote:

> On 07/01/2014 12:07 PM, Mattan Shalev wrote:
>
>> Hey guys,
>> I'm getting signal 6 during sqlite_step in WAL mode. Working on Ubuntu
>> 12.04, sqlite3 3.7.9.
>> One process is the writing continuously,  while other process reads from
>> the DB in a multi threaded access. I made sure that sqlite is configured
>> to
>> serialised mode.
>>
>
> Looks like a corrupted heap. If you run your app under valgrind it will
> probably point you to the problem.
>
> Dan.
>
>
>
>
>
>
>> Here is the backtrace:
>>
>> #0 0x7f4f78f0d08d in nanosleep () from /lib/x86_64-linux-gnu/libc.so.
>> 6
>> #1 0x7f4f78f0cf2c in sleep () from /lib/x86_64-linux-gnu/libc.so.6
>> #2 0x7f4f7982d881 in signal_catch (signo=6) at signal_masker.c:35
>> #3 
>> #4 0x7f4f78e83425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
>> #5 0x7f4f78e86b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
>> #6 0x7f4f78ec139e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>> #7 0x7f4f78ecbb96 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
>> #8 0x7f4f77661690 in sqlite3_free () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #9 0x7f4f77664b96 in ?? () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #10 0x7f4f77664c7c in ?? () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #11 0x7f4f77664cb0 in ?? () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #12 0x7f4f7768b477 in ?? () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #13 0x7f4f7769a249 in ?? () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #14 0x7f4f776a9689 in ?? () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #15 0x7f4f776ae451 in sqlite3_step () from
>> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
>> #16 0x7f4f79416557 in get_down_intervals_size (start_time=1403711040,
>> end_time=1577836800,
>> size=0x7f3c49c68958) at stats_ext.c:202
>> #17 0x00407f80 in ns__get_stats_performance (s=0x1e4cb00,
>> start_time=1403711040, end_time=1577836800,
>> time_unit=NS_MINUTES, resource_id=0, ret=0x7f3c49c68990) at
>> msg_handlers/capi_wrappers.c:636
>> #18 0x004461c1 in soap_serve_ns__get_stats_performance
>> (soap=0x1e4cb00) at soapServer.c:1294
>> #19 0x004415ae in soap_serve_request (soap=0x1e4cb00) at
>> soapServer.c:119
>> #20 0x00440f3e in soap_serve (soap=0x1e4cb00) at soapServer.c:39
>> #21 0x00403c90 in process_request (s=0x1e4cb00) at be_server.c:111
>> #22 0x7f4f79a3de9a in start_thread () from
>> /lib/x86_64-linux-gnu/libpthread.so.0
>> #23 0x7f4f78f413fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
>> #24 0x in ?? ()
>>
>> Does anyone has an idea?
>>
>>
>> Thanks!
>>
>> Mattan.
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Mattan Shalev | Software Engineer

*Reduxio Systems*


2 Granit St. | Petach Tikva 4951446, Israel

Mobile: +972 (52) 5115750

mat...@reduxio.com  | www.reduxio.com
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] 64-bit precompiled command-line shell binary

[Gert Van Assche]

All,

where could I download a 64-bit sqlite3.exe for running on Windows 8?

Thanks,

Gert
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Importing a lot of data from many databases

[Gert Van Assche]

All,

I'm using the windows shell and I have to import one table from 50.000
small sqlite files into one big sqlite file.

These are the 2 options I see:

1 - Open the BIG db, attach all small files, ten by 10, and copy the tables
from the attached databases to the big table. I can speed up the import by
putting the INSERT in a transaction.

2 - Export one by one the table from all databases to a CSV file; import
all CSV files into the BIG db; I don't have to worry about grouping the
files by 10, but I cannot use transaction.

What would be the best way to do this? One of these 2 ways, or is there a
third way to do this?

thanks

gert
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] error "malformed database schema" on view in attached table

[Gert Van Assche]

Bingo! That was indeed the problem, Kevin.

I'm sorry I did not find that user group posting.

Thanks.


2014-07-01 11:30 GMT+02:00 Kevin Benson :

> On Tue, Jul 1, 2014 at 5:07 AM, Gert Van Assche  wrote:
>
> > All,
> >
> > I get this error message:
> >
> > malformed database schema (WrongTargetLang) - view [WrongTargetLang]
> cannot
> > reference objects in database main
> >
> > when I try to attach a database containing a view. If I delete the view,
> I
> > get this message for another view. I get that message for all views.
> >
> > I can open that DB without any problem, but when I open it as an attached
> > db in the sqlite3 shell (in windows), this happens.
> >
> > Anyone any idea what is happening?
> >
> > thanks
> >
> > gert
>
>
> Perhaps you're referencing- main -in your view?
> http://comments.gmane.org/gmane.comp.db.sqlite.general/51366
>
> --
>--
>   --
>  --Ô¿Ô--
> K e V i N
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] signal 6 during sqlite_step in WAL mode

[Mattan Shalev]

Hey guys,
I'm getting signal 6 during sqlite_step in WAL mode. Working on Ubuntu
12.04, sqlite3 3.7.9.
One process is the writing continuously,  while other process reads from
the DB in a multi threaded access. I made sure that sqlite is configured to
serialised mode.

Here is the backtrace:

#0  0x7f4f78f0d08d in nanosleep () from
/lib/x86_64-linux-gnu/libc.so.6#1  0x7f4f78f0cf2c in sleep () from
/lib/x86_64-linux-gnu/libc.so.6#2  0x7f4f7982d881 in signal_catch
(signo=6) at signal_masker.c:35#3  #4
0x7f4f78e83425 in raise () from /lib/x86_64-linux-gnu/libc.so.6#5
0x7f4f78e86b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6#6
0x7f4f78ec139e in ?? () from /lib/x86_64-linux-gnu/libc.so.6#7
0x7f4f78ecbb96 in ?? () from /lib/x86_64-linux-gnu/libc.so.6#8
0x7f4f77661690 in sqlite3_free () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0#9  0x7f4f77664b96 in ??
() from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0#10
0x7f4f77664c7c in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0#11 0x7f4f77664cb0 in ??
() from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0#12
0x7f4f7768b477 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0#13 0x7f4f7769a249 in ??
() from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0#14
0x7f4f776a9689 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0#15 0x7f4f776ae451 in
sqlite3_step () from /usr/lib/x86_64-linux-gnu/libsqlite3.s

Does anyone has an idea?


Thanks!

Mattan.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Fwd: Regarding SQlite support for accessing documents library in windows store app.

[Vedika Neha Shabraya]

Hi,

I have a windows metro app which uses SQLite database.

I want to create and write to a SQLite db file present in user's documents
library.

I am able to do so only in the app's local data folder and app's installed
location which I do not want. But unable to so so in the users documents
library.

 Even though I have permissions to access the user's documents folder, when
I run the app, I get an exception of type 'SQLite.SQLiteException'  ; Could
not open database file: C:\\Users\\\\ Mydata.s3db (CannotOpen)

Would be very grateful if you could please let me know how this can be
fixed.

Thanks

Vedika
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Aggragate functions

[James Nash]

Suggestions for aggregate functions:

median(),  percentile()
reason - this functions are costly due to time to transfer.  Would be 
nice if they were computed locally
 
Suggestion for function to deprecate (keep for backward compatibility):

total()
reason - cost to distinguish 0.0 from null is small and good 
programming should deal with the question
reason - mathematically the sum is not defined if there is no data.

___
James Nash   +46 70-160 29 54,   home  +46 8-437 541 01




___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] error "malformed database schema" on view in attached table

[Richard Hipp]

Can you please send us the complete schema (as shown by the ".schema"
command in the command-line shell) for both the original database and the
database you are trying to attach?


On Tue, Jul 1, 2014 at 5:07 AM, Gert Van Assche  wrote:

> All,
>
> I get this error message:
>
> malformed database schema (WrongTargetLang) - view [WrongTargetLang] cannot
> reference objects in database main
>
> when I try to attach a database containing a view. If I delete the view, I
> get this message for another view. I get that message for all views.
>
> I can open that DB without any problem, but when I open it as an attached
> db in the sqlite3 shell (in windows), this happens.
>
> Anyone any idea what is happening?
>
> thanks
>
> gert
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] error "malformed database schema" on view in attached table

[Kevin Benson]

On Tue, Jul 1, 2014 at 5:07 AM, Gert Van Assche  wrote:

> All,
>
> I get this error message:
>
> malformed database schema (WrongTargetLang) - view [WrongTargetLang] cannot
> reference objects in database main
>
> when I try to attach a database containing a view. If I delete the view, I
> get this message for another view. I get that message for all views.
>
> I can open that DB without any problem, but when I open it as an attached
> db in the sqlite3 shell (in windows), this happens.
>
> Anyone any idea what is happening?
>
> thanks
>
> gert


Perhaps you're referencing- main -in your view?
http://comments.gmane.org/gmane.comp.db.sqlite.general/51366

--
   --
  --
 --Ô¿Ô--
K e V i N
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] error "malformed database schema" on view in attached table

[Gert Van Assche]

All,

I get this error message:

malformed database schema (WrongTargetLang) - view [WrongTargetLang] cannot
reference objects in database main

when I try to attach a database containing a view. If I delete the view, I
get this message for another view. I get that message for all views.

I can open that DB without any problem, but when I open it as an attached
db in the sqlite3 shell (in windows), this happens.

Anyone any idea what is happening?

thanks

gert
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: signal 6 during sqlite_step in WAL mode

[Dan Kennedy]

On 07/01/2014 12:07 PM, Mattan Shalev wrote:

Hey guys,
I'm getting signal 6 during sqlite_step in WAL mode. Working on Ubuntu
12.04, sqlite3 3.7.9.
One process is the writing continuously,  while other process reads from
the DB in a multi threaded access. I made sure that sqlite is configured to
serialised mode.


Looks like a corrupted heap. If you run your app under valgrind it will 
probably point you to the problem.


Dan.







Here is the backtrace:

#0 0x7f4f78f0d08d in nanosleep () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7f4f78f0cf2c in sleep () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x7f4f7982d881 in signal_catch (signo=6) at signal_masker.c:35
#3 
#4 0x7f4f78e83425 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#5 0x7f4f78e86b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x7f4f78ec139e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#7 0x7f4f78ecbb96 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#8 0x7f4f77661690 in sqlite3_free () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#9 0x7f4f77664b96 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#10 0x7f4f77664c7c in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#11 0x7f4f77664cb0 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#12 0x7f4f7768b477 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#13 0x7f4f7769a249 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#14 0x7f4f776a9689 in ?? () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#15 0x7f4f776ae451 in sqlite3_step () from
/usr/lib/x86_64-linux-gnu/libsqlite3.so.0
#16 0x7f4f79416557 in get_down_intervals_size (start_time=1403711040,
end_time=1577836800,
size=0x7f3c49c68958) at stats_ext.c:202
#17 0x00407f80 in ns__get_stats_performance (s=0x1e4cb00,
start_time=1403711040, end_time=1577836800,
time_unit=NS_MINUTES, resource_id=0, ret=0x7f3c49c68990) at
msg_handlers/capi_wrappers.c:636
#18 0x004461c1 in soap_serve_ns__get_stats_performance
(soap=0x1e4cb00) at soapServer.c:1294
#19 0x004415ae in soap_serve_request (soap=0x1e4cb00) at
soapServer.c:119
#20 0x00440f3e in soap_serve (soap=0x1e4cb00) at soapServer.c:39
#21 0x00403c90 in process_request (s=0x1e4cb00) at be_server.c:111
#22 0x7f4f79a3de9a in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#23 0x7f4f78f413fd in clone () from /lib/x86_64-linux-gnu/libc.so.6
#24 0x in ?? ()

Does anyone has an idea?


Thanks!

Mattan.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Fwd: signal 6 during sqlite_step in WAL mode

[Eduardo Morras]

On Tue, 1 Jul 2014 08:07:52 +0300
Mattan Shalev  wrote:

> Hey guys,
> I'm getting signal 6 during sqlite_step in WAL mode. Working on Ubuntu
> 12.04, sqlite3 3.7.9.
> One process is the writing continuously,  while other process reads
> from the DB in a multi threaded access. I made sure that sqlite is
> configured to serialised mode.
> 
> Here is the backtrace:
> 
> #0 0x7f4f78f0d08d in nanosleep ()
> #from /lib/x86_64-linux-gnu/libc.so.6 1 0x7f4f78f0cf2c in sleep
> #() from /lib/x86_64-linux-gnu/libc.so.6 2 0x7f4f7982d881 in
> #signal_catch (signo=6) at signal_masker.c:35 3  #called> 4 0x7f4f78e83425 in raise ()
> #called> from /lib/x86_64-linux-gnu/libc.so.6
> #5 0x7f4f78e86b8b in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #6 0x7f4f78ec139e in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #7 0x7f4f78ecbb96 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #8 0x7f4f77661690 in sqlite3_free () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #9 0x7f4f77664b96 in ?? () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #10 0x7f4f77664c7c in ?? () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #11 0x7f4f77664cb0 in ?? () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #12 0x7f4f7768b477 in ?? () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #13 0x7f4f7769a249 in ?? () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #14 0x7f4f776a9689 in ?? () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #15 0x7f4f776ae451 in sqlite3_step () from
> /usr/lib/x86_64-linux-gnu/libsqlite3.so.0
> #16 0x7f4f79416557 in get_down_intervals_size
> #(start_time=1403711040,
> end_time=1577836800,
> size=0x7f3c49c68958) at stats_ext.c:202
> #17 0x00407f80 in ns__get_stats_performance (s=0x1e4cb00,
> start_time=1403711040, end_time=1577836800,
> time_unit=NS_MINUTES, resource_id=0, ret=0x7f3c49c68990) at
> msg_handlers/capi_wrappers.c:636
> #18 0x004461c1 in soap_serve_ns__get_stats_performance
> (soap=0x1e4cb00) at soapServer.c:1294
> #19 0x004415ae in soap_serve_request (soap=0x1e4cb00) at
> soapServer.c:119
> #20 0x00440f3e in soap_serve (soap=0x1e4cb00) at
> #soapServer.c:39 21 0x00403c90 in process_request
> #(s=0x1e4cb00) at be_server.c:111 22 0x7f4f79a3de9a in
> #start_thread () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #23 0x7f4f78f413fd in clone ()
> #from /lib/x86_64-linux-gnu/libc.so.6 24 0x in ?? ()
> 
> Does anyone has an idea?

Don't know with the information you show. But I suggest you to not use 
sqlite3.so from system and add sqlite3.c and sqlite3.h to your project. 
Therefore you can, among other improvements, build a newer and more error free 
sqlite3 version with debug information on and see what happens between the #8 
sqlite3_free() call and #14 sqlite3_step(). Increase in compile time is minimal 
(less than 2 seconds) in my laptop in a fresh rebuild.

> 
> Thanks!
> 
> Mattan.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


---   ---
Eduardo Morras 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] hoe to create index to a big table - resolution

[Hadashi, Rinat]

Hi

Thanks Gerd and Dan and Ryan.

I received answers that helped me understand and solve my problem by 
redirecting temp data to a directory rather than in-memory.
I tried Dan's proposal to SETENV TMPDIR and afterwards I could successfully 
create index.
I will try Gerd's pragma proposal as well

Thanks
Rinat

-Original Message-
From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] 
On Behalf Of GB
Sent: Monday, June 30, 2014 9:15 PM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] hoe to create index to a big table?

Do you possibly have redirected temporary storage to memory, either by compile 
switch or #pragma temp_store? If so, try explicitly setting #pragma 
temp_store=1, this will force temporary data to be stored on disk.

Gerd.

Hadashi, Rinat schrieb am 30.06.2014 10:30:
> I work with a very big table, on Linux.
> I fail to create index getting the following error:
> SQL error near line 1: database or disk is full
>
> Any advice?
>
> Rinat Hadashi
>
>
> -
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for 
> the sole use of the intended recipient(s). Any review or distribution 
> by others is strictly prohibited. If you are not the intended 
> recipient, please contact the sender and delete all copies.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
-
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users