Re: [sqlite] sqlite3 crashes mysteriously on 3.6.20-1

[Richard Hipp]

On 11/10/16, pisymbol .  wrote:
> sqlite-3.6.20-1.el6_7.2.i686
> sqlite-3.6.20-1.el6_7.2.x86_64

Are you aware that SQLite 3.6.20 was published over 7 years ago on
2009-11-04 and that there have been 82 subsequent releases of SQLite
and that the newer releases run more than twice as fast?

Maybe the right solution is updated to SQLite 3.15.1?

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] sqlite3 crashes mysteriously on 3.6.20-1

[pisymbol .]

Hi:

I'm on Centos 6.8 latest, x86-84.

$ uname -a
Linux  (stubbed out) 2.6.32-642.3.1.el6.x86_64 #1 SMP Tue Jul 12 18:30:56
UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ rpm -qa sqlite
sqlite-3.6.20-1.el6_7.2.i686
sqlite-3.6.20-1.el6_7.2.x86_64

I have a strange issue that I was hoping familiar with sqlite3 innards
could explain to me.

I have the following application that does this:

Thread 1 - sqlite3_exec("SELECT column FROM table1;")

Thread 2 - sqlite3_exec("BEGIN EXCLUSIVE;") -> sqlite3_exec("iNSERT INTO
table2 VALUES (number)" -> sqlite3_exec("COMMIT;")

My queries are very basic, take no callback, and are valid at the time of
execution (verified by gdb).
Also, both have separate handles to the database, i.e. both use
sqlite3_open() to establish a connection.

When these two threads execute at the same time however, it appears that
sqlite3 SIGSEGV's deep in the heart of prepare, step, etc.

However, if I open the database with SQLITE3_NOMUTEX|SQLITE3_PRIVATECACHE
using sqlite3_open_v2() on 3.6.20-1 everything works (this was suggested by
a colleague) OR if I download the latest version (3.15 etc.) and set my
LD_LIBRARY_PATH to use it, no crash.

Were their patches in this area by chance that did not get into 3.6.20-1
(which was built fairly recently by the CentOS folks) which may cause a
crash? Or am I using it sqlite3 open incorerctly (assuming that multiple
threads can all call sqlite3_open() freely and go to town).

Thanks!

-aps
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] import-prefixed symbols in sqlite3.obj file

[Max Vlasov]

On Thu, Nov 10, 2016 at 1:51 PM, Dominique Devienne  wrote:
> On Thu, Nov 10, 2016 at 11:38 AM, Max Vlasov  wrote:
>
>> __imp_EnterCriticalSection
>
>
> Is this post [1] related to your issue?
> Maybe you're not using the 64-bit SDK too? --DD

thanks for mentioning. As this topic and many other discussions at the
web are c++ linker related. So mostly it's about setting the correct
path/library information when the linker knows what different prefixes
mean. I suppose that in my case delphi linker understands only
non-prefixed symbol entries. Even if I failed switching to the correct
SDK, then I'd get many additional unresolved external entries, but in
my case other windows symbols used by sqlite are correctly resolved
when linking.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite ext3 settings

[Simon Slavin]

On 10 Nov 2016, at 10:03am, Dan Kennedy  wrote:

> It seems to be that if your drive has write-caching enabled, you need 
> barrier=1 to avoid risking corruption. Or you can get away with barrier=0 if 
> write-caching is disabled.

To supplement Dan's answer, having write-caching enabled is always a problem 
with ACID storage, unless it's defeated by other settings.  Because the thing 
which looks at the cache and writes it to disk is not limited to writing writes 
in order.  So you cannot guarantee that your changes are written to disk in the 
order they were made.

Illustration: Suppose your program modifies the contents of these sectors in 
order:

A B C D B E

When it gets to the second 'B' the mechanism which caches writes notices that 
it already has data for B and just replaces that data in the cache.  The first 
write to B never happens.  Instead, even if it happens to write all the other 
sectors in order it actually writes one of these two sequences, depending on 
how it's programmed:

A B C D E
A C D B E

In both sequences the first the change to sector C gets written before the 
first change to sector B.  And power could fail, or the disk subsystem could 
glitch, between them.  If SQLite recovers a file after a crash it goes some way 
towards detecting this but it's impossible to completely compensate for it.

You can ask why write-caching does this thing.  Well, it has two advantages.  
One is that it uses less memory (store just one copy of sector B in memory even 
if it's part of a frequently-changed log file).  The other is that it saves 
time (write all the changes to sector B in one operation).  And those two 
things are the main reasons people would turn write-caching on in the first 
place.

Some storage drivers have an EIOW (Enforce In Order Writes) 
directive/switch/patch.  Or an equivalent, like the 'barrier=1' switch when you 
mount the drive, that Dan mentioned, but one of the things that does is ensure 
that write-caching is not fully used.

> the possibility of losing data due to power loss

A useful way to proceed if you care about that, is to look at each setting 
which looks like it speeds things up and ask yourself "If it was okay to have 
this on all the time, why would they have provided the other option ?".  It's 
not nice, but it does help you consider the aspects.  And it can slow down 
operation considerably.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] import-prefixed symbols in sqlite3.obj file

[Dominique Devienne]

On Thu, Nov 10, 2016 at 11:38 AM, Max Vlasov  wrote:

> __imp_EnterCriticalSection


Is this post [1] related to your issue?
Maybe you're not using the 64-bit SDK too? --DD

[1]
http://stackoverflow.com/questions/17840857/linker-errors-while-migrating-from-x32-to-x64
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] import-prefixed symbols in sqlite3.obj file

[Max Vlasov]

Hi,

Trying to link sqlite object file generated by visual c++ command-line
compiler(cl.exe sqlite3.c /c) with the Delphi 64 bit executable, I
encountered that some functions by windows api used by sqlite contains
"___imp_" prefix (total: 7). This happened since recent delphi 64 bit
compilers accept COFF format files  but probably lack some assumptions
about semantics so doesn't automatically interprets such symbols as
imported function names.

The examples of imp-prefixed functions are
  __imp_EnterCriticalSection
  __imp_InitializeCriticalSection

All the other windows api imported function is linked without problems
and they don't have a prefix in the symbol table of the obj file.

I tried to compare the first appearances of normal and imp-prefixed
functions in the sqlite3.c. The normal ones are usually part of
win_syscall structure, while imp-prefixed appears randomly inside
win*** functions, for example EnterCriticalSection in winMutexEnter.

I will probably fix this by making special imported entries containing
this prefix, but just wonder why Visual c++ compiler interprets some
windows symbols as strict dll-imported entries and some as general
external symbols.

Thanks,

Max
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite ext3 settings

[Dan Kennedy]

On 11/10/2016 06:25 AM, Andrii Motsok wrote:

Hi,


I have sqlite database on top of ext3 (on ssd).


Does

   1) ext3 parameters (data=writeback|ordered,barrier=0|1, etc)

   2) drive write-caching setting (hdparm -W) value

   3) ???


There's a thread here:

  https://www.redhat.com/archives/ext3-users/2010-July/msg0.html

It seems to be that if your drive has write-caching enabled, you need 
barrier=1 to avoid risking corruption. Or you can get away with 
barrier=0 if write-caching is disabled.


Also:

  http://www.sqlite.org/draft/lockingv3.html#ext3-barrier-problem

Dan.





have impact on the possibility of losing data due to power loss? If yes which 
parameters should be used to achieve maximum safety against loosing of the data 
due to power loss during writing to WAL database?

   3) Are there any other settings (system, sqlite, etc) which could increase 
protection against data corruption after power lost or system crash?


Regards,

Andrii

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users