Re: [sqlite] Bus Error on OpenBSD

[Mark Lawrence]

On Fri Oct 28, 2016 at 05:48:48PM +0700, Dan Kennedy wrote:
> On 10/28/2016 05:39 PM, no...@null.net wrote:
> > Hi Rowan,
> > 
> > On Fri Oct 28, 2016 at 06:19:59PM +0800, Rowan Worth wrote:
> > > Every sqlite_stmt you use *must* be finalized via sqlite3_finalize.
> > > I'm not exactly sure what that looks like from the other side of DBD,
> > > but I would be checking your perl code for a statement/resultset
> > > object which outlives the database connection itself.
> > >
> > Some of my new debug statements appear to confirm that: database
> > handles are being cleaned up before statement handles, even though
> > presumably the statement handle still has a reference back to the
> > database.
> 
> SQLite should handle that. If you call sqlite3_close() before all statement
> handles have been cleaned up, the call fails with SQLITE_MISUSE. Or if you
> use sqlite3_close_v2(), the call succeeds, but a reference count is used to
> ensure that the db handle object is not actually deleted until all
> statements are. close_v2() was added for this situation - where a garbage
> collectors or similar is responsible for closing db handles and finalizing
> statements.

The "handles" I was referring to above were Perl DBI handles. No doubt
they contain a real SQLite handle somewhere, but I don't think it is
safe to assume a one-to-one mapping. For example, the following appears
to create two Perl objects for each of the $db and $sth variables:

use DBI;

sub DBI::db::DESTROY {
warn "DESTROY @_";
}

sub DBI::st::DESTROY {
warn "DESTROY @_";
}

my $db = DBI->connect('dbi:SQLite:dbname=:memory:');
my $sth = $db->prepare('select 1');


#   DESTROY DBI::db=HASH(0x9acf68c) at x line 4.
#   DESTROY DBI::st=HASH(0x9acf95c) at x line 8.
#   DESTROY DBI::st=HASH(0x9acf754) at x line 8.
#   DBI st handle 0x9acf95c has uncleared implementors data.
#   dbih_clearcom (sth 0x9acf95c, com 0x9ad1518, imp 
DBD::SQLite::st):
#  FLAGS 0x100113: COMSET IMPSET Warn PrintError 
PrintWarn 
#  PARENT DBI::db=HASH(0x9acf600)
#  KIDS 0 (0 Active)
#  NUM_OF_FIELDS 1
#  NUM_OF_PARAMS 0
#   DESTROY DBI::db=HASH(0x9acf600) at x line 4.
#   DBI db handle 0x9acf68c has uncleared implementors data.
#   dbih_clearcom (dbh 0x9acf68c, com 0x9ac1fd8, imp 
DBD::SQLite::db):
#  FLAGS 0x100317: COMSET IMPSET Active Warn PrintError
#PrintWarn AutoCommit 
#  PARENT DBI::dr=HASH(0x9acf1a0)
#  KIDS 0 (0 Active)
#

It is not obvious to me why that is so and I don't know the DBD::SQLite
code base so I won't speculate.

> If this is repeatable, try running it under valgrind. The valgrind
> error should make it pretty clear whether or not the statement handle
> really has already been finalized.

Well I have found what is probably the original source of the error: I
was keeping (Perl) statement handles around after the database handles
had expired. That doesn't mean that there isn't an issue with
how DBD::SQLite is using SQLite, but I no longer have the motivation to
track down that error when the easy answer to my problem is "don't do
that." Plus I have another error to report in a new thread :-(

Thanks everyone for listening.

-- 
Mark Lawrence
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bus Error on OpenBSD

[Dan Kennedy]

On 10/28/2016 05:39 PM, no...@null.net wrote:

Hi Rowan,

On Fri Oct 28, 2016 at 06:19:59PM +0800, Rowan Worth wrote:

Every sqlite_stmt you use *must* be finalized via sqlite3_finalize.
I'm not exactly sure what that looks like from the other side of DBD,
but I would be checking your perl code for a statement/resultset
object which outlives the database connection itself.

Some of my new debug statements appear to confirm that: database
handles are being cleaned up before statement handles, even though
presumably the statement handle still has a reference back to the
database.


SQLite should handle that. If you call sqlite3_close() before all 
statement handles have been cleaned up, the call fails with 
SQLITE_MISUSE. Or if you use sqlite3_close_v2(), the call succeeds, but 
a reference count is used to ensure that the db handle object is not 
actually deleted until all statements are. close_v2() was added for this 
situation - where a garbage collectors or similar is responsible for 
closing db handles and finalizing statements.


This looks like the statement handle being passed to sqlite3_finalize() 
has already been finalized. Or perhaps that it is just a stray pointer 
that was never a statement handle. To confirm, jump back to the 
sqlite3VdbeFinalize() frame of your stacktrace and do "print *p". The 
entire object has likely been 0xdf'd out.


If this is repeatable, try running it under valgrind. The valgrind error 
should make it pretty clear whether or not the statement handle really 
has already been finalized.


Dan.






So I also did some googling on that topic, and it appears that during
Perl's global destruction phase objects may not necessarily be
destroyed in the right order.  That is something I unfortunately don't
have any easy control over :-( Perhaps I can be more explicit
somewhere...

In any event this is probably not an sqlite issue.

Thanks for commenting.

Mark.



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bus Error on OpenBSD

[nomad]

Hi Rowan,

On Fri Oct 28, 2016 at 06:19:59PM +0800, Rowan Worth wrote:
> 
> Every sqlite_stmt you use *must* be finalized via sqlite3_finalize.
> I'm not exactly sure what that looks like from the other side of DBD,
> but I would be checking your perl code for a statement/resultset
> object which outlives the database connection itself.

Some of my new debug statements appear to confirm that: database
handles are being cleaned up before statement handles, even though
presumably the statement handle still has a reference back to the
database.

So I also did some googling on that topic, and it appears that during
Perl's global destruction phase objects may not necessarily be
destroyed in the right order.  That is something I unfortunately don't
have any easy control over :-( Perhaps I can be more explicit
somewhere...

In any event this is probably not an sqlite issue.

Thanks for commenting.

Mark.
-- 
Mark Lawrence
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bus Error on OpenBSD

[Rowan Worth]

Hi Mark,

A quick google suggests this is a use after free error, as OpenBSD's
allocator apparently fills freed memory pages with the pattern 0xdfdfdfdfdf.

The stack trace reads like it is crashing while finalizing an sqlite_stmt,
as part of some automatic perl destructor logic.

Every sqlite_stmt you use *must* be finalized via sqlite3_finalize. I'm not
exactly sure what that looks like from the other side of DBD, but I would
be checking your perl code for a statement/resultset object which outlives
the database connection itself.

-Rowan


On 28 October 2016 at 18:10,  wrote:

> I am seeing a Bus Error at the end of a program that to my
> inexperienced eye appears to have something to do with SQLite:
>
> This GDB was configured as "amd64-unknown-openbsd6.0"...
> Core was generated by `bif'.
> Program terminated with signal 10, Bus error.
> Loaded symbols for /mark/src/bif/static/bif
> Reading symbols from /usr/lib/libm.so.9.0...done.
> Loaded symbols for /usr/lib/libm.so.9.0
> Reading symbols from /usr/lib/libc.so.88.0...done.
> Loaded symbols for /usr/lib/libc.so.88.0
> Reading symbols from /usr/libexec/ld.so...done.
> Loaded symbols for /usr/libexec/ld.so
> #0  releaseMemArray (p=0xdfdfdfdfdfdfdfdf, N=286555) at sqlite3.c:71943
> 71943   sqlite3 *db = p->db;
>
> (gdb) backtrace
> #0  releaseMemArray (p=0xdfdfdfdfdfdfdfdf, N=286555) at sqlite3.c:71943
> #1  0x1285167d22e4 in sqlite3VdbeClearObject
> (db=0xdfdfdfdfdfdfdfdf, p=0x12873d135408) at sqlite3.c:73412
> #2  0x1285167d23d2 in sqlite3VdbeDelete (p=0x12873d135408) at
> sqlite3.c:73444
> #3  0x128516815191 in sqlite3VdbeFinalize (p=0x12873d135408) at
> sqlite3.c:73362
> #4  0x1285168152bd in sqlite3_finalize (pStmt=0x12873d135408) at
> sqlite3.c:75209
> #5  0x1285167b5ef5 in sqlite_st_destroy (sth=0x1287bf19a198,
> imp_sth=0x128791361b00) at dbdimp.c:1256
> #6  0x1285167ab7b3 in XS_DBD__SQLite__st_DESTROY (cv=Variable "cv"
> is not available.) at SQLite.xsi:799
> #7  0x12851685e60a in XS_DBI_dispatch (cv=0x12878d7123c8) at
> DBI.xs:3781
> #8  0x1285168e8ab7 in Perl_pp_entersub () at pp_hot.c:2794
> #9  0x128516884036 in Perl_call_sv (sv=0x12878d7123c8, flags=45)
> at perl.c:2775
> #10 0x1285168f0c21 in S_curse (sv=0x1287b8a0f1a8,
> check_refcnt=true) at sv.c:6704
> #11 0x1285168f0e07 in Perl_sv_clear (orig_sv=0x1287b8a0f1a8) at
> sv.c:6326
> #12 0x1285168f15b9 in Perl_sv_free2 (sv=0x1287b8a0f1a8,
> rc=Variable "rc" is not available.) at sv.c:6805
> #13 0x1285168e8bc3 in S_visit (f=0x1285168f181c ,
> flags=2048, mask=2048) at sv.c:485
> #14 0x1285168f1ab0 in Perl_sv_clean_objs () at sv.c:640
> #15 0x128516886bc3 in perl_destruct (my_perl=Variable "my_perl" is
> not available.) at perl.c:804
> #16 0x128516742a66 in main (argc=5, argv=0x7f7bef58) at
> bundle.c:15988
>
> I find the pointer address 0xdfdfdfdfdfdfdfdf to be a little
> suspicious.
>
> The program is a static build of Perl that embeds DBD::SQLite which
> embeds sqlite.  I have seen the error with sqlite version 3.10.2 and
> 3.15.0. I have only seen the error on OpenBSD - my Linux builds seem to
> have no problem.
>
> Any ideas how I could debug this further?
>
> Mark
> --
> Mark Lawrence
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Bus Error on OpenBSD

[nomad]

I am seeing a Bus Error at the end of a program that to my
inexperienced eye appears to have something to do with SQLite:

This GDB was configured as "amd64-unknown-openbsd6.0"...
Core was generated by `bif'.
Program terminated with signal 10, Bus error.
Loaded symbols for /mark/src/bif/static/bif
Reading symbols from /usr/lib/libm.so.9.0...done.
Loaded symbols for /usr/lib/libm.so.9.0
Reading symbols from /usr/lib/libc.so.88.0...done.
Loaded symbols for /usr/lib/libc.so.88.0
Reading symbols from /usr/libexec/ld.so...done.
Loaded symbols for /usr/libexec/ld.so
#0  releaseMemArray (p=0xdfdfdfdfdfdfdfdf, N=286555) at sqlite3.c:71943
71943   sqlite3 *db = p->db;

(gdb) backtrace
#0  releaseMemArray (p=0xdfdfdfdfdfdfdfdf, N=286555) at sqlite3.c:71943
#1  0x1285167d22e4 in sqlite3VdbeClearObject (db=0xdfdfdfdfdfdfdfdf, 
p=0x12873d135408) at sqlite3.c:73412
#2  0x1285167d23d2 in sqlite3VdbeDelete (p=0x12873d135408) at 
sqlite3.c:73444
#3  0x128516815191 in sqlite3VdbeFinalize (p=0x12873d135408) at 
sqlite3.c:73362
#4  0x1285168152bd in sqlite3_finalize (pStmt=0x12873d135408) at 
sqlite3.c:75209
#5  0x1285167b5ef5 in sqlite_st_destroy (sth=0x1287bf19a198, 
imp_sth=0x128791361b00) at dbdimp.c:1256
#6  0x1285167ab7b3 in XS_DBD__SQLite__st_DESTROY (cv=Variable "cv" is 
not available.) at SQLite.xsi:799
#7  0x12851685e60a in XS_DBI_dispatch (cv=0x12878d7123c8) at DBI.xs:3781
#8  0x1285168e8ab7 in Perl_pp_entersub () at pp_hot.c:2794
#9  0x128516884036 in Perl_call_sv (sv=0x12878d7123c8, flags=45) at 
perl.c:2775
#10 0x1285168f0c21 in S_curse (sv=0x1287b8a0f1a8, check_refcnt=true) at 
sv.c:6704
#11 0x1285168f0e07 in Perl_sv_clear (orig_sv=0x1287b8a0f1a8) at 
sv.c:6326
#12 0x1285168f15b9 in Perl_sv_free2 (sv=0x1287b8a0f1a8, rc=Variable 
"rc" is not available.) at sv.c:6805
#13 0x1285168e8bc3 in S_visit (f=0x1285168f181c , 
flags=2048, mask=2048) at sv.c:485
#14 0x1285168f1ab0 in Perl_sv_clean_objs () at sv.c:640
#15 0x128516886bc3 in perl_destruct (my_perl=Variable "my_perl" is not 
available.) at perl.c:804
#16 0x128516742a66 in main (argc=5, argv=0x7f7bef58) at 
bundle.c:15988

I find the pointer address 0xdfdfdfdfdfdfdfdf to be a little
suspicious.

The program is a static build of Perl that embeds DBD::SQLite which
embeds sqlite.  I have seen the error with sqlite version 3.10.2 and
3.15.0. I have only seen the error on OpenBSD - my Linux builds seem to
have no problem.

Any ideas how I could debug this further?

Mark
-- 
Mark Lawrence
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users