[sqlite] Invalid table name provided in sqlite3_update_hook callback

2013-08-01 Thread psryland 
Hi All,

I am working on an Android project that uses Xamarin.Android and the sqlite
binary distributed with the Android OS (various versions, depending on the
Android OS version). 

Within the application we have an object caching scheme that relies on the
callback provided to the sqlite3_update_hook() function to invalidate cached
objects whenever they are changed in the db. For the most part everything
works fine, however, in some cases I've observed the callback function being
called with the table name parameter as null. For example, I've seen the
callback called with parameters such as:
  operation: SQLITE_INSERT
  db name: "main"
  table name: null
  row id:  

I acknowledge that this is most likely an issue with the marshalling from
native to C# however due to the difficulty of debugging native code on an
android device from within a C# application, I've made little progress
diagnosing the cause of the problem. Basically I'm getting desperate, and
was wondering if anyone could provide some insight into possible reasons why
the provided table name would be null or invalid.

For more background on the problem, this is what know:
(Observed in sqlite version 3.7.4 mostly, although also in 3.7.11)
(Running on an ARM device)

With a C# delegate signature like this:
void UpdateHookCB(IntPtr context, int change_type, string db_name, string
table_name, long row_id)
'table_name' is sometimes null.

With a signature like this:
void UpdateHookCB(IntPtr context, int change_type, IntPtr db_name, IntPtr
table_name, long row_id)
'table_name' is not IntPtr.Zero, but appears to be a pointer to
non-string-like data. I determined this using the Marshal.PtrToStructure
method to read a small number of bytes from the table_name pointer. The data
looked like low value 32 bit ints, but that is just a guess.

Currently I work around this problem by flushing my entire object cache
whenever a callback containing an invalid table name happens, but this is
killing the performance of my application.

Any help/insight would be greatly appreciated,
Thanks,
Paul



--
View this message in context: 
http://sqlite.1065341.n5.nabble.com/Invalid-table-name-provided-in-sqlite3-update-hook-callback-tp70348.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] proper use of sqlite3_file_control()

2013-08-01 Thread Stephan Beal 
On Fri, Aug 2, 2013 at 12:30 AM, Richard Hipp  wrote:

> That File Control was added for 3.7.16, just a few months ago.
>

Yeah, i found that out the hard way ;) and had to go update my local
sqlite3, but you were half right - i was linking to the wrong one, but i'm
still seeing that behaviour with my fresh build:


 My guess is
> that you are linking against an older SQLite.
>


stephan@tiny:~/cvs/fossil/f2$ make test
+ g++ -o test -pedantic -Wall -Werror -fPIC -Wno-long-long -g test.o -fPIC
-L. -lfossil -lsqlite3 -lz

wrong copy, so rebuild with -L$(HOME)/lib:

stephan@tiny:~/cvs/fossil/f2$ g++ -o test -pedantic -Wall -Werror -fPIC
-Wno-long-long -g test.o -fPIC -L. -lfossil -L$HOME/lib -lsqlite3 -lz

stephan@tiny:~/cvs/fossil/f2$ ldd test
...
libfossil.so => ./libfossil.so (0xb7795000)
libsqlite3.so.0 => /home/stephan/lib/libsqlite3.so.0 (0xb7703000)
... /lib/ld-linux.so.2 (0xb77b)

stephan@tiny:~/cvs/fossil/f2$ l /home/stephan/lib/libsqlite3.so.0
lrwxrwxrwx 1 stephan stephan 19 Aug  2 00:06
/home/stephan/lib/libsqlite3.so.0 -> libsqlite3.so.0.8.6

stephan@tiny:~/cvs/fossil/f2$ l /home/stephan/lib/libsqlite3.so.0.8.6
-rwxr-xr-x 1 stephan stephan 2107154 Aug  2 00:06
/home/stephan/lib/libsqlite3.so.0.8.6

stephan@tiny:~/cvs/fossil/f2$ ./test
MARKER: test.c:69:test_tmpfile_0(): tmpfile=[(null)]
MARKER: test.c:250:main(): Done! rc=0 (FSL_RC_OK)

That was built from:

stephan@tiny:~/cvs/fossil/sqlite3$ fst
repository:   /home/stephan/cvs/fossil/sqlite3.fsl
local-root:   /home/stephan/cvs/fossil/sqlite3/
config-db:/home/stephan/.fossil
checkout: e436b2f4e5c5e6b2f70e65332c0c7d618e2ef20a 2013-08-01 20:26:04
UTC
parent:   c3baca99f4580652afb2c3f73036ab83796a1557 2013-08-01 19:17:39
UTC
tags: trunk
comment:  Fix a potential buffer overread in sqlite3VdbeRecordCompare()
  when a serial_type specifies a field that starts in bounds but
  is much too large for the allocated buffer. Mostly harmless.
The
  overread is unlikely to go more than one or two bytes past the
  end of the buffer. (user: drh)


Anyway - not a big deal (==no action necessary, as far as i'm concerned).
My goal here is only to find/steal a way to generate a temporary file name
(including the platform-specific dir part, preferably), and i figured
sqlite3 already had something for that. If there is another solution you
know about which i can steal (==incorporate into libfossil, meaning you
are/would be the license holder), i'm all for it.

Thanks :)

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] proper use of sqlite3_file_control()

2013-08-01 Thread Richard Hipp 
On Thu, Aug 1, 2013 at 6:23 PM, Stephan Beal  wrote:

> Hi, all,
>
> i'm trying to use:
>
> http://www.sqlite.org/capi3ref.html#sqlite3_file_control
>
> to create a name for a temporary file, as described here:
>
> http://www.sqlite.org/capi3ref.html#sqlitefcntltempfilename
>

That File Control was added for 3.7.16, just a few months ago.  My guess is
that you are linking against an older SQLite.


>
> And it looks like:
>
> #include 
> #ifdef SQLITE_FCNTL_TEMPFILENAME
> static int test_tmpfile_0(){
>   char * tmpN = NULL;
>   sqlite3 * db = App.f->dbMain.dbh;
>   assert(db);
>   sqlite3_file_control(db, NULL, SQLITE_FCNTL_TEMPFILENAME, );
>   MARKER(("tmpfile=[%s]\n", tmpN));
>   sqlite3_free(tmpN);
>   return 0;
> }
> #endif
>
> (This is a debug build - assert() works here and App.f->db is used
> elsewhere in the app.)
>
> Which outputs:
>
> MARKER: test.c:69:test_tmpfile_0(): tmpfile=[(null)]
>
> Am i misusing or misunderstanding how the TEMPFILENAME fcntl is supposed to
> be used? i tried "main" and "temp" as the 2nd argument, but no difference.
>
> :-?
>
> --
> - stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] proper use of sqlite3_file_control()

2013-08-01 Thread Stephan Beal 
Hi, all,

i'm trying to use:

http://www.sqlite.org/capi3ref.html#sqlite3_file_control

to create a name for a temporary file, as described here:

http://www.sqlite.org/capi3ref.html#sqlitefcntltempfilename

And it looks like:

#include 
#ifdef SQLITE_FCNTL_TEMPFILENAME
static int test_tmpfile_0(){
  char * tmpN = NULL;
  sqlite3 * db = App.f->dbMain.dbh;
  assert(db);
  sqlite3_file_control(db, NULL, SQLITE_FCNTL_TEMPFILENAME, );
  MARKER(("tmpfile=[%s]\n", tmpN));
  sqlite3_free(tmpN);
  return 0;
}
#endif

(This is a debug build - assert() works here and App.f->db is used
elsewhere in the app.)

Which outputs:

MARKER: test.c:69:test_tmpfile_0(): tmpfile=[(null)]

Am i misusing or misunderstanding how the TEMPFILENAME fcntl is supposed to
be used? i tried "main" and "temp" as the 2nd argument, but no difference.

:-?

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bug report: Sqlite seg fault, probably after database gets corrupt

2013-08-01 Thread Richard Hipp 
On Thu, Aug 1, 2013 at 3:30 PM, Brian Vincent  wrote:

> if( d1>=(u32)nKey1 && sqlite3VdbeSerialTypeLen(serial_type1)>0 ) break;
>
> The next line will likely segfault if d1>=nKey1, right?  What if d1>=nKey1,
> but it's not true that sqlite3VdbeSerialTypeLen(serial_type1)>0 ?  Wouldn't
> this still cause a segfault?  Is that a valid concern?
>

Not a concern.

The [d1] is just an address.  And it never gets dereferenced if the
SerialTypeLen is zero.



>
> -Brian Vincent
>
>
>
> On Thu, Aug 1, 2013 at 2:19 PM, Richard Hipp  wrote:
>
> > On Thu, Aug 1, 2013 at 2:20 PM, Brian Vincent  wrote:
> >
> > > I think I can describe, is a
> > > possibly way that a corrupt database is causing sqlite to segfault.
> > >
> >
> > Thanks.  Fixed in http://www.sqlite.org/src/info/c3baca99f4 including a
> > test case.
> >
> >
> > --
> > D. Richard Hipp
> > d...@sqlite.org
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bug report: Sqlite seg fault, probably after database gets corrupt

2013-08-01 Thread Brian Vincent 
if( d1>=(u32)nKey1 && sqlite3VdbeSerialTypeLen(serial_type1)>0 ) break;

The next line will likely segfault if d1>=nKey1, right?  What if d1>=nKey1,
but it's not true that sqlite3VdbeSerialTypeLen(serial_type1)>0 ?  Wouldn't
this still cause a segfault?  Is that a valid concern?

-Brian Vincent



On Thu, Aug 1, 2013 at 2:19 PM, Richard Hipp  wrote:

> On Thu, Aug 1, 2013 at 2:20 PM, Brian Vincent  wrote:
>
> > I think I can describe, is a
> > possibly way that a corrupt database is causing sqlite to segfault.
> >
>
> Thanks.  Fixed in http://www.sqlite.org/src/info/c3baca99f4 including a
> test case.
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bug report: Sqlite seg fault, probably after database gets corrupt

2013-08-01 Thread Richard Hipp 
On Thu, Aug 1, 2013 at 2:20 PM, Brian Vincent  wrote:

> I think I can describe, is a
> possibly way that a corrupt database is causing sqlite to segfault.
>

Thanks.  Fixed in http://www.sqlite.org/src/info/c3baca99f4 including a
test case.


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bug report: Sqlite seg fault, probably after database gets corrupt

2013-08-01 Thread Stephan Beal 
On Thu, Aug 1, 2013 at 8:20 PM, Brian Vincent  wrote:

> next line assigns it to d1, which is a signed integer, so d1 gets a
> negative value.


To be strictly pedantic, overflow/underflow are undefined for _signed_
types in C. Here are some details:
http://en.wikipedia.org/wiki/Integer_overflow
The "Origin" section.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bug report: Sqlite seg fault, probably after database gets corrupt

2013-08-01 Thread Warren Young 

On 8/1/2013 12:20, Brian Vincent wrote:

Let me first say that we sometimes see databases that go corrupt.  I
haven't pinpointed the cause yet,


This may be enlightening: "How to Corrupt an SQLite Database File"

https://www.sqlite.org/howtocorrupt.html
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Bug report: Sqlite seg fault, probably after database gets corrupt

2013-08-01 Thread Brian Vincent 
Let me first say that we sometimes see databases that go corrupt.  I
haven't pinpointed the cause yet, but what I think I can describe, is a
possibly way that a corrupt database is causing sqlite to segfault.

I use a java wrapper for sqlite.  I've seen this exact segfault happen on
an older wrapper I used, and the current wrapper I use.  I have several
detailed java dumps if anyone wants to see them.

Here is what I've determined is happening.  Please look at the
function sqlite3VdbeRecordCompare.

  int d1;/* Offset into aKey[] of next data element */
  u32 idx1;  /* Offset into aKey[] of next header element */
  u32 szHdr1;/* Number of bytes in header */

...

  idx1 = getVarint32(aKey1, szHdr1);
  d1 = szHdr1;
  if( pPKey2->flags & UNPACKED_IGNORE_ROWID ){
szHdr1--;
  }
  nField = pKeyInfo->nField;
  while( idx1nField ){
u32 serial_type1;

/* Read the serial types for the next element in each key. */
idx1 += getVarint32( aKey1+idx1, serial_type1 );
if( d1>=nKey1 && sqlite3VdbeSerialTypeLen(serial_type1)>0 ) break;

/* Extract the values to be compared.
*/
d1 += sqlite3VdbeSerialGet([d1], serial_type1, );



Correct me if I'm wrong, but aKey is just a buffer directly read from the
database.  What's happening to me is that the getVarint32 that reads szHdr1
is reading a very large 32 bit value, >= 2^31, such as 0xbd21d318.  The
next line assigns it to d1, which is a signed integer, so d1 gets a
negative value.  Sqlite will then segfault inside of sqlite3VdbeSerialGet
when it tries to read from the bogus memory location [d1].  I have no
idea if this impacts security of programs that read sqlite databases.

I realize that this function is a very high runner, so more sanity checking
might have a measurable performance impact.  But can d1 be unsigned to
prevent it from going negative?  Is d1 supposed to always be http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bitwise Not Operator Precedence

2013-08-01 Thread Richard Hipp 
On Thu, Aug 1, 2013 at 1:06 PM, Filipe Oliveira wrote:

> Hi,
>
> Here http://www.sqlite.org/lang_expr.html says that "The COLLATE operator
> is a unary postfix operator that assigns a collating
> sequence to
> an expression. The COLLATE operator has a higher precedence (binds more
> tightly) than any prefix unary operator or any binary operator.".
>
> But checking the parser.y file the bitwise not operator seems to have a
> higher precedence than COLLATE.
>
> parser.y - line 227
> %left OR.
> %left AND.
> %right NOT.
> %left IS MATCH LIKE_KW BETWEEN IN ISNULL NOTNULL NE EQ.
> %left GT LE LT GE.
> %right ESCAPE.
> %left BITAND BITOR LSHIFT RSHIFT.
> %left PLUS MINUS.
> %left STAR SLASH REM.
> %left CONCAT.
> %left COLLATE.
> %right BITNOT.
>
> So, which one is correct?
>

The parser.

I'll fix the documentation.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] making a meal of text substitution

2013-08-01 Thread dean gwilliam 

It's ok I cracked it...
I just needed to enclose the script in nothing more than double quotes 
to get the variables to

substitute correctly i.e.
set script "CREATE TABLE $nm ( id integer primary key, $flds )"
db eval $script
and ignoring the example syntax's '{}'i.e.
gDb eval {CREATE TABLE std( fld defs here )}


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] making a meal of text substitution

2013-08-01 Thread dean gwilliam 
Can someone please help me to understand why my substitution of the 
single line that creates the table 'std'

isn't working. Any help much appreciated!
#==
proc msg {vlu} {tk_messageBox -message "$vlu"}

::oo::class create cDbase {
variable db tmp
constructor {} {
package require sqlite3
set tmp {}
set db {}
}

method db_open {fPth} {
sqlite3 db $fPth ;# :memory:
}

method tbl_open {nm flds} {
db eval {CREATE TABLE std( id integer primary key, epic text, 
itm text, yr integer, vlu real)}

why isn't this a replacement for the line above???
#==
#msg "nm=$nm\nflds=$flds"
#set script {CREATE TABLE [$nm]( id integer primary key, [$flds] )}
#db eval $script
#==

db eval {INSERT INTO std VALUES(1,'mcro','revs',2003, 10)}
db eval {INSERT INTO std VALUES(2,'mcro','revs',2004, 11)}
set x [db eval {SELECT vlu FROM std WHERE itm='revs' ORDER BY id}]
tk_messageBox -message $x
db close
}

method add {} {

}

method destructor {} {
}
}

cDbase create d
d db_open :memory:
d tbl_open std "epic text, itm text, yr integer, vlu real"
d destroy
cDbase destroy
exit

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users