date:20130820

Re: [sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Larry Brasfield


Ashok wrote:

Hi All,

Have to come across any such issues?


Most experienced developers have come across such issues.  As Richard 
suggested, you are likely seeing the effect of a heap corruption that is 
occurring somewhat earlier than the "crashing" which finally reveals a 
problem.  His question, "Have you run your application using valgrind?" 
merits your attention.  If you do not yet use any tools for the early 
detection of heap corruption, you should learn to do so if you are a 
serious developer.  If you are using such a tool to help diagnose this 
particular problem, it would help others to help you if you were to say 
so.  If you cannot be troubled to use such a tool, it makes little sense 
for others to speculate as to how your not-yet-shown code is corrupting 
the heap upon which sqlite3DbMallocRaw() depends.  To be blunt, your 
hope that somebody will recognize and help cure your bug is unrealistic. 
 You have some work to do, and use of valgrind or a similar tool is the 
best advice you are likely to get that will help you do that work.


Best regards,
--
Larry Brasfield

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Ashok Pitambar

Hi All,

Have to come across any such issues?

Regards,
Ashok


On Tue, Aug 20, 2013 at 7:59 PM, Ashok Pitambar wrote:

> Hi Dan,
>
> Yes you are right , I think in my case pBuf has corrupted memory and while
> trying to access this it
> is crashing.
>
> Regards,
> Ashok
>
>
> On Tue, Aug 20, 2013 at 7:47 PM, Dan Kennedy wrote:
>
>> On 08/20/2013 09:08 PM, Ashok Pitambar wrote:
>>
>>> Hi Richard,
>>>
>>>   If list of available buffers (*db->lookaside.pFree*)  is NULL
>>> and
>>>
>>> it is assigned
>>> to pBuf and if you access  pBuf->pNext it will crash. read my comments in
>>> below code
>>> snippet.
>>>
>>>  There is good chance that sqlite may end up with empty
>>> lookaside pool(*db->lookaside.pFree*) list when there were many prepare
>>>
>>> statements are executed.
>>>
>>> please read my comments in code:
>>>
>>> if( db->lookaside.bEnabled ){
>>>if( n>db->lookaside.sz ){
>>>  db->lookaside.anStat[1]++;
>>>}
>>> *  //pBuf assigned with list of available buffers *
>>> * else if( (pBuf = db->lookaside.pFree)==0 ){*
>>>
>>>  db->lookaside.anStat[2]++;
>>>}else{
>>> *   //it will crash here and there is no check for NULL here*
>>> *db->lookaside.pFree = pBuf->pNext;*
>>>
>>>  db->lookaside.nOut++;
>>>  db->lookaside.anStat[0]++;
>>>  if( db->lookaside.nOut>db->**lookaside.mxOut ){
>>>db->lookaside.mxOut = db->lookaside.nOut;
>>>  }
>>>  return (void*)pBuf;
>>>
>>
>> You're misreading the "else if" condition. If pBuf is set to NULL,
>> the condition will be true and the "db->lookaside.anStat[2]++;"
>> line executed. Not the block below it.
>>
>>
>> __**_
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users
>>
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Ashok Pitambar

Hi Dan,

Yes you are right , I think in my case pBuf has corrupted memory and while
trying to access this it
is crashing.

Regards,
Ashok


On Tue, Aug 20, 2013 at 7:47 PM, Dan Kennedy  wrote:

> On 08/20/2013 09:08 PM, Ashok Pitambar wrote:
>
>> Hi Richard,
>>
>>   If list of available buffers (*db->lookaside.pFree*)  is NULL
>> and
>>
>> it is assigned
>> to pBuf and if you access  pBuf->pNext it will crash. read my comments in
>> below code
>> snippet.
>>
>>  There is good chance that sqlite may end up with empty
>> lookaside pool(*db->lookaside.pFree*) list when there were many prepare
>>
>> statements are executed.
>>
>> please read my comments in code:
>>
>> if( db->lookaside.bEnabled ){
>>if( n>db->lookaside.sz ){
>>  db->lookaside.anStat[1]++;
>>}
>> *  //pBuf assigned with list of available buffers *
>> * else if( (pBuf = db->lookaside.pFree)==0 ){*
>>
>>  db->lookaside.anStat[2]++;
>>}else{
>> *   //it will crash here and there is no check for NULL here*
>> *db->lookaside.pFree = pBuf->pNext;*
>>
>>  db->lookaside.nOut++;
>>  db->lookaside.anStat[0]++;
>>  if( db->lookaside.nOut>db->**lookaside.mxOut ){
>>db->lookaside.mxOut = db->lookaside.nOut;
>>  }
>>  return (void*)pBuf;
>>
>
> You're misreading the "else if" condition. If pBuf is set to NULL,
> the condition will be true and the "db->lookaside.anStat[2]++;"
> line executed. Not the block below it.
>
>
> __**_
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Dan Kennedy


On 08/20/2013 09:08 PM, Ashok Pitambar wrote:

Hi Richard,

  If list of available buffers (*db->lookaside.pFree*)  is NULL and
it is assigned
to pBuf and if you access  pBuf->pNext it will crash. read my comments in
below code
snippet.

 There is good chance that sqlite may end up with empty
lookaside pool(*db->lookaside.pFree*) list when there were many prepare
statements are executed.

please read my comments in code:

if( db->lookaside.bEnabled ){
   if( n>db->lookaside.sz ){
 db->lookaside.anStat[1]++;
   }
*  //pBuf assigned with list of available buffers *
* else if( (pBuf = db->lookaside.pFree)==0 ){*
 db->lookaside.anStat[2]++;
   }else{
*   //it will crash here and there is no check for NULL here*
*db->lookaside.pFree = pBuf->pNext;*
 db->lookaside.nOut++;
 db->lookaside.anStat[0]++;
 if( db->lookaside.nOut>db->lookaside.mxOut ){
   db->lookaside.mxOut = db->lookaside.nOut;
 }
 return (void*)pBuf;


You're misreading the "else if" condition. If pBuf is set to NULL,
the condition will be true and the "db->lookaside.anStat[2]++;"
line executed. Not the block below it.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Ashok Pitambar

Hi Richard,

 If list of available buffers (*db->lookaside.pFree*)  is NULL and
it is assigned
to pBuf and if you access  pBuf->pNext it will crash. read my comments in
below code
snippet.

There is good chance that sqlite may end up with empty
lookaside pool(*db->lookaside.pFree*) list when there were many prepare
statements are executed.

please read my comments in code:

if( db->lookaside.bEnabled ){
  if( n>db->lookaside.sz ){
db->lookaside.anStat[1]++;
  }
*  //pBuf assigned with list of available buffers *
* else if( (pBuf = db->lookaside.pFree)==0 ){*
db->lookaside.anStat[2]++;
  }else{
*   //it will crash here and there is no check for NULL here*
*db->lookaside.pFree = pBuf->pNext;*
db->lookaside.nOut++;
db->lookaside.anStat[0]++;
if( db->lookaside.nOut>db->lookaside.mxOut ){
  db->lookaside.mxOut = db->lookaside.nOut;
}
return (void*)pBuf;

Regards,
Ashok


On Tue, Aug 20, 2013 at 7:10 PM, Richard Hipp  wrote:

> On Tue, Aug 20, 2013 at 9:38 AM, Ashok Pitambar  >wrote:
>
> > Hi All,
> >
> > I am facing an issue where in sqlite3DbMallocRaw function
> > crashes while trying to access pBuf->pNext. Did anybody came across this
> > issue? any help is appreciated.
> >
>
> This is generally an indication of heap corruption.  Have you run your
> application using valgrind?
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Threading makes SQLite 3x slower??

2013-08-20 Thread nobre

What is the output of "EXPLAIN QUERY PLAN " ?



--
View this message in context: 
http://sqlite.1065341.n5.nabble.com/Threading-makes-SQLite-3x-slower-tp62243p70582.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Richard Hipp

On Tue, Aug 20, 2013 at 9:38 AM, Ashok Pitambar wrote:

> Hi All,
>
> I am facing an issue where in sqlite3DbMallocRaw function
> crashes while trying to access pBuf->pNext. Did anybody came across this
> issue? any help is appreciated.
>

This is generally an indication of heap corruption.  Have you run your
application using valgrind?

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] sqlite3DbMallocRaw is crashing

2013-08-20 Thread Ashok Pitambar

Hi All,

I am facing an issue where in sqlite3DbMallocRaw function
crashes while trying to access pBuf->pNext. Did anybody came across this
issue? any help is appreciated.

stack trace:

sqlite3DbMallocRaw(
db = 0x24EFBF34,
  ?)
  pBuf = 0x0B94256B -> (
pNext = 0x0 -> NULL
   )

function:

SQLITE_PRIVATE void *sqlite3DbMallocRaw(sqlite3 *db, int n){
  void *p;
  assert( db==0 || sqlite3_mutex_held(db->mutex) );
  assert( db==0 || db->pnBytesFreed==0 );
#ifndef SQLITE_OMIT_LOOKASIDE
  if( db ){
LookasideSlot *pBuf;
if( db->mallocFailed ){
  return 0;
}
if( db->lookaside.bEnabled ){
  if( n>db->lookaside.sz ){
db->lookaside.anStat[1]++;
  }else if( (pBuf = db->lookaside.pFree)==0 ){
db->lookaside.anStat[2]++;
  }else{
db->lookaside.pFree = pBuf->pNext;
db->lookaside.nOut++;
db->lookaside.anStat[0]++;
if( db->lookaside.nOut>db->lookaside.mxOut ){
  db->lookaside.mxOut = db->lookaside.nOut;
}
return (void*)pBuf;
  }
}
  }
#else
  if( db && db->mallocFailed ){
return 0;
  }
#endif
  p = sqlite3Malloc(n);
  if( !p && db ){
db->mallocFailed = 1;
  }
  sqlite3MemdebugSetType(p, MEMTYPE_DB |
 ((db && db->lookaside.bEnabled) ? MEMTYPE_LOOKASIDE :
MEMTYPE_HEAP));
  return p;
}
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bind Parameters Documentation

2013-08-20 Thread Richard Hipp

On Tue, Aug 20, 2013 at 9:24 AM, Filipe Oliveira
wrote:

>
> From my point of view the documentation isn't according to the
> implementation. Can anyone clarify?
>

Undocumented behavior is subject to change.  You are advised to use only
documented behavior.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bind Parameters Documentation

2013-08-20 Thread Filipe Oliveira


On 08/12/2013 08:26 PM, Filipe Oliveira wrote:

Hi,

In parameters section of this page 
http://www.sqlite.org/lang_expr.html says that


"A dollar-sign followed by an identifier name also holds a spot for a 
named parameter with the name $. The identifier name in this case 
can include one or more occurrences of "::" ...".


It gave me the impression (because of "in this case") that only 
parameters started with '$' can have occurrences of "::" but the 
tokenizer is accepting them for parameters started with ":" and "@".


Am I misunderstanding the documentation?

Thanks,

Filipe Oliveira

Hi,

For what I can see from the code it is possible to have parameters 
started by '#', '$', '@', and ':' that are succeeded by an identifier. 
This identifier can contain not only occurrences of "::" but also a 
suffix enclosed in "(...)".


tokenize.c - line 316
  /* Fall through into the next case if the '#' is not followed by
  ** a digit. Try to match # where  is a parameter name. */
}
#ifndef SQLITE_OMIT_TCL_VARIABLE
case '$':
#endif
case '@':  /* For compatibility with MS SQL Server */
case ':': {
...

From my point of view the documentation isn't according to the 
implementation. Can anyone clarify?


Thanks,

Filipe Oliveira

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Threading makes SQLite 3x slower??

2013-08-20 Thread Niall O'Reilly

On 4 Aug 2011, at 20:40, Seth Price wrote:

> so THREADSAFE=2 should work fine (as I understand it).

What makes you think it isn't?

> [...] it destroys performance.

My guess is that you've moved the bottle-neck to your disk,
and are suffering from seek latency.

Have you a way of looking at the activity queue for your
disk subsystem?  If so, what does it tell you?

Best regards,
Niall O'Reilly

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Threading makes SQLite 3x slower??

2013-08-20 Thread Seth Price

So far I've sped it up by 2x-3x by increasing the cache size by 10x, removing 
some extraneous SQL, and moving a bit of processing in-app. But using the full 
dataset with two threads instead of one still results in a 3x-5x time increase 
with either THREADSAFE=1 or 2. The fastest is single-threaded with 
THREADSAFE=0. My prepared queries now look like this:

SELECT class FROM data_r JOIN data USING (rowNum) WHERE ?1 < col0min AND 
col0max < ?2 AND ?3 < col1min AND col1max < ?4 AND ?5 < col2min AND col2max < 
?6 AND ?7 < col3min AND col3max < ?8 AND ?9 < col4min AND col4max < ?10 AND ?11 
< col5 AND col5 < ?12;

Am I configuring locking right?

I have 17 million queries like the above. Since they are all read-only, I 
shouldn't be taking such a hit. Every thread has it's own DB connection and 
prepared statements, so THREADSAFE=2 should work fine (as I understand it). 
Instead it destroys performance.
Thanks,
Seth

On Aug 4, 2011, at 12:36 PM, Seth Price wrote:

> Removing "COUNT(*) AS count" and "GROUP BY class" and doing it in-program 
> shaved ~10% off of the time. I'll keep it. :)
> ~Seth
> 
> On Aug 4, 2011, at 11:30 AM, Eduardo Morras wrote:
> 
>> 
>> Oks, another let's try another thing/think.
>> 
>> Try the select without the COUNT(*):
>> 
>> SELECT class FROM data_r JOIN data USING (rowNum) WHERE 57 < col0min 
>> AND col0max < 61 AND 52 < col1min AND col1max < 56 AND 66 < col2min 
>> AND col2max < 70 AND 88 < col3min AND col3max < 92 AND 133 < col4min 
>> AND col4max < 137 AND 57 < col0 AND col0 < 61 AND 52 < col1 AND col1 
>> < 56 AND 66 < col2 AND col2 < 70 AND 88 < col3 AND col3 < 92 AND 133 
>> < col4 AND col4 < 137 AND 81 < col5 AND col5 < 85 GROUP BY class;
>> 
>> In some rdbms (don' know in sqlite), count, avg, sum, etc... implies 
>> a table scan, making the select very slow.
>> 
>> HTH
>> 
>> 
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite Input with validation and lookup

2013-08-20 Thread deltagam...@gmx.net


Am 16.08.2013 17:47, schrieb Kai Peters:

Stephen,

you might want to look at

http://dabodev.com/

Among other DBs it does support Sqlite and is fully cross-platform

Cheers,
Kai




Hello Stephen,

maybe another alternative is, to run FoxPro on a linux machine with the 
windows emulator wine.


There is a lot of windows software running directly on linux, so give it 
a try. At least with older FoxPro versions, people achieved reasonable 
results.


http://appdb.winehq.org/objectManager.php?sClass=application=296

http://paulmcnett.com/vfp/wine/VFPonLinux_foxtalk2.html


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] System.Data.SQLite version 1.0.88.0 released

2013-08-20 Thread Jan Slodicka

Again - rather a comment than a bug:

If sqlite3_finalize() fails, SQLiteBase.FinalizeStatement() throws an
exception. The only use of SQLiteBase.FinalizeStatement() is in
SqliteStatementHandle.ReleaseHandle() where this exception is caught and
ignored.

SQLite docs say that sqlite3_finalize() reports errors that were previously
reported, i.e. they must have been handled by the user already.

I think the exception should not be thrown.

Jan Slodicka



--
View this message in context: 
http://sqlite.1065341.n5.nabble.com/System-Data-SQLite-version-1-0-88-0-released-tp70425p70575.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] sqlite3DbMallocRaw is crashing

Re: [sqlite] sqlite3DbMallocRaw is crashing

Re: [sqlite] sqlite3DbMallocRaw is crashing

Re: [sqlite] sqlite3DbMallocRaw is crashing

Re: [sqlite] sqlite3DbMallocRaw is crashing

Re: [sqlite] Threading makes SQLite 3x slower??

Re: [sqlite] sqlite3DbMallocRaw is crashing

[sqlite] sqlite3DbMallocRaw is crashing

Re: [sqlite] Bind Parameters Documentation

Re: [sqlite] Bind Parameters Documentation

Re: [sqlite] Threading makes SQLite 3x slower??

Re: [sqlite] Threading makes SQLite 3x slower??

Re: [sqlite] SQLite Input with validation and lookup

Re: [sqlite] System.Data.SQLite version 1.0.88.0 released

14 matches

Site Navigation

Mail list logo

Footer information