[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

Yes, I did exactly that and posted the log, but it doesn't catch any error.
Sofar though I have been unable to log sqlite3.dll.

RBS
On 26 Dec 2015 8:02 pm, "Keith Medcalf"  wrote:

>
> When the functionis entered, open a file for appending.
> Just before each line have it write:
> "Just about to execute line #" to the file.
> Flush the file buffers after each write.
>
> When your code crashes, open the file with a text editor.
> The last line of the file will point to the line which caused the error
> condition to be detected.
>
> Then you will know which line could be related to the problem.
>
> > -Original Message-
> > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-
> > bounces at mailinglists.sqlite.org] On Behalf Of Bart Smissaert
> > Sent: Saturday, 26 December, 2015 12:55
> > To: General Discussion of SQLite Database
> > Subject: Re: [sqlite] Malloc getting segmentation fault in sqlite
> >
> > Not sure, what is instrumented?
> >
> > RBS
> > On 26 Dec 2015 7:52 pm, "Keith Medcalf"  wrote:
> >
> > >
> > > Have you instrumented your code?
> > >
> > > > -Original Message-
> > > > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-
> > users-
> > > > bounces at mailinglists.sqlite.org] On Behalf Of Bart Smissaert
> > > > Sent: Saturday, 26 December, 2015 12:33
> > > > To: SQLite mailing list
> > > > Subject: Re: [sqlite] Malloc getting segmentation fault in sqlite
> > > >
> > > > VB6 is the original Visual Basic, unfortunately ditched by MS some
> > years
> > > > ago and replaced by .Net.
> > > >
> > > > Yes, I am sure it is some kind of memory problem, but where (what
> > line)
> > > > could it possibly occur?
> > > > All I can think of is this line:
> > > > sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
> > > > SQLITE_TRANSIENT
> > > > But as it uses SQLITE_TRANSIENT, I understand SQLite will make  local
> > > copy
> > > > and deal with it properly itself.
> > > >
> > > > All the other code is just copying data from SQLite to local VB6
> > > > variables.
> > > >
> > > > RBS
> > > >
> > > >
> > > >
> > > >
> > > > On Sat, Dec 26, 2015 at 6:52 PM, Simon Slavin 
> > > > wrote:
> > > >
> > > > >
> > > > > On 26 Dec 2015, at 5:23pm, Bart Smissaert <
> bart.smissaert at gmail.com>
> > > > > wrote:
> > > > >
> > > > > >> all is perfectly fine when the SQL statement has only one UDF in
> > it
> > > > > >
> > > > > > Just a correction on this. Have now also seen problems when there
> > is
> > > > only
> > > > > > one UDF in the SQL statement.
> > > > > > It is not entirely predictable, so sometimes get crash and on
> > other
> > > > > > occasions I don't.
> > > > >
> > > > > That is typical of memory mis-management problems everyone runs
> into
> > > > from
> > > > > time to time.  You are probably releasing or overwriting memory
> > which
> > > > > SQLite thinks it can depend on.  Unfortunately I don't know the
> > version
> > > > of
> > > > > BASIC you're using, but your commands 'ReDim', for example, make me
> > > > think
> > > > > memory is being released.
> > > > >
> > > > > Simon.
> > > > > ___
> > > > > sqlite-users mailing list
> > > > > sqlite-users at mailinglists.sqlite.org
> > > > >
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> > > > >
> > > > ___
> > > > sqlite-users mailing list
> > > > sqlite-users at mailinglists.sqlite.org
> > > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> > >
> > >
> > >
> > > ___
> > > sqlite-users mailing list
> > > sqlite-users at mailinglists.sqlite.org
> > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> > >
> > ___
> > sqlite-users mailing list
> > sqlite-users at mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

Not sure, what is instrumented?

RBS
On 26 Dec 2015 7:52 pm, "Keith Medcalf"  wrote:

>
> Have you instrumented your code?
>
> > -Original Message-
> > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-
> > bounces at mailinglists.sqlite.org] On Behalf Of Bart Smissaert
> > Sent: Saturday, 26 December, 2015 12:33
> > To: SQLite mailing list
> > Subject: Re: [sqlite] Malloc getting segmentation fault in sqlite
> >
> > VB6 is the original Visual Basic, unfortunately ditched by MS some years
> > ago and replaced by .Net.
> >
> > Yes, I am sure it is some kind of memory problem, but where (what line)
> > could it possibly occur?
> > All I can think of is this line:
> > sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
> > SQLITE_TRANSIENT
> > But as it uses SQLITE_TRANSIENT, I understand SQLite will make  local
> copy
> > and deal with it properly itself.
> >
> > All the other code is just copying data from SQLite to local VB6
> > variables.
> >
> > RBS
> >
> >
> >
> >
> > On Sat, Dec 26, 2015 at 6:52 PM, Simon Slavin 
> > wrote:
> >
> > >
> > > On 26 Dec 2015, at 5:23pm, Bart Smissaert 
> > > wrote:
> > >
> > > >> all is perfectly fine when the SQL statement has only one UDF in it
> > > >
> > > > Just a correction on this. Have now also seen problems when there is
> > only
> > > > one UDF in the SQL statement.
> > > > It is not entirely predictable, so sometimes get crash and on other
> > > > occasions I don't.
> > >
> > > That is typical of memory mis-management problems everyone runs into
> > from
> > > time to time.  You are probably releasing or overwriting memory which
> > > SQLite thinks it can depend on.  Unfortunately I don't know the version
> > of
> > > BASIC you're using, but your commands 'ReDim', for example, make me
> > think
> > > memory is being released.
> > >
> > > Simon.
> > > ___
> > > sqlite-users mailing list
> > > sqlite-users at mailinglists.sqlite.org
> > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> > >
> > ___
> > sqlite-users mailing list
> > sqlite-users at mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

VB6 is the original Visual Basic, unfortunately ditched by MS some years
ago and replaced by .Net.

Yes, I am sure it is some kind of memory problem, but where (what line)
could it possibly occur?
All I can think of is this line:
sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
SQLITE_TRANSIENT
But as it uses SQLITE_TRANSIENT, I understand SQLite will make  local copy
and deal with it properly itself.

All the other code is just copying data from SQLite to local VB6 variables.

RBS




On Sat, Dec 26, 2015 at 6:52 PM, Simon Slavin  wrote:

>
> On 26 Dec 2015, at 5:23pm, Bart Smissaert 
> wrote:
>
> >> all is perfectly fine when the SQL statement has only one UDF in it
> >
> > Just a correction on this. Have now also seen problems when there is only
> > one UDF in the SQL statement.
> > It is not entirely predictable, so sometimes get crash and on other
> > occasions I don't.
>
> That is typical of memory mis-management problems everyone runs into from
> time to time.  You are probably releasing or overwriting memory which
> SQLite thinks it can depend on.  Unfortunately I don't know the version of
> BASIC you're using, but your commands 'ReDim', for example, make me think
> memory is being released.
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Simon Slavin]

On 26 Dec 2015, at 5:23pm, Bart Smissaert  wrote:

>> all is perfectly fine when the SQL statement has only one UDF in it
> 
> Just a correction on this. Have now also seen problems when there is only
> one UDF in the SQL statement.
> It is not entirely predictable, so sometimes get crash and on other
> occasions I don't.

That is typical of memory mis-management problems everyone runs into from time 
to time.  You are probably releasing or overwriting memory which SQLite thinks 
it can depend on.  Unfortunately I don't know the version of BASIC you're 
using, but your commands 'ReDim', for example, make me think memory is being 
released.

Simon.

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

> all is perfectly fine when the SQL statement has only one UDF in it

Just a correction on this. Have now also seen problems when there is only
one UDF in the SQL statement.
It is not entirely predictable, so sometimes get crash and on other
occasions I don't.

RBS

On Fri, Dec 25, 2015 at 12:31 PM, Bart Smissaert 
wrote:

> Thanks for clearing that up.
> I know all this has very little to do with SQLite, but people using SQLite
> with VB6 or VBA might be interested in this.
> What puzzles me that all is perfectly fine when the SQL statement has only
> one UDF in it, but there is a serious problem
> when there are two in that same statement.
>
> I made a simple procedure that finds string x in string y and clears that
> string y, starting at the point where string x occurs:
>
> Sub ClearStartAtFixedStringX(ByVal lPtr_ObjContext As Long, _
>  ByVal lArgCount As Long, _
>  ByVal lPtr_ObjSQLite3_Value As Long)
> Dim i As Long
> Dim lPtr1 As Long
> Dim lPtr2 As Long
> Dim lPos As Long
> Dim lBytes1 As Long
> Dim arrBytes1() As Byte
> 10  On Error GoTo ERROROUT
> 'field value to alter
> '
> 20  lPtr1 = MemLong(lPtr_ObjSQLite3_Value)
> 30  LogData strLogFile, vbCrLf & "ClearStartAtFixedStringX line 20,
> lPtr1: " & lPtr1
> 40  lBytes1 = sqlite3_value_bytes(lPtr1)
> 50  If lBytes1 = 0 Then
> 60sqlite3_result_null lPtr_ObjContext
> 70Exit Sub
> 80  End If
> 90  LogData strLogFile, "ClearStartAtFixedStringX line 90, lBytes1: "
> & lBytes1
> 100 lPtr2 = sqlite3_value_text(lPtr1)
> 110 LogData strLogFile, "ClearStartAtFixedStringX line 110, lPtr2: " &
> lPtr2
> 120 ReDim arrBytes1(lBytes1 - 1) As Byte
> 'just CopyMemory API might be faster here
> 130 For i = 0 To lBytes1 - 1
> 140   arrBytes1(i) = MemByte(lPtr2 + i)
> 150 Next i
> 'this shows that for normal ANSI characters we have one byte
> 'per character here for the SQLite UTF-8 string
> '---
> 160 LogData strLogFile, "ClearStartAtFixedStringX line 160,
> BytesAsString(arrBytes1): " & _
> BytesAsString(arrBytes1)
> 170 If bDoneArrBytes2 = False Then
>   'this will fill up arrBytes2 and set lBytes2
>   'as the string to find is fixed we only need to do this once
>   '---
> 180   On Error GoTo 0
> 190   MakeArrBytes2 lPtr_ObjSQLite3_Value + 4
> 200   On Error GoTo ERROROUT
> 210 End If
> 220 On Error GoTo 0
> 230 lPos = GetStringPosB(arrBytes1, arrBytes2, False)
> 240 On Error GoTo ERROROUT
> 250 LogData strLogFile, "ClearStartAtFixedStringX line 250, lPos: " &
> lPos
> 'string not found, so return original field string
> '-
> 260 If lPos = 0 Then
> 270   sqlite3_result_value lPtr_ObjContext, lPtr1
> 280   Exit Sub
> 290 End If
> 'lPos -1 because if string found at byte position 2 then we only
> want one byte
>
> '-
> 300 LogData strLogFile, "ClearStartAtFixedStringX line 250,
> VarPtr(arrBytes1(0)): " & _
> VarPtr(arrBytes1(0))
> 310 sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos -
> 1, SQLITE_TRANSIENT
> 320 LogData strLogFile, "ClearStartAtFixedStringX line 310 (last line
> in UDF), error message: " & _
>
> cSQL.PointerToString(sqlite3_errmsg(lDBHandle)) & vbCrLf
> 330 Exit Sub
> ERROROUT:
> 340 cMsgBoxClass.MsgBoxDLL oExcel, _
>Err.Description, _
>"ClearStartAtFixedStringX error at line " &
> Erl, _
>lFormColour:=ColourMainForm,
> lButtonColour:=ColourButtons, _
>
> lActiveButtonColour:=ColourMsgBoxActiveButton
> End Sub
>
> I am testing this on a table with a text field holding this string:
> a(((bcdefghijklmnopqrstuvwyz
>
> This string data comes from Excel, so they are Unicode strings, 2 bytes
> per character.
> They are stored in SQLite as UTF-8.
>
> Now if I run this SQL:
> SELECT
> ClearStartAtFixedStringX(FIELD1,'(((') as x1
> FROM
> CLEAR_FROM
>
> Then all is perfectly fine, so it will produce the string: a
>
> But when I run instead this SQL:
> SELECT
> ClearStartAtFixedStringX(FIELD1,'(((') as x1,
> ClearStartAtFixedStringX(FIELD1,'(((') as x2
> FROM
> CLEAR_FROM
>
> I get a crash, producing the following message in Excel:
>
>   Problem Event Name: APPCRASH
>   Application Name: EXCEL.EXE
>   Application Version: 12.0.6739.5000
>   Application Timestamp: 5643f6e7
>   Fault Module Name: sqlite3.dll
>   Fault Module Version: 3.9.2.0
>   Fault Module Timestamp: 5637af4b
>   Exception Code: c005

[sqlite] Malloc getting segmentation fault in sqlite

[Keith Medcalf]

When the functionis entered, open a file for appending.
Just before each line have it write:
"Just about to execute line #" to the file.
Flush the file buffers after each write.

When your code crashes, open the file with a text editor.
The last line of the file will point to the line which caused the error 
condition to be detected.

Then you will know which line could be related to the problem.

> -Original Message-
> From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-
> bounces at mailinglists.sqlite.org] On Behalf Of Bart Smissaert
> Sent: Saturday, 26 December, 2015 12:55
> To: General Discussion of SQLite Database
> Subject: Re: [sqlite] Malloc getting segmentation fault in sqlite
> 
> Not sure, what is instrumented?
> 
> RBS
> On 26 Dec 2015 7:52 pm, "Keith Medcalf"  wrote:
> 
> >
> > Have you instrumented your code?
> >
> > > -Original Message-
> > > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-
> users-
> > > bounces at mailinglists.sqlite.org] On Behalf Of Bart Smissaert
> > > Sent: Saturday, 26 December, 2015 12:33
> > > To: SQLite mailing list
> > > Subject: Re: [sqlite] Malloc getting segmentation fault in sqlite
> > >
> > > VB6 is the original Visual Basic, unfortunately ditched by MS some
> years
> > > ago and replaced by .Net.
> > >
> > > Yes, I am sure it is some kind of memory problem, but where (what
> line)
> > > could it possibly occur?
> > > All I can think of is this line:
> > > sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
> > > SQLITE_TRANSIENT
> > > But as it uses SQLITE_TRANSIENT, I understand SQLite will make  local
> > copy
> > > and deal with it properly itself.
> > >
> > > All the other code is just copying data from SQLite to local VB6
> > > variables.
> > >
> > > RBS
> > >
> > >
> > >
> > >
> > > On Sat, Dec 26, 2015 at 6:52 PM, Simon Slavin 
> > > wrote:
> > >
> > > >
> > > > On 26 Dec 2015, at 5:23pm, Bart Smissaert 
> > > > wrote:
> > > >
> > > > >> all is perfectly fine when the SQL statement has only one UDF in
> it
> > > > >
> > > > > Just a correction on this. Have now also seen problems when there
> is
> > > only
> > > > > one UDF in the SQL statement.
> > > > > It is not entirely predictable, so sometimes get crash and on
> other
> > > > > occasions I don't.
> > > >
> > > > That is typical of memory mis-management problems everyone runs into
> > > from
> > > > time to time.  You are probably releasing or overwriting memory
> which
> > > > SQLite thinks it can depend on.  Unfortunately I don't know the
> version
> > > of
> > > > BASIC you're using, but your commands 'ReDim', for example, make me
> > > think
> > > > memory is being released.
> > > >
> > > > Simon.
> > > > ___
> > > > sqlite-users mailing list
> > > > sqlite-users at mailinglists.sqlite.org
> > > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> > > >
> > > ___
> > > sqlite-users mailing list
> > > sqlite-users at mailinglists.sqlite.org
> > > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> >
> >
> > ___
> > sqlite-users mailing list
> > sqlite-users at mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Malloc getting segmentation fault in sqlite

[Keith Medcalf]

Have you instrumented your code?

> -Original Message-
> From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-
> bounces at mailinglists.sqlite.org] On Behalf Of Bart Smissaert
> Sent: Saturday, 26 December, 2015 12:33
> To: SQLite mailing list
> Subject: Re: [sqlite] Malloc getting segmentation fault in sqlite
> 
> VB6 is the original Visual Basic, unfortunately ditched by MS some years
> ago and replaced by .Net.
> 
> Yes, I am sure it is some kind of memory problem, but where (what line)
> could it possibly occur?
> All I can think of is this line:
> sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
> SQLITE_TRANSIENT
> But as it uses SQLITE_TRANSIENT, I understand SQLite will make  local copy
> and deal with it properly itself.
> 
> All the other code is just copying data from SQLite to local VB6
> variables.
> 
> RBS
> 
> 
> 
> 
> On Sat, Dec 26, 2015 at 6:52 PM, Simon Slavin 
> wrote:
> 
> >
> > On 26 Dec 2015, at 5:23pm, Bart Smissaert 
> > wrote:
> >
> > >> all is perfectly fine when the SQL statement has only one UDF in it
> > >
> > > Just a correction on this. Have now also seen problems when there is
> only
> > > one UDF in the SQL statement.
> > > It is not entirely predictable, so sometimes get crash and on other
> > > occasions I don't.
> >
> > That is typical of memory mis-management problems everyone runs into
> from
> > time to time.  You are probably releasing or overwriting memory which
> > SQLite thinks it can depend on.  Unfortunately I don't know the version
> of
> > BASIC you're using, but your commands 'ReDim', for example, make me
> think
> > memory is being released.
> >
> > Simon.
> > ___
> > sqlite-users mailing list
> > sqlite-users at mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

Thanks for that.
I do have a simple stdcall dll that sits in between my VB6 ActiveX dll and
sqlite3.dll
This does nil else than simply things like this:

SQLITE3_STDCALL_API int __stdcall
sqlite3_stdcall_column_count(sqlite3_stmt* pStmt)
{
return sqlite3_column_count(pStmt);
}

So, that takes care of your point 1 and this works all perfectly fine.

 > I think that using the AddressOf keyword won't really work with SQLite
in this context due to mismatched calling conventions (cdecl versus
stdcall).

This is interesting as I do use VB6 AddressOf to pass the pointer of the
UDF to SQLite.
It must be working OK though as I pass this pointer again via that stdcall
dll and also because there is no problem with only one UDF in a SQL
statement.

I have to mention that I do use a .tlb that sits in between my ActiveX dll
and that stdcall dll.
I don't think this is the problem, because it makes no difference if I do
instead plain Declares in my VB6 dll.

I take it your .net dll handles UDF's fine, even when there are 2 in one
SQL statement?

I agree it is all tricky, but apart from this particular UDF problem all
runs perfectly fine (both data producing SQL and non-data producing SQL)
and very fast as well. See also my reply to Mr Hipp.


RBS





On Thu, Dec 24, 2015 at 9:56 PM, Joe Mistachkin 
wrote:

>
> It's been quite a long while since I used vb6 on a regular basis; however,
> integrating with native DLLs can be quite tricky for several reasons:
>
> 1.  It cannot call any native function that does not conform to the
> "stdcall" calling convention.
>
> 2.  It has a very Win32-centric way of marshalling data types.
>
> 3.  Using 64-bit integers at all is somewhat tricky, IIRC.  You may need
> to use a ByVal structure to pass them and I cannot remember how to use them
> as returned values.
>
> 4.  Properly declaring and calling (e.g. using the ByVal keyword
> strategically) is critically important.  Another issue is ANSI versus
> Unicode (UCS2 for COM) versus UTF-8 (SQLite) and knowing when to use which
> and how to convert between them (or marshal them).
>
> 5.  Doing inbound callbacks to VB6 code is very very tricky, mostly due to
> [apartment] threading issues.  I think that using the AddressOf keyword
> won't really work with SQLite in this context due to mismatched calling
> conventions (cdecl versus stdcall).
>
> I'll try looking for my old VB6 SQLite integration code when I have some
> spare cycles.
>
> Sent from my iPhone
>
> > On Dec 24, 2015, at 11:10 AM, Bart Smissaert 
> wrote:
> >
> > OK, thanks
> > I don't use any of 1 to 3, I only use the standard SQLite functions such
> as
> > sqlite3_open_v2, sqlite3_prepare16_v2, sqlite3_step, sqlite3_bind,
> > sqlite3_column, sqlite3_create_function_v2, sqlite3_finalize,
> > sqlite3_reset, sqlite3_result, sqlite3_value and qlite3_close.
> > So, in that case I don't need sqlite3_free, sqlite3_malloc or
> > sqlite3_realloc, I take it.
> > Given that I don't use theses it then likely that my problem is to do
> with
> > a buffer overwrite?
> >
> > RBS
> >
> >
> >
> >> On Thu, Dec 24, 2015 at 8:35 PM, Richard Hipp  wrote:
> >>
> >>> On 12/24/15, Bart Smissaert  wrote:
> >>> My question is if there is ever any need in this situation to run one
> of
> >>> the sqlite3 memory procedures, that is
> >>> sqlite3_free, sqlite3_malloc or sqlite3_realloc?
> >>> Currently I am not using this anywhere in my VB6 code.
> >>> Should I?
> >>
> >> Cases when you might use sqlite3_malloc():
> >>
> >> (1) You are using on of SQLite's built-in memory allocators.  The
> >> built-in memory allocators are disabled unless you use certain
> >> compile-time options.  And even then, you have to turn them on using
> >> sqlite3_config(SQLITE_CONFIG_HEAP,...).
> >>
> >> (2) You need to use sqlite3_msize().
> >>
> >> (3) If you use sqlite3_mprintf(), then sqlite3_free() must be used to
> >> release the string once you are done with it.
> >>
> >> Otherwise, there is no real advantage to using SQLite memory allocator
> >> interface in place of your standards system memory allocator.
> >>
> >> --
> >> D. Richard Hipp
> >> drh at sqlite.org
> >> ___
> >> sqlite-users mailing list
> >> sqlite-users at mailinglists.sqlite.org
> >> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> > ___
> > sqlite-users mailing list
> > sqlite-users at mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

Thanks for clearing that up.
I know all this has very little to do with SQLite, but people using SQLite
with VB6 or VBA might be interested in this.
What puzzles me that all is perfectly fine when the SQL statement has only
one UDF in it, but there is a serious problem
when there are two in that same statement.

I made a simple procedure that finds string x in string y and clears that
string y, starting at the point where string x occurs:

Sub ClearStartAtFixedStringX(ByVal lPtr_ObjContext As Long, _
 ByVal lArgCount As Long, _
 ByVal lPtr_ObjSQLite3_Value As Long)
Dim i As Long
Dim lPtr1 As Long
Dim lPtr2 As Long
Dim lPos As Long
Dim lBytes1 As Long
Dim arrBytes1() As Byte
10  On Error GoTo ERROROUT
'field value to alter
'
20  lPtr1 = MemLong(lPtr_ObjSQLite3_Value)
30  LogData strLogFile, vbCrLf & "ClearStartAtFixedStringX line 20,
lPtr1: " & lPtr1
40  lBytes1 = sqlite3_value_bytes(lPtr1)
50  If lBytes1 = 0 Then
60sqlite3_result_null lPtr_ObjContext
70Exit Sub
80  End If
90  LogData strLogFile, "ClearStartAtFixedStringX line 90, lBytes1: " &
lBytes1
100 lPtr2 = sqlite3_value_text(lPtr1)
110 LogData strLogFile, "ClearStartAtFixedStringX line 110, lPtr2: " &
lPtr2
120 ReDim arrBytes1(lBytes1 - 1) As Byte
'just CopyMemory API might be faster here
130 For i = 0 To lBytes1 - 1
140   arrBytes1(i) = MemByte(lPtr2 + i)
150 Next i
'this shows that for normal ANSI characters we have one byte
'per character here for the SQLite UTF-8 string
'---
160 LogData strLogFile, "ClearStartAtFixedStringX line 160,
BytesAsString(arrBytes1): " & _
BytesAsString(arrBytes1)
170 If bDoneArrBytes2 = False Then
  'this will fill up arrBytes2 and set lBytes2
  'as the string to find is fixed we only need to do this once
  '---
180   On Error GoTo 0
190   MakeArrBytes2 lPtr_ObjSQLite3_Value + 4
200   On Error GoTo ERROROUT
210 End If
220 On Error GoTo 0
230 lPos = GetStringPosB(arrBytes1, arrBytes2, False)
240 On Error GoTo ERROROUT
250 LogData strLogFile, "ClearStartAtFixedStringX line 250, lPos: " &
lPos
'string not found, so return original field string
'-
260 If lPos = 0 Then
270   sqlite3_result_value lPtr_ObjContext, lPtr1
280   Exit Sub
290 End If
'lPos -1 because if string found at byte position 2 then we only
want one byte

'-
300 LogData strLogFile, "ClearStartAtFixedStringX line 250,
VarPtr(arrBytes1(0)): " & _
VarPtr(arrBytes1(0))
310 sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos -
1, SQLITE_TRANSIENT
320 LogData strLogFile, "ClearStartAtFixedStringX line 310 (last line
in UDF), error message: " & _
cSQL.PointerToString(sqlite3_errmsg(lDBHandle))
& vbCrLf
330 Exit Sub
ERROROUT:
340 cMsgBoxClass.MsgBoxDLL oExcel, _
   Err.Description, _
   "ClearStartAtFixedStringX error at line " &
Erl, _
   lFormColour:=ColourMainForm,
lButtonColour:=ColourButtons, _
   lActiveButtonColour:=ColourMsgBoxActiveButton
End Sub

I am testing this on a table with a text field holding this string:
a(((bcdefghijklmnopqrstuvwyz

This string data comes from Excel, so they are Unicode strings, 2 bytes per
character.
They are stored in SQLite as UTF-8.

Now if I run this SQL:
SELECT
ClearStartAtFixedStringX(FIELD1,'(((') as x1
FROM
CLEAR_FROM

Then all is perfectly fine, so it will produce the string: a

But when I run instead this SQL:
SELECT
ClearStartAtFixedStringX(FIELD1,'(((') as x1,
ClearStartAtFixedStringX(FIELD1,'(((') as x2
FROM
CLEAR_FROM

I get a crash, producing the following message in Excel:

  Problem Event Name: APPCRASH
  Application Name: EXCEL.EXE
  Application Version: 12.0.6739.5000
  Application Timestamp: 5643f6e7
  Fault Module Name: sqlite3.dll
  Fault Module Version: 3.9.2.0
  Fault Module Timestamp: 5637af4b
  Exception Code: c005
  Exception Offset: 0005a319
  OS Version: 6.1.7601.2.1.0.256.48
  Locale ID: 2057

This crash happens directly after the second run of the above callback
procedure, although that procedure
runs to full completion with no errors.

My own debug output (LogData strLogFile etc.) up to that point is all fine:

ClearStartAtFixedStringX line 20, lPtr1: 33418304
ClearStartAtFixedStringX line 90, lBytes1: 28
ClearStartAtFixedStringX line 110, lPtr2: 33539976
ClearStartAtFixedStringX line 160, 

[sqlite] Malloc getting segmentation fault in sqlite

[Dan Kennedy]

On 12/24/2015 05:02 PM, santosh dasimanth wrote:
> Hi All,
> I am working on Sqlite in multi threaded environment on ARM v7 platform.
>
> I am facing problems with malloc() function returning segmentation fault.
> The problem is not frequent but out of 100 times am getting this once.
>
> The backtrace is pasted below.
>
> (gdb) bt
> #0  0x4038eb18 in malloc () from /lib/libc.so.0
> #1  0x401e0758 in sqlite3MallocRaw () from /pfrm2.0/lib/libsqlite3.so.0
> #2  0x401e08a8 in sqlite3Malloc () from /pfrm2.0/lib/libsqlite3.so.0
> #3  0x401e6254 in sqlite3VdbeCreate () from /pfrm2.0/lib/libsqlite3.so.0
> #4  0x401d99cc in sqlite3GetVdbe () from /pfrm2.0/lib/libsqlite3.so.0
> #5  0x401bd780 in sqlite3FinishCoding () from /pfrm2.0/lib/libsqlite3.so.0
> #6  0x401d2464 in sqlite3Parser () from /pfrm2.0/lib/libsqlite3.so.0
> #7  0x401dd664 in sqlite3RunParser () from /pfrm2.0/lib/libsqlite3.so.0
> #8  0x401d650c in sqlite3Prepare () from /pfrm2.0/lib/libsqlite3.so.0
> #9  0x401d69a4 in sqlite3_prepare () from /pfrm2.0/lib/libsqlite3.so.0
> #10 0x401ed5c0 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
> #11 0x401d5dbc in sqlite3InitCallback () from /pfrm2.0/lib/libsqlite3.so.0
> #12 0x401ed6f8 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
> #13 0x401d6184 in ?? () from /pfrm2.0/lib/libsqlite3.so.0
>
> The traces are pointing to different functions when I hit the issue at
> times.
> Please let me know if anyone of people faced this problem before with
> sqlite.

You have a corrupted heap in your application. Usually this is caused by 
calling free() or similar on a pointer that you should not have, but can 
also be due to large buffer overwrites and so on.

If possible, run your application under valgrind - either on the ARM 
platform or on a workstation. It will very likely tell you what is going 
wrong.

   http://valgrind.org/

Dan.

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

I have no problems at all with sqlite3_bind, but I understand that the
sqlite3_result group of procedures is very similar to sqlite3_bind group.

In a UDF procedure (callback procedure in the ActiveX dll) I do things like
this:

sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
SQLITE_TRANSIENT

where arrBytes1 is a local variable, a Byte array.
This goes out of scope once the callback procedure finishes.
Could that be a problem?

RBS

On Thu, Dec 24, 2015 at 10:17 PM, Simon Slavin  wrote:

>
> On 24 Dec 2015, at 9:10pm, Bart Smissaert 
> wrote:
>
> > Given that I don't use theses it then likely that my problem is to do
> with
> > a buffer overwrite?
>
> You say you use sqlite3_bind().  See the details about the fifth parameter
> in
>
> 
>
> A common mistake is to bind a string to a parameter then free the space
> before SQLite is finished with it.  This, of course, leads to OS errors.
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Simon Slavin]

On 24 Dec 2015, at 9:10pm, Bart Smissaert  wrote:

> Given that I don't use theses it then likely that my problem is to do with
> a buffer overwrite?

You say you use sqlite3_bind().  See the details about the fifth parameter in



A common mistake is to bind a string to a parameter then free the space before 
SQLite is finished with it.  This, of course, leads to OS errors.

Simon.

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

OK, thanks
I don't use any of 1 to 3, I only use the standard SQLite functions such as
sqlite3_open_v2, sqlite3_prepare16_v2, sqlite3_step, sqlite3_bind,
sqlite3_column, sqlite3_create_function_v2, sqlite3_finalize,
sqlite3_reset, sqlite3_result, sqlite3_value and qlite3_close.
So, in that case I don't need sqlite3_free, sqlite3_malloc or
sqlite3_realloc, I take it.
Given that I don't use theses it then likely that my problem is to do with
a buffer overwrite?

RBS



On Thu, Dec 24, 2015 at 8:35 PM, Richard Hipp  wrote:

> On 12/24/15, Bart Smissaert  wrote:
> > My question is if there is ever any need in this situation to run one of
> > the sqlite3 memory procedures, that is
> > sqlite3_free, sqlite3_malloc or sqlite3_realloc?
> > Currently I am not using this anywhere in my VB6 code.
> > Should I?
>
> Cases when you might use sqlite3_malloc():
>
> (1) You are using on of SQLite's built-in memory allocators.  The
> built-in memory allocators are disabled unless you use certain
> compile-time options.  And even then, you have to turn them on using
> sqlite3_config(SQLITE_CONFIG_HEAP,...).
>
> (2) You need to use sqlite3_msize().
>
> (3) If you use sqlite3_mprintf(), then sqlite3_free() must be used to
> release the string once you are done with it.
>
> Otherwise, there is no real advantage to using SQLite memory allocator
> interface in place of your standards system memory allocator.
>
> --
> D. Richard Hipp
> drh at sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Bart Smissaert]

I am interested in this as I have bug that I am sure is to do with some
sort of memory problem.
It only occurs when I run a procedure defined with sqlite3_create_function.
This procedure is not
in sqlite3.dll but in a VB6 ActiveX dll. I use the unaltered Windows
sqlite3.dll.
My question is if there is ever any need in this situation to run one of
the sqlite3 memory procedures, that is
sqlite3_free, sqlite3_malloc or sqlite3_realloc?
Currently I am not using this anywhere in my VB6 code.
Should I?

RBS




On Thu, Dec 24, 2015 at 4:12 PM, Dan Kennedy  wrote:

> On 12/24/2015 05:02 PM, santosh dasimanth wrote:
>
>> Hi All,
>> I am working on Sqlite in multi threaded environment on ARM v7 platform.
>>
>> I am facing problems with malloc() function returning segmentation fault.
>> The problem is not frequent but out of 100 times am getting this once.
>>
>> The backtrace is pasted below.
>>
>> (gdb) bt
>> #0  0x4038eb18 in malloc () from /lib/libc.so.0
>> #1  0x401e0758 in sqlite3MallocRaw () from /pfrm2.0/lib/libsqlite3.so.0
>> #2  0x401e08a8 in sqlite3Malloc () from /pfrm2.0/lib/libsqlite3.so.0
>> #3  0x401e6254 in sqlite3VdbeCreate () from /pfrm2.0/lib/libsqlite3.so.0
>> #4  0x401d99cc in sqlite3GetVdbe () from /pfrm2.0/lib/libsqlite3.so.0
>> #5  0x401bd780 in sqlite3FinishCoding () from /pfrm2.0/lib/libsqlite3.so.0
>> #6  0x401d2464 in sqlite3Parser () from /pfrm2.0/lib/libsqlite3.so.0
>> #7  0x401dd664 in sqlite3RunParser () from /pfrm2.0/lib/libsqlite3.so.0
>> #8  0x401d650c in sqlite3Prepare () from /pfrm2.0/lib/libsqlite3.so.0
>> #9  0x401d69a4 in sqlite3_prepare () from /pfrm2.0/lib/libsqlite3.so.0
>> #10 0x401ed5c0 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
>> #11 0x401d5dbc in sqlite3InitCallback () from /pfrm2.0/lib/libsqlite3.so.0
>> #12 0x401ed6f8 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
>> #13 0x401d6184 in ?? () from /pfrm2.0/lib/libsqlite3.so.0
>>
>> The traces are pointing to different functions when I hit the issue at
>> times.
>> Please let me know if anyone of people faced this problem before with
>> sqlite.
>>
>
> You have a corrupted heap in your application. Usually this is caused by
> calling free() or similar on a pointer that you should not have, but can
> also be due to large buffer overwrites and so on.
>
> If possible, run your application under valgrind - either on the ARM
> platform or on a workstation. It will very likely tell you what is going
> wrong.
>
>   http://valgrind.org/
>
> Dan.
>
>
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Malloc getting segmentation fault in sqlite

[Richard Hipp]

On 12/24/15, Bart Smissaert  wrote:
> I have no problems at all with sqlite3_bind, but I understand that the
> sqlite3_result group of procedures is very similar to sqlite3_bind group.
>
> In a UDF procedure (callback procedure in the ActiveX dll) I do things like
> this:
>
> sqlite3_result_text lPtr_ObjContext, VarPtr(arrBytes1(0)), lPos - 1,
> SQLITE_TRANSIENT
>
> where arrBytes1 is a local variable, a Byte array.
> This goes out of scope once the callback procedure finishes.
> Could that be a problem?
>

No.  SQLITE_TRANSIENT causes SQLite to make its own private copy of the string.
-- 
D. Richard Hipp
drh at sqlite.org

[sqlite] Malloc getting segmentation fault in sqlite

[Richard Hipp]

On 12/24/15, Bart Smissaert  wrote:
> My question is if there is ever any need in this situation to run one of
> the sqlite3 memory procedures, that is
> sqlite3_free, sqlite3_malloc or sqlite3_realloc?
> Currently I am not using this anywhere in my VB6 code.
> Should I?

Cases when you might use sqlite3_malloc():

(1) You are using on of SQLite's built-in memory allocators.  The
built-in memory allocators are disabled unless you use certain
compile-time options.  And even then, you have to turn them on using
sqlite3_config(SQLITE_CONFIG_HEAP,...).

(2) You need to use sqlite3_msize().

(3) If you use sqlite3_mprintf(), then sqlite3_free() must be used to
release the string once you are done with it.

Otherwise, there is no real advantage to using SQLite memory allocator
interface in place of your standards system memory allocator.

-- 
D. Richard Hipp
drh at sqlite.org

[sqlite] Malloc getting segmentation fault in sqlite

[santosh dasimanth]

Hi All,
I am working on Sqlite in multi threaded environment on ARM v7 platform.

I am facing problems with malloc() function returning segmentation fault.
The problem is not frequent but out of 100 times am getting this once.

The backtrace is pasted below.

(gdb) bt
#0  0x4038eb18 in malloc () from /lib/libc.so.0
#1  0x401e0758 in sqlite3MallocRaw () from /pfrm2.0/lib/libsqlite3.so.0
#2  0x401e08a8 in sqlite3Malloc () from /pfrm2.0/lib/libsqlite3.so.0
#3  0x401e6254 in sqlite3VdbeCreate () from /pfrm2.0/lib/libsqlite3.so.0
#4  0x401d99cc in sqlite3GetVdbe () from /pfrm2.0/lib/libsqlite3.so.0
#5  0x401bd780 in sqlite3FinishCoding () from /pfrm2.0/lib/libsqlite3.so.0
#6  0x401d2464 in sqlite3Parser () from /pfrm2.0/lib/libsqlite3.so.0
#7  0x401dd664 in sqlite3RunParser () from /pfrm2.0/lib/libsqlite3.so.0
#8  0x401d650c in sqlite3Prepare () from /pfrm2.0/lib/libsqlite3.so.0
#9  0x401d69a4 in sqlite3_prepare () from /pfrm2.0/lib/libsqlite3.so.0
#10 0x401ed5c0 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
#11 0x401d5dbc in sqlite3InitCallback () from /pfrm2.0/lib/libsqlite3.so.0
#12 0x401ed6f8 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
#13 0x401d6184 in ?? () from /pfrm2.0/lib/libsqlite3.so.0

The traces are pointing to different functions when I hit the issue at
times.
Please let me know if anyone of people faced this problem before with
sqlite.

Regards,
Santosh

[sqlite] Malloc getting segmentation fault in sqlite

[Joe Mistachkin]

It's been quite a long while since I used vb6 on a regular basis; however, 
integrating with native DLLs can be quite tricky for several reasons:

1.  It cannot call any native function that does not conform to the "stdcall" 
calling convention.

2.  It has a very Win32-centric way of marshalling data types.

3.  Using 64-bit integers at all is somewhat tricky, IIRC.  You may need to use 
a ByVal structure to pass them and I cannot remember how to use them as 
returned values.

4.  Properly declaring and calling (e.g. using the ByVal keyword strategically) 
is critically important.  Another issue is ANSI versus Unicode (UCS2 for COM) 
versus UTF-8 (SQLite) and knowing when to use which and how to convert between 
them (or marshal them).

5.  Doing inbound callbacks to VB6 code is very very tricky, mostly due to 
[apartment] threading issues.  I think that using the AddressOf keyword won't 
really work with SQLite in this context due to mismatched calling conventions 
(cdecl versus stdcall).

I'll try looking for my old VB6 SQLite integration code when I have some spare 
cycles.

Sent from my iPhone

> On Dec 24, 2015, at 11:10 AM, Bart Smissaert  
> wrote:
> 
> OK, thanks
> I don't use any of 1 to 3, I only use the standard SQLite functions such as
> sqlite3_open_v2, sqlite3_prepare16_v2, sqlite3_step, sqlite3_bind,
> sqlite3_column, sqlite3_create_function_v2, sqlite3_finalize,
> sqlite3_reset, sqlite3_result, sqlite3_value and qlite3_close.
> So, in that case I don't need sqlite3_free, sqlite3_malloc or
> sqlite3_realloc, I take it.
> Given that I don't use theses it then likely that my problem is to do with
> a buffer overwrite?
> 
> RBS
> 
> 
> 
>> On Thu, Dec 24, 2015 at 8:35 PM, Richard Hipp  wrote:
>> 
>>> On 12/24/15, Bart Smissaert  wrote:
>>> My question is if there is ever any need in this situation to run one of
>>> the sqlite3 memory procedures, that is
>>> sqlite3_free, sqlite3_malloc or sqlite3_realloc?
>>> Currently I am not using this anywhere in my VB6 code.
>>> Should I?
>> 
>> Cases when you might use sqlite3_malloc():
>> 
>> (1) You are using on of SQLite's built-in memory allocators.  The
>> built-in memory allocators are disabled unless you use certain
>> compile-time options.  And even then, you have to turn them on using
>> sqlite3_config(SQLITE_CONFIG_HEAP,...).
>> 
>> (2) You need to use sqlite3_msize().
>> 
>> (3) If you use sqlite3_mprintf(), then sqlite3_free() must be used to
>> release the string once you are done with it.
>> 
>> Otherwise, there is no real advantage to using SQLite memory allocator
>> interface in place of your standards system memory allocator.
>> 
>> --
>> D. Richard Hipp
>> drh at sqlite.org
>> ___
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> 

[sqlite] Malloc getting segmentation fault in sqlite

[J Decker]

Yes when I free'd memory I shouldn't have and kept a reference to it
in other places... or that after freeing I had modified memory.  But
not from sqlite itself.

On Thu, Dec 24, 2015 at 2:02 AM, santosh dasimanth
 wrote:
> Hi All,
> I am working on Sqlite in multi threaded environment on ARM v7 platform.
>
> I am facing problems with malloc() function returning segmentation fault.
> The problem is not frequent but out of 100 times am getting this once.
>
> The backtrace is pasted below.
>
> (gdb) bt
> #0  0x4038eb18 in malloc () from /lib/libc.so.0
> #1  0x401e0758 in sqlite3MallocRaw () from /pfrm2.0/lib/libsqlite3.so.0
> #2  0x401e08a8 in sqlite3Malloc () from /pfrm2.0/lib/libsqlite3.so.0
> #3  0x401e6254 in sqlite3VdbeCreate () from /pfrm2.0/lib/libsqlite3.so.0
> #4  0x401d99cc in sqlite3GetVdbe () from /pfrm2.0/lib/libsqlite3.so.0
> #5  0x401bd780 in sqlite3FinishCoding () from /pfrm2.0/lib/libsqlite3.so.0
> #6  0x401d2464 in sqlite3Parser () from /pfrm2.0/lib/libsqlite3.so.0
> #7  0x401dd664 in sqlite3RunParser () from /pfrm2.0/lib/libsqlite3.so.0
> #8  0x401d650c in sqlite3Prepare () from /pfrm2.0/lib/libsqlite3.so.0
> #9  0x401d69a4 in sqlite3_prepare () from /pfrm2.0/lib/libsqlite3.so.0
> #10 0x401ed5c0 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
> #11 0x401d5dbc in sqlite3InitCallback () from /pfrm2.0/lib/libsqlite3.so.0
> #12 0x401ed6f8 in sqlite3_exec () from /pfrm2.0/lib/libsqlite3.so.0
> #13 0x401d6184 in ?? () from /pfrm2.0/lib/libsqlite3.so.0
>
> The traces are pointing to different functions when I hit the issue at
> times.
> Please let me know if anyone of people faced this problem before with
> sqlite.
>
> Regards,
> Santosh
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users