Dear Andrea,
I am also facing problem with sqlite closed by windows because of crash.
On same data this query crash:
select load_extension("libstringmetrics.dll");
select a.vorname, b.vorname, a.nachname, b.nachname,
stringmetrics("qgrams_distance","similarity",a.nachname, b.nachname, "")
nach_dist
from r2 a, allUsers b
where
a.vorname like 'r%' and b.vorname like 'r%' and
round(nach_dist) between 50 and 100
;
but this query pass:
select load_extension("libstringmetrics.dll");
select a.vorname, b.vorname, a.nachname, b.nachname,
stringmetrics("qgrams_distance_custom","similarity",a.nachname,
b.nachname, "") nach_dist
from r2 a, allUsers b
where
a.vorname like 'r%' and b.vorname like 'r%' and
round(nach_dist) between 50 and 100
;
The first query is able to pass on some small sample, but not on all my
data. I am trying to find what concrete string comparison is crashing
it, but it is slow with crashes. Is there any way how to start logging
the calls of the library to find the exact data that are causing the
crash?
Thank you
Best Regards
Milan
On 2015-03-11 16:07, Milan Roubal wrote:
> Dear Andrea,
> thank you for the answer. The only idea I have so far is the line 452
> in file wrapper_functions.c
> sqlite3_result_text(context, metrics, strlen(metrics)+1,
> SQLITE_STATIC);
> I would change that to
> sqlite3_result_text(context, metrics, strlen(metrics)+1,
> SQLITE_TRANSIENT);
> but I don't have compiler to verify if this idea makes sense.
> Thank you
> Best Regards
> Milan
>
> On 2015-03-11 00:15, aperi2007 wrote:
>> Hi Milan,
>>
>> thx for your report.
>>
>> I try to see a check, and effectively I notice the problem is in the
>> qgrams_distance when used the metrics option.
>>
>> As reported in the readme, the stringmetrics extension is based on the
>> https://github.com/jokillsya/libsimmetrics
>> library.
>> The code on the metrics is integrally take from the sample of that
>> library.
>>
>> I guess the problem you report is due to a static allocation (or
>> similar effect) that is not appropriated when in the same query the
>> same function is called more than one time.
>>
>> Actually I dont know if the problem is in the sqlite side of my code
>> or in the original code of the symmetrics library.
>>
>> I try to resolve this firstly question when and if I find a few of
>> time.
>>
>> However patches are welcomes.
>> :)
>>
>> Regards,
>>
>> Andrea Peri.
>>
>> Il 06/03/2015 17:00, Milan Roubal ha scritto:
>>> Dear all,
>>> I have some problems with
>>> https://github.com/aperi2007/libstringmetrics . For example:
>>> When I use "similarity" in qgrams_distance, I get good results. But
>>> when I use "metric", it works only if it is only once in the query.
>>> When there are 2 different usages, they somehow interfere together.
>>> Is this the right place where to report such problem?
>>>
>>>> sqlite3
>>> SQLite version 3.8.8.3 2015-02-25 13:29:11
>>> Enter ".help" for usage hints.
>>> Connected to a transient in-memory database.
>>> Use ".open FILENAME" to reopen on a persistent database.
>>>
>>> sqlite> select load_extension("libstringmetrics.dll");
>>>
>>> sqlite> select a.firstname, b.firstname, a.lastname, b.lastname,
>>> ...> stringmetrics("qgrams_distance","similarity",a.firstname,
>>> b.firstname,"") first_dist,
>>> ...> stringmetrics("qgrams_distance","similarity",a.lastname,
>>> b.lastname,"") last_dist
>>> ...> from
>>> ...> (select "Milan" as firstname, "Roubal" as lastname ) a,
>>> ...> (select "Milan" as firstname, "roubal" as lastname ) b
>>> ...> ;
>>> Milan|Milan|Roubal|roubal|100.0|62.5
>>>
>>> sqlite> select a.firstname, b.firstname, a.lastname, b.lastname,
>>> ...> stringmetrics("qgrams_distance","metric",a.firstname,
>>> b.firstname,"") first_dist,
>>> ...> stringmetrics("qgrams_distance","metric",a.lastname,
>>> b.lastname,"") last_dist
>>> ...> from
>>> ...> (select "Milan" as firstname, "Roubal" as lastname ) a,
>>> ...> (select "Milan" as firstname, "roubal" as lastname ) b
>>> ...> ;
>>> Milan|Milan|Roubal|roubal|6|6
>>>
>>> sqlite> select a.firstname, b.firstname, a.lastname, b.lastname,
>>> ...> stringmetrics("qgrams_distance","metric",a.firstname,
>>> b.firstname,"") first_dist
>>> ...> from
>>> ...> (select "Milan" as firstname, "Roubal" as lastname ) a,
>>> ...> (select "Milan" as firstname, "roubal" as lastname ) b
>>> ...> ;
>>> Milan|Milan|Roubal|roubal|0
>>>
>>> Thank you
>>> Best Regards
>>> Milan
>>> _______________________________________________
>>> sqlite-users mailing list
>>> sqlite-users at mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users at mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
