Re: [sqlite] Error or Crash using OR in MATCH clause

2016-07-21 Thread Julian Dohmen 
Dan

Thank you, and another member - Simon Slavin, for replying so quickly. It does 
look exactly like my problem: we'll await 3.14 with interest.


Julian Dohmen | developer
Evelyn Manufacturing | Alexandria & KeepnTrack 
801.943.7277 ext 565 | jdoh...@companioncorp.com 





 From:   Dan Kennedy  
 To:
 Sent:   7/21/2016 12:01 PM 
 Subject:   Re: [sqlite] Error or Crash using OR in MATCH clause 

On 07/21/2016 11:16 PM, Julian Dohmen wrote: 
> I am encountering errors or crashes when using FTS5 and MATCH expressions 
> involving 
>   OR  
> combined with 
>   ORDER BY rank 
> = 
 
Thanks for the detailed report. I think this one was fixed a couple of  
weeks after 3.13 was released: 
 
   http://sqlite.org/src/info/64ca1a835a89fd21 
 
The fix will be part of 3.14. 
 
Dan. 
 
 
 
 
 
 
> 
> Simplest example [using SQLite program]: 
> --- 
> DROP TABLE IF EXISTS VTest; 
> CREATE virtual TABLE VTest USING FTS5(Title, AUthor, tokenize ='porter 
> unicode61 remove_diacritics 1', columnsize='1', detail=full); 
> INSERT INTO VTest (Title, Author) VALUES ('wrinkle in time', 'Bill Smith'); 
> 
> SELECT * FROM VTest; 
> OK 
> 
> SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time'; 
> OK 
> 
> SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time' ORDER BY rank 
> OK 
> 
> SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time' 
> ORDER BY rank; 
> ERROR 
> Error: database disk image is malformed 
> 
> Note that the error [or commonly a crash in my system with a real table/data] 
> seems to occur when the OR is combined with ORDER BY 
> These [simplified] results were obtained with the SQLite shell program built 
> with SQLite 3.13. 
> = 
> 
> SQLite build: [Windows 10, VS 2015] Preprocessor flags: 
> --- 
> WIN32 
> _DEBUG 
> _WINDOWS 
> SQLITE_THREADSAFE=2 
> SQLITE_ALLOW_COVERING_INDEX_SCAN=1 
> SQLITE_ENABLE_STAT4=1 
> SQLITE_TEMP_STORE=2 
> SQLITE_ENABLE_FTS5=1 
> SQLITE_ENABLE_FTS4=1 
> SQLITE_ENABLE_FTS3=1 
> SQLITE_ENABLE_FTS3_PARENTHESIS=1 
>  
> 
> I also see this kind of problem in SQLiteStudio with SQLite 3.12.2 and [most 
> importantly to us] in our application built with SQLite3.13 [with same flags 
> above] 
> I assume that I misunderstood something about the syntax or flags, but 
> reviewing the SQLite docs on FTS5 I don't see where I went wrong. 
> = 
> 
> Further details: 
> --- 
> In our program [32-bit debug build with above flags] with VS 2015 debugger 
> set to trap exceptions we see this stack trace 
> 
>       ()    Unknown 
>       [Frames below may be incorrect and/or missing] 
>      Alexandria.exe!fts5MultiIterNext(Fts5Index * p, Fts5Iter * pIter, int 
> bFrom, __int64 iFrom) Line 186755    C 
>       Alexandria.exe!fts5MultiIterNextFrom(Fts5Index * p, Fts5Iter * pIter, 
> __int64 iMatch) Line 187418    C 
>       Alexandria.exe!sqlite3Fts5IterNextFrom(Fts5IndexIter * pIndexIter, 
> __int64 iMatch) Line 189325    C 
>       Alexandria.exe!fts5ExprNodeNext_TERM(Fts5Expr * pExpr, Fts5ExprNode * 
> pNode, int bFromValid, __int64 iFrom) Line 181773    C 
>       Alexandria.exe!fts5ExprNodeTest_AND(Fts5Expr * pExpr, Fts5ExprNode * 
> pAnd) Line 181852    C 
>       Alexandria.exe!fts5ExprNodeNext_AND(Fts5Expr * pExpr, Fts5ExprNode * 
> pNode, int bFromValid, __int64 iFrom) Line 181891    C 
>       Alexandria.exe!fts5ExprNodeNext_OR(Fts5Expr * pExpr, Fts5ExprNode * 
> pNode, int bFromValid, __int64 iFrom) Line 181821    C 
>       Alexandria.exe!sqlite3Fts5ExprNext(Fts5Expr * p, __int64 iLast) Line 
> 182079    C 
>       Alexandria.exe!fts5NextMethod(sqlite3_vtab_cursor * pCursor) Line 
> 191208    C 
>       Alexandria.exe!sqlite3VdbeExec(Vdbe * p) Line 83260    C 
>       Alexandria.exe!sqlite3Step(Vdbe * p) Line 75131    C 
>       Alexandria.exe!sqlite3_step(sqlite3_stmt * pStmt) Line 75193    C 
>       Alexandria.exe!fts5SorterNext(Fts5Cursor * pCsr) Line 191102    C 
>       Alexandria.exe!fts5CursorFirstSorted(Fts5Table * pTab, Fts5Cursor * 
> pCsr, int bDesc) Line 191303    C 
>       Alexandria.exe!fts5FilterMethod(sqlite3_vtab_cursor * pCursor, int 
> idxNum, const char * zUnused, int nVal, Mem * * apVal) Line 191588    C 
>       Alexandria.exe!sqlite3VdbeExec(Vdbe * p) Line 83174    C 
>       Alexandria.exe!sqlite3Step(Vdbe * p) Line 75131    C 
>       Alexandria.exe!sqlite3_step(sqlite3_stmt *

Re: [sqlite] Error or Crash using OR in MATCH clause

2016-07-21 Thread Dan Kennedy 

On 07/21/2016 11:16 PM, Julian Dohmen wrote:

I am encountering errors or crashes when using FTS5 and MATCH expressions 
involving
  OR 
combined with
  ORDER BY rank
=


Thanks for the detailed report. I think this one was fixed a couple of 
weeks after 3.13 was released:


  http://sqlite.org/src/info/64ca1a835a89fd21

The fix will be part of 3.14.

Dan.








Simplest example [using SQLite program]:
---
DROP TABLE IF EXISTS VTest;
CREATE virtual TABLE VTest USING FTS5(Title, AUthor, tokenize ='porter 
unicode61 remove_diacritics 1', columnsize='1', detail=full);
INSERT INTO VTest (Title, Author) VALUES ('wrinkle in time', 'Bill Smith');

SELECT * FROM VTest;
OK

SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time';
OK

SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time' ORDER BY rank
OK

SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time' 
ORDER BY rank;
ERROR
Error: database disk image is malformed

Note that the error [or commonly a crash in my system with a real table/data] 
seems to occur when the OR is combined with ORDER BY
These [simplified] results were obtained with the SQLite shell program built 
with SQLite 3.13.
=

SQLite build: [Windows 10, VS 2015] Preprocessor flags:
---
WIN32
_DEBUG
_WINDOWS
SQLITE_THREADSAFE=2
SQLITE_ALLOW_COVERING_INDEX_SCAN=1
SQLITE_ENABLE_STAT4=1
SQLITE_TEMP_STORE=2
SQLITE_ENABLE_FTS5=1
SQLITE_ENABLE_FTS4=1
SQLITE_ENABLE_FTS3=1
SQLITE_ENABLE_FTS3_PARENTHESIS=1


I also see this kind of problem in SQLiteStudio with SQLite 3.12.2 and [most 
importantly to us] in our application built with SQLite3.13 [with same flags 
above]
I assume that I misunderstood something about the syntax or flags, but 
reviewing the SQLite docs on FTS5 I don't see where I went wrong.
=

Further details:
---
In our program [32-bit debug build with above flags] with VS 2015 debugger set 
to trap exceptions we see this stack trace

  ()Unknown
  [Frames below may be incorrect and/or missing]
 Alexandria.exe!fts5MultiIterNext(Fts5Index * p, Fts5Iter * pIter, int 
bFrom, __int64 iFrom) Line 186755C
  Alexandria.exe!fts5MultiIterNextFrom(Fts5Index * p, Fts5Iter * pIter, 
__int64 iMatch) Line 187418C
  Alexandria.exe!sqlite3Fts5IterNextFrom(Fts5IndexIter * pIndexIter, 
__int64 iMatch) Line 189325C
  Alexandria.exe!fts5ExprNodeNext_TERM(Fts5Expr * pExpr, Fts5ExprNode * 
pNode, int bFromValid, __int64 iFrom) Line 181773C
  Alexandria.exe!fts5ExprNodeTest_AND(Fts5Expr * pExpr, Fts5ExprNode * 
pAnd) Line 181852C
  Alexandria.exe!fts5ExprNodeNext_AND(Fts5Expr * pExpr, Fts5ExprNode * 
pNode, int bFromValid, __int64 iFrom) Line 181891C
  Alexandria.exe!fts5ExprNodeNext_OR(Fts5Expr * pExpr, Fts5ExprNode * 
pNode, int bFromValid, __int64 iFrom) Line 181821C
  Alexandria.exe!sqlite3Fts5ExprNext(Fts5Expr * p, __int64 iLast) Line 
182079C
  Alexandria.exe!fts5NextMethod(sqlite3_vtab_cursor * pCursor) Line 191208  
  C
  Alexandria.exe!sqlite3VdbeExec(Vdbe * p) Line 83260C
  Alexandria.exe!sqlite3Step(Vdbe * p) Line 75131C
  Alexandria.exe!sqlite3_step(sqlite3_stmt * pStmt) Line 75193C
  Alexandria.exe!fts5SorterNext(Fts5Cursor * pCsr) Line 191102C
  Alexandria.exe!fts5CursorFirstSorted(Fts5Table * pTab, Fts5Cursor * pCsr, 
int bDesc) Line 191303C
  Alexandria.exe!fts5FilterMethod(sqlite3_vtab_cursor * pCursor, int 
idxNum, const char * zUnused, int nVal, Mem * * apVal) Line 191588C
  Alexandria.exe!sqlite3VdbeExec(Vdbe * p) Line 83174C
  Alexandria.exe!sqlite3Step(Vdbe * p) Line 75131C
  Alexandria.exe!sqlite3_step(sqlite3_stmt * pStmt) Line 75193C
  Alexandria.exe!nsFTS5SearchService::`anonymous namespace'::DB_Worker::operator()(const 
std::basic_string,std::allocator > & csQuery, unsigned int 
nMaxToFind, 
std::vector
 > & vecResp) Line 76C++

With this query:
SELECT T2TFTSrsn FROM T2TitleFTS WHERE  T2TitleFTS MATCH '{T2TFTStitle 
T2TFTSseries}:a wrinkle in time OR wrinkle in time' ORDER BY rank LIMIT 50;
=



Julian Dohmen | developer
Evelyn Manufacturing | Alexandria & KeepnTrack
801.943.7277 ext 565 | jdoh...@companioncorp.com

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Error or Crash using OR in MATCH clause

2016-07-21 Thread Julian Dohmen 
In SQLite.exe [the source of the simple example I sent]
PRAGMA integrity_check;
Ok


Julian Dohmen | developer
Evelyn Manufacturing | Alexandria & KeepnTrack 
801.943.7277 ext 565 | jdoh...@companioncorp.com 





 From:   Simon Slavin  
 To:   SQLite mailing list  
 Sent:   7/21/2016 11:45 AM 
 Subject:   Re: [sqlite] Error or Crash using OR in MATCH clause 

 
On 21 Jul 2016, at 5:16pm, Julian Dohmen  wrote: 
 
> SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time' 
> ORDER BY rank; 
> ERROR 
> Error: database disk image is malformed 
 
What is the result of 
 
PRAGMA integrity_check 
 
? 
 
Simon. 
___ 
sqlite-users mailing list 
sqlite-users@mailinglists.sqlite.org 
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users 
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Error or Crash using OR in MATCH clause

2016-07-21 Thread Simon Slavin 

On 21 Jul 2016, at 5:16pm, Julian Dohmen  wrote:

> SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time' 
> ORDER BY rank;
> ERROR
> Error: database disk image is malformed

What is the result of

PRAGMA integrity_check

?

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Error or Crash using OR in MATCH clause

2016-07-21 Thread Julian Dohmen 
I am encountering errors or crashes when using FTS5 and MATCH expressions 
involving
 OR 
combined with
 ORDER BY rank
=

Simplest example [using SQLite program]:
---
DROP TABLE IF EXISTS VTest;
CREATE virtual TABLE VTest USING FTS5(Title, AUthor, tokenize ='porter 
unicode61 remove_diacritics 1', columnsize='1', detail=full);
INSERT INTO VTest (Title, Author) VALUES ('wrinkle in time', 'Bill Smith');

SELECT * FROM VTest;
OK

SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time';
OK

SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time' ORDER BY rank
OK

SELECT * FROM VTest WHERE VTest MATCH 'wrinkle in time OR a wrinkle in time' 
ORDER BY rank;
ERROR
Error: database disk image is malformed

Note that the error [or commonly a crash in my system with a real table/data] 
seems to occur when the OR is combined with ORDER BY
These [simplified] results were obtained with the SQLite shell program built 
with SQLite 3.13.
=

SQLite build: [Windows 10, VS 2015] Preprocessor flags:
---
WIN32
_DEBUG
_WINDOWS
SQLITE_THREADSAFE=2
SQLITE_ALLOW_COVERING_INDEX_SCAN=1
SQLITE_ENABLE_STAT4=1
SQLITE_TEMP_STORE=2
SQLITE_ENABLE_FTS5=1
SQLITE_ENABLE_FTS4=1
SQLITE_ENABLE_FTS3=1
SQLITE_ENABLE_FTS3_PARENTHESIS=1


I also see this kind of problem in SQLiteStudio with SQLite 3.12.2 and [most 
importantly to us] in our application built with SQLite3.13 [with same flags 
above]
I assume that I misunderstood something about the syntax or flags, but 
reviewing the SQLite docs on FTS5 I don't see where I went wrong.
=

Further details:
---
In our program [32-bit debug build with above flags] with VS 2015 debugger set 
to trap exceptions we see this stack trace

 ()    Unknown
 [Frames below may be incorrect and/or missing]    
    Alexandria.exe!fts5MultiIterNext(Fts5Index * p, Fts5Iter * pIter, int 
bFrom, __int64 iFrom) Line 186755    C
 Alexandria.exe!fts5MultiIterNextFrom(Fts5Index * p, Fts5Iter * pIter, 
__int64 iMatch) Line 187418    C
 Alexandria.exe!sqlite3Fts5IterNextFrom(Fts5IndexIter * pIndexIter, __int64 
iMatch) Line 189325    C
 Alexandria.exe!fts5ExprNodeNext_TERM(Fts5Expr * pExpr, Fts5ExprNode * 
pNode, int bFromValid, __int64 iFrom) Line 181773    C
 Alexandria.exe!fts5ExprNodeTest_AND(Fts5Expr * pExpr, Fts5ExprNode * pAnd) 
Line 181852    C
 Alexandria.exe!fts5ExprNodeNext_AND(Fts5Expr * pExpr, Fts5ExprNode * 
pNode, int bFromValid, __int64 iFrom) Line 181891    C
 Alexandria.exe!fts5ExprNodeNext_OR(Fts5Expr * pExpr, Fts5ExprNode * pNode, 
int bFromValid, __int64 iFrom) Line 181821    C
 Alexandria.exe!sqlite3Fts5ExprNext(Fts5Expr * p, __int64 iLast) Line 
182079    C
 Alexandria.exe!fts5NextMethod(sqlite3_vtab_cursor * pCursor) Line 191208   
 C
 Alexandria.exe!sqlite3VdbeExec(Vdbe * p) Line 83260    C
 Alexandria.exe!sqlite3Step(Vdbe * p) Line 75131    C
 Alexandria.exe!sqlite3_step(sqlite3_stmt * pStmt) Line 75193    C
 Alexandria.exe!fts5SorterNext(Fts5Cursor * pCsr) Line 191102    C
 Alexandria.exe!fts5CursorFirstSorted(Fts5Table * pTab, Fts5Cursor * pCsr, 
int bDesc) Line 191303    C
 Alexandria.exe!fts5FilterMethod(sqlite3_vtab_cursor * pCursor, int idxNum, 
const char * zUnused, int nVal, Mem * * apVal) Line 191588    C
 Alexandria.exe!sqlite3VdbeExec(Vdbe * p) Line 83174    C
 Alexandria.exe!sqlite3Step(Vdbe * p) Line 75131    C
 Alexandria.exe!sqlite3_step(sqlite3_stmt * pStmt) Line 75193    C
 Alexandria.exe!nsFTS5SearchService::`anonymous 
namespace'::DB_Worker::operator()(const 
std::basic_string,std::allocator > & csQuery, 
unsigned int nMaxToFind, 
std::vector
 > & vecResp) Line 76    C++

With this query:
SELECT T2TFTSrsn FROM T2TitleFTS WHERE  T2TitleFTS MATCH '{T2TFTStitle 
T2TFTSseries}:a wrinkle in time OR wrinkle in time' ORDER BY rank LIMIT 50;
=



Julian Dohmen | developer
Evelyn Manufacturing | Alexandria & KeepnTrack 
801.943.7277 ext 565 | jdoh...@companioncorp.com 

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users