Bugs item #2545711, was opened at 2009-01-29 16:25
Message generated for change (Comment added) made by jflokstra
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2545711&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: PFtijah
Group: Pathfinder CVS Head
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Loredana Afanasiev (lafanasi)
Assigned to: Jan Flokstra (jflokstra)
Summary: buggy ft-index-info
Initial Comment:
Hi,
problem with ft-index-info(). See the action sequence below.
mcl = mclient -lx
[lafan...@zookst4 xq]$ more tijah-index-extend.xq
let $opt := <TijahOptions ft-index="polietiekedata" stemmer="snowball-dutch"/>
let $col := ("KVR-DE")
return
tijah:extend-ft-index($col, $opt)
[lafan...@zookst4 xq]$ mcl tijah-index-extend.xq
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR,KVR-DE" tokenizer="flex"
stemmer="snowball-dutch">polietiekedata</ftindex>,
<ftindex collections="*" tokenizer="flex"
stemmer="nostemming">DFLT_FT_INDEX</ftindex>
[lafan...@zookst4 xq]$ stopMserver
[lafan...@zookst4 xq]$ startMserver
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR" tokenizer="flex"
stemmer="snowball-dutch">polietiekedata</ftindex>,
<ftindex collections="*" tokenizer="flex"
stemmer="nostemming">DFLT_FT_INDEX</ftindex>
cheers,
l.
OS: Linux, 2.6.24.3-50.fc8
Mem: 19GB
Compiler: gcc 4.1.2
Parser tools: ?
autoconf (GNU Autoconf) 2.61
automake (GNU automake) 1.10
ltmain.sh (GNU libtool) 1.5.24
----------------------------------------------------------------------
>Comment By: Jan Flokstra (jflokstra)
Date: 2009-02-24 09:51
Message:
Loredana,
As Stefan mentioned the fix was indeed not in the Nov2008 Stable. I just
checked and it is in the current HEAD so this is no problem. Your problem
was reproduced on several architectures and the fix fixed the original
problem. Your 'new' problems have other causes.
- PART I: the problem here is that it looks as if the Nov2008 doesn not
support pf:string-value(elem) which causes the line "return <pair>{(
string($res), tijah:score($qid, $res))}</pair>" to fail. I looked in the
current HEAD and there is an algebra version defined for this function
- PART II: this is an older problem and has our attention. I talked about
this with Henning also in connection with earlier problems you had. When
there have been a lot of problems/crahses there is a chance of a half baked
index. The solution for us is to ignore all errors when deleting the index
bats and modifying the pftijah adminstration bats. This will be implemented
soon. Current best practice is to throw away the monet files
($PREFIX/var/MonetDB4/dbfarm/<DATABASENAME>)
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-02-24 02:09
Message:
Just for the records:
The Nov2008-SP2 version (second and last "service pack" of "bug fix
release" of the ("old") Nov2008 branch) was released on 2009/01/23.
Henning checked in Jan's bug fix on 2009/01/30 to the ("new") Feb2009
branch (assuming cvs log message "checked in a bug-fix for Jan F." refers
to the fix Jan mentioned below ...).
Sjoerd propagated Henning's checkin (i.e., Jan's fix) on the same day
(2009/01/30) from the Feb2009 branch to the CVS HEAD.
In other words:
- Nov2008-SP2 does not contain Jan's fix.
- Feb2009 branch and CVS HEAD contain Jan's fix since 2009/01/30.
- In case Loredana's HEAD is more recent than 2009/01/30, Jan's fix might
not be sufficient, yet.
----------------------------------------------------------------------
Comment By: Loredana Afanasiev (lafanasi)
Date: 2009-02-24 01:03
Message:
Hi JanF,
I've tested this problem again on the HEAD and on the stable PS2 versions.
Both versions behave erroneously along the lines I've described before. The
stable version gives more errors. Below (Part I), I've copied parts of my
terminal scripting for the stable version. Further, I've tried to delete
and create the ft-index again and it didn't work (Part II).
Part I: working with the stable Nov2008-SP2 version
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR,HAN-B,KVR-B"
tokenizer="flex" stemmer="snowball-dutch">polietiekedata</ftindex>
[lafan...@zookst4 xq]$ more tijah-index-extend.xq
let $opt := <TijahOptions ft-index="polietiekedata"
stemmer="snowball-dutch"/>
let $col := ("KVR-DE")
return
tijah:extend-ft-index($col, $opt)
[lafan...@zookst4 xq]$ mcl tijah-index-extend.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"
stemmer="snowball-dutch"/>
ERROR = !ERROR: interpret_params: insert(param 3): invalid BAT.
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR,HAN-B,KVR-B,KVR-DE"
tokenizer="flex" stemmer="snowball-dutch">polietiekedata</ftindex>
[lafan...@zookst4 xq]$ mcl q2.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"
ir-model="NLLR"/>
ERROR = !fatal error: Algebra implementation for function
`#pf:string-value' is missing.
[lafan...@zookst4 xq]$ more q2.xq
let $opt := <TijahOptions ft-index="polietiekedata" ir-model="NLLR"/>
let $c := collection("KVR-DE")
let $qid := tijah:query-id($c, "//ANTWOORD[about(.,Polizei)]", $opt)
for $res in tijah:nodes($qid)
return <pair>{( string($res), tijah:score($qid, $res))}</pair>
[lafan...@zookst4 xq]$ sudo su ilps_bg
ilps...@zookst4:/scratch/PolitiekeData/KVR-DE/xq$ stopMserver
ilps...@zookst4:/scratch/PolitiekeData/KVR-DE/xq$ startMserver
exit
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR,HAN-B,KVR-B"
tokenizer="flex" stemmer="snowball-dutch">polietiekedata</ftindex>
Part II: the index is corrupt
[lafan...@zookst4 xq]$ mcl delete-tijah-index.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"/>
ERROR = !ERROR: interpret_params: persists(param 1): invalid BAT.
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
[lafan...@zookst4 xq]$ mcl delete-tijah-index.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"/>
ERROR = !ERROR: tj_get_ft_index, ft-index "polietiekedata" does not exist
[lafan...@zookst4 xq]$ mcl tijah-index.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"
stemmer="snowball-dutch"/>
ERROR = !ERROR: interpret_params: access(param 1): invalid BAT.
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR,HAN-B,KVR-B"
tokenizer="flex" stemmer="snowball-dutch">polietiekedata</ftindex>
[lafan...@zookst4 xq]$ mcl delete-tijah-index.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"/>
ERROR = !ERROR: interpret_params: persists(param 1): invalid BAT.
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
[lafan...@zookst4 xq]$ mcl tijah-index.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions ft-index="polietiekedata"
stemmer="snowball-dutch"/>
ERROR = !ERROR: interpret_params: insert(param 3): invalid BAT.
[lafan...@zookst4 xq]$ mcl -s "tijah:ft-index-info()"
<ftindex collections="MotiesTweedeKamer,HAN,KVR,HAN-B,KVR-B"
tokenizer="flex" stemmer="snowball-dutch">polietiekedata</ftindex>
[lafan...@zookst4 xq]$ more q3.xq
let $opt := <TijahOptions ft-index="polietiekedata" ir-model="NLLR"
returnNumber="5" />
let $coll := collection("HAN")
(:let $query := tijah:tokenize("knettergek") :)
let $query := "knettergek"
(: We need the "else" because empty queries are not allowed :)
let $nexi :=
if ($query)
then concat("//spreker[about(.,",$query,")]")
else "//spreker[about(.,bogusqueryblablabl)]"
let $qid := tijah:query-id($coll, $nexi, $opt)
let $nodes := tijah:nodes($qid)
for $res in $nodes
return
<pair>{(string($res/@naam), tijah:score($qid, $res))}</pair>
[lafan...@zookst4 xq]$ mcl q3.xq
MAPI = mone...@localhost:50100
QUERY = let $opt := <TijahOptions
milfile="/scratch/PolitiekeData/HAN/xq/q3.mil" ft-index="polietiekedata"
ir-model="NLLR"
ERROR = !ERROR: CMDtijah_query: tijahParse: parse error.
!ERROR: CMDtijah_query: operation failed.
I hope this helps to fix the problem(s).
cheers,
l.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2009-02-15 21:26
Message:
Loredana, JanF,
could you please
(1) check whether the problem is indeed fixed
(2) provide/add a test
(3) close the bug report once the fix is confirmed and a test has been
added.
Thanks!
Stefan
----------------------------------------------------------------------
Comment By: Jan Flokstra (jflokstra)
Date: 2009-01-30 10:01
Message:
It looks as if the ft-index vs. pf-collection dependency administration was
not properly committed in the case where an extend specifies collections
which are not defined yet. It should be fixed in the Stable branch now. It
will take some time before it appears in the HEAD which I think Loredana is
using. When Loredane confirms it also fixes her problem I will close the
case.
JanF.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2545711&group_id=56967
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
