Jan,
On Mon, May 07, 2007 at 09:19:17AM +0200, Jan Flokstra wrote:
> On Monday 07 May 2007 09:09, Stefan Manegold wrote:
> > On Mon, May 07, 2007 at 07:01:43AM +0000, Jan Flokstra wrote:
> > > Update of /cvsroot/monetdb/pathfinder/modules/pftijah
> > > In directory sc8-pr-cvs16:/tmp/cvs-serv20990/modules/pftijah
> > >
> > > Modified Files:
> > > nexi.c nexi_generate_mil.c pftijah.mx
> > > Log Message:
> > > * Implement private term- and tag dictionaries. Default is still the
> > > global version so nothing changed yet in the format. I'm having terrible
> > > trouble with the Mtest's again. This version could not be tested by our
> > > Mtest tests because the first query executes OK but all queries after
> > > that seem to 'forget' the instance variables of module pftijah. I will
> > > look if the same happens on other machine and post a bugreport for it.
> >
> > does the same happen if you run the test(s) by hand i.e., without Mtest?
>
> No, without Mtest the queries run OK. eg, any FIRST query runs OK. Any query
> after this fails.
do they run OK or NOT ???
If also without Mtest only the first runs OK and any other fails (that what
I read here), it is NOT an Mtest problem, but a general problem, right???
> > How do the queries and the test script(s) look like?
>
> The normal 28 tests from $PF/modules/pftijah/Tests So a mix of mil scripts
> and .xq XQueries.
>
> > How do you (successfully?) run them by hand?
>
> Just run the scripts in a test directory using MapiClient (-lx)
> > How do you (unsuccesfully) run them via Mtest?
>
> Mtest.py -d0 --TSTSRCBASE /local/flokstra/software/pathfinder
>
> in directory /local/flokstra/software/pathfinder/modules/pftijah
Before this mornings changes, everything did work fine, right?
Hence, your latest changes seem to *trigger* (not necessarily cause) the
problem. Looking in detail in your changes might give a hint, which part in
fact triggers the problem, which in turn would help a lot to analyse, locate
and eventually fix the problem.
Speaking of debugging here's a little excercise thet shows that the second
call to tj_init_collection() on the same database does fail, i.e., only the
first call to tj_init_collection() on a virgin database does not fail:
========
.../pathfinder/modules/pftijah/Tests $ export TSTSRCDIR=$PWD
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder);
tj_init_collection("testcoll0",
new(str,str).insert("stemmer","snowball-english"));' | Mserver --dbname=DB-1
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB># XRPC administrative console at http://localhost:50001/admin
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder);
tj_init_collection("testcoll0",
new(str,str).insert("stemmer","snowball-english"));' | Mserver --dbname=DB-1
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB># XRPC administrative console at http://localhost:50001/admin
!ERROR: interpret: unknown variable 'verbose'.
========
so, let's "trace" the variables:
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder); vars();' |
Mserver --dbname=DB-2 > /tmp/vars.0
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder); vars();' |
Mserver --dbname=DB-2 > /tmp/vars.1
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder);
tj_init_collection("testcoll0",
new(str,str).insert("stemmer","snowball-english"));' | Mserver --dbname=DB-2
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB># XRPC administrative console at http://localhost:50001/admin
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder); vars();' |
Mserver --dbname=DB-2 > /tmp/vars.2
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder);
tj_init_collection("testcoll0",
new(str,str).insert("stemmer","snowball-english"));' | Mserver --dbname=DB-2
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB># XRPC administrative console at http://localhost:50001/admin
!ERROR: interpret: unknown variable 'verbose'.
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder); vars();' |
Mserver --dbname=DB-2 > /tmp/vars.3
========
.../pathfinder/modules/pftijah/Tests $ grep --color verbose /tmp/vars.?
--------
/tmp/vars.0:[ "verbose", "bit", "liquid",
"changeable", "false" ]
/tmp/vars.1:[ "verbose", "bit", "liquid",
"changeable", "false" ]
========
I have no idea, what tj_init_collection() does but it somehow triggers the
disappeasing of some variables --- see also:
========
.../pathfinder/modules/pftijah/Tests $ diff -dU0 -Bbw /tmp/vars.[12] | cdiff
--------
--- /tmp/vars.1 2007-05-07 10:40:40.000000000 +0200
+++ /tmp/vars.2 2007-05-07 10:41:03.000000000 +0200
@@ -6 +6 @@
-#-------------------------------------------------------------------------------------------------------------------------#
+#---------------------------------------------------------------------------------------------------------#
@@ -9,2 +9,2 @@
-#-------------------------------------------------------------------------------------------------------------------------#
-[ "pf_logger", "logger", "liquid", "changeable",
"19792504" ]
+#---------------------------------------------------------------------------------------------------------#
+[ "pf_logger", "logger", "liquid", "changeable",
"19781832" ]
@@ -41 +41 @@
-[ "pf_chkpt_barrier", "sema", "liquid", "changeable",
"14646584" ]
+[ "pf_chkpt_barrier", "sema", "liquid", "changeable",
"15961944" ]
@@ -43 +43 @@
-[ "pf_chkpt", "lock", "liquid", "changeable",
"19243832" ]
+[ "pf_chkpt", "lock", "liquid", "changeable",
"19100760" ]
@@ -47 +47 @@
-[ "pf_collbarrier_lock", "lock", "liquid", "changeable",
"19242456" ]
+[ "pf_collbarrier_lock", "lock", "liquid", "changeable",
"19099384" ]
@@ -63 +63 @@
-[ "ws_log_lock", "lock", "liquid", "changeable",
"19094712" ]
+[ "ws_log_lock", "lock", "liquid", "changeable",
"19090936" ]
@@ -219,12 +218,0 @@
-[ "virtRoot", "oid", "frozen", "constant",
"[EMAIL PROTECTED]" ]
-[ "tracefile_handle", "void", "liquid", "changeable",
"nil" ]
-[ "equivalences", "BAT", "liquid", "changeable",
"<tmp_162>" ]
-[ "tj_coll_mutex", "lock", "liquid", "changeable",
"18815256" ]
-[ "GLOBAL_TTBAT", "bit", "frozen", "constant",
"true" ]
-[ "dflt_score_base", "str", "frozen", "constant",
"\"0\"" ]
-[ "dflt_bg_index", "str", "frozen", "constant",
"\"DFLT_FT_INDEX\"" ]
-[ "dflt_ft_index", "str", "frozen", "constant",
"\"DFLT_FT_INDEX\"" ]
-[ "verbose", "bit", "liquid", "changeable",
"false" ]
-[ "QENV_SCOREBASE", "oid", "frozen", "constant",
"[EMAIL PROTECTED]" ]
-[ "QENV_FTIBGNAME", "oid", "frozen", "constant",
"[EMAIL PROTECTED]" ]
-[ "QENV_FTINAME", "oid", "frozen", "constant",
"[EMAIL PROTECTED]" ]
@@ -247,3 +235,3 @@
-[ "tj_coll_lockbat", "BAT", "liquid", "changeable",
"<tmp_161>" ]
-[ "tj_dep_lock", "lock", "liquid", "changeable",
"18788840" ]
-[ "tj_adm_lock", "lock", "liquid", "changeable",
"18788504" ]
+[ "tj_coll_lockbat", "BAT", "liquid", "changeable",
"<tmp_277>" ]
+[ "tj_dep_lock", "lock", "liquid", "changeable",
"18793288" ]
+[ "tj_adm_lock", "lock", "liquid", "changeable",
"18792952" ]
========
Alll these seem to be pftijah variables; hence, it might also be that
loading the pftijah module the second time does fail --- unnoticed because
the pathfinder's `module("pftijah")` is wrapped in a CATCH()
(which is admittedly unhandy for debugging things like these...):
========
.../pathfinder/modules/pftijah/Tests $ grep 'module("pftijah")'
../../../runtime/pathfinder.mx
--------
CATCH(module("pftijah")); # load pftijah only if available
========
Let's try:
========
.../pathfinder/modules/pftijah/Tests $ grep 'module("pftijah")'
$PATHFINDER_PREFIX/lib64/MonetDB4/pathfinder.mil
--------
CATCH(module("pftijah")); # load pftijah only if available
========
.../pathfinder/modules/pftijah/Tests $ perl -i -p -e
's|CATCH\((module\("pftijah"\))\)|$1|'
$PATHFINDER_PREFIX/lib64/MonetDB4/pathfinder.mil
========
.../pathfinder/modules/pftijah/Tests $ grep 'module("pftijah")'
$PATHFINDER_PREFIX/lib64/MonetDB4/pathfinder.mil
--------
module("pftijah"); # load pftijah only if available
========
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder);
tj_init_collection("testcoll0",
new(str,str).insert("stemmer","snowball-english"));' | Mserver --dbname=DB-3
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB># XRPC administrative console at http://localhost:50001/admin
========
.../pathfinder/modules/pftijah/Tests $ echo 'module(pathfinder);
tj_init_collection("testcoll0",
new(str,str).insert("stemmer","snowball-english"));' | Mserver --dbname=DB-3
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB>!ERROR: interpret: no matching MIL operator to 'tj_setCollName(str)'.
========
AHA!!!
========
.../pathfinder/modules/pftijah/Tests $ Mserver --dbname=DB-4
--------
# Monet Database Server V4.17.1
# Copyright (c) 1993-2007, CWI. All rights reserved.
# Compiled for x86_64-redhat-linux-gnu/64bit with 64bit OIDs; dynamically
linked.
# Visit http://monetdb.cwi.nl/ for further information.
MonetDB>module(pathfinder);
# XRPC administrative console at http://localhost:50001/admin
MonetDB>help("tj_setCollName");
no help available
MonetDB>tj_setCollName();
!ERROR: interpret: no matching MIL operator to 'tj_setCollName()'.
========
.../pathfinder $ find . -type f | xargs grep --color tj_setCollName
--------
./runtime/Tests/procs.stable.out:[ "tj_setCollName(str) : void"
]
./modules/pftijah/Tests/test_nllr.milS:tj_setCollName("testcoll1");
./modules/pftijah/Tests/test_nllr.stable.out:tj_setCollName("testcoll1");
./modules/pftijah/pftijah.mx: tj_setCollName(name);
./modules/pftijah/nexi.c: if ( 0 ) { MILPRINTF(MILOUT,
"tj_setCollName(\"%s\");\n", parserCtx->collection); /* DELETE */ /* DELETE */ }
========
Apparently, you are still using a PROC ("tj_setCollName") that you removed in
your latest checkin --- running test runtime/Tests/procs.milS (would have)
show(s|ed) you that the respective signature has gone.
Hence, it's neither "Mtest" nor any other "mysterious bug", but quite a
simple problem.
Stefan
ps: ... yes, also this debugging service was free of charge ...
> >
> > Stefan
>
> The text of the bug report I had in mind was:
> ====================================================
> I am having Mtest module variable problems again with pftijah. It looks a
> little bit like the never really solved problem of bug:
>
> [ 1638729 ] Mtest/Mapi module load problem
>
> The problem here was that the module was loaded but the instance variables
> and the non-ADDHELP'ed functions were unknown. We solved this by loading
> pftijah always from pathfinder but did not find the cause of the problem.
>
> I'm now having the same problem but in a different form. When I use Mtest
> on the pftijah Testset the first test where module(pftijah) is used works
> fine. But all tests after this fail. The problem is again that the MIL
> functions in the module are executed but the functions do not know the
> instance variables of the module. Nearly all tests fail on the variables
> "verbose" and "dflt_ft_index".
> You can for instance check this by changing $PF/modules/pftijah/Tests/All
> The first test which normally fails is createcoll0 because it is the
> second test. But when you remove loadthesis from All createcoll0 succeeds
> and createcoll1, which is then the second, fails.
>
> ====================================================
>
> Jan.
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
>
>
--
| Dr. Stefan Manegold | mailto:[EMAIL PROTECTED] |
| CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ |
| 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 |
| The Netherlands | Fax : +31 (20) 592-4312 |
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Monetdb-developers mailing list
Monetdb-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
