Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
Where did the file fots.xq go? Is it replaced by cli.xq? If so, the README.txt
and the help
in cli.xq should be adapted.
I would be nice to get better error messages if one of the command line
parameters is missing.
For example,
./bin/zorba -f -q ../sandbox/test/fots_driver/cli.xq -e
fotsPath:=/home/mbrantner/w3c/checkout/2011/QT3-test-suite/ -e
mode:=run-test-sets -e testSetPrefixes:=prod-WindowClause
/home/mbrantner/zorba/build/URI_PATH/org/expath/ns/file.xq:359,3: type error
[err:XPTY0004]: empty path; raised at
/home/mbrantner/zorba/sandbox/src/util/fs_util.cpp:196
=
file:read-text#1 http://expath.org/ns/file
/home/mbrantner/zorba/sandbox/test/fots_driver/zorba-fots-driver.xqy at line
175 column 40
=
driver:run-fots#9 http://www.zorba-xquery.com/zorba-fots-driver
/home/mbrantner/zorba/build/../sandbox/test/fots_driver/cli.xq at line 127
column 10
Also, I think it became to hard to use. It would be nice if at least the path
to the manifest
file doesn't need to be given on the command line. Maybe a CMake-configured
bash/bat
script in the build directory would help to make it easier to run.
- Rename fots_driver to fots
- Rename zorba-fots-driver.xqy to fots-driver.xq
People must get used to the terminology:
- FOTS means the testsuite from W3C: hence when a test is wrong = file a bug
to W3C
- FOTS driver is the thing that runs the testsuite: hence when an error is
detected in the FOTS driver = file a bug in Launchpad with a fots-driver
TAG
- test case is a single test
- test set is unit (a bunch) of tests grouped together because they test a
certain feature/function/functionality
The renaming you suggest would confuse people.
At least the zorba in zorba-fots-driver is redundant.
- We generally use xq instead of xqy for xquery files. Maybe we should do
this
here to.
If you really insist then I can rename the files, sure...
It's not a strong reject but I love consistency in the repository.
- Does the dependency output have to appear in every run?
Do not know what you mean by dependency output: can you please be more
specific?
The output file is very verbose. At the beginning, it contains a huge list
starting with the dependencies-satisfied
file. Does it need to be there.
The output of a failing test is very long. You can't immediately see why
something fails.
For example,
test-case name=WindowingUseCase04 result=fail
environment ref=WindowingUseCases04/
dependency type=spec value=XQ30+/
test
let $MAX_DIFF := 2
for sliding window $w in ./stream/event
start $s_curr at $s_pos previous $s_prev
when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
only end next $e_next
when $e_next/@time - $s_curr/@time gt $MAX_DIFF
return
avg( $w/@temp )
/test
result
assert-xml8 9 17 18/assert-xml
/result
info
environment name=WindowingUseCases04
source role=. file=WindowClause/temp_events.xml
description temp_events.xml from the XQuery 3.0 Use Cases
/description
created by=Matthias Brantner on=2011-07-28/
/source
/environment
queryimport module namespace xqxq =
'http://www.zorba-xquery.com/modules/xqxq';
variable $queryID := xqxq:prepare-main-module('
xquery version ''3.0'';
let $MAX_DIFF := 2
for sliding window $w in ./stream/event
start $s_curr at $s_pos previous $s_prev
when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
only end next $e_next
when $e_next/@time - $s_curr/@time gt $MAX_DIFF
return
avg( $w/@temp )
'
);
variable $contextItem :=
doc(/home/mbrantner/w3c/checkout/2011/QT3-test-suite//prod/WindowClause/temp_events.xml);
xqxq:bind-context-item($queryID, $contextItem);
xqxq:evaluate($queryID)/query
out xmlns=
expected-result
assert-xml8 9 17 18/assert-xml
/expected-result
result8 9 17 18/result
errorsResult
'891718'
is different from the expected result
'8 9 17 18'./errors
/out
/info
/test-case
This output contains a lot of stuff that might be helpful in some cases but it
would be really nice
to just have a summary of the failures and the reason why something failed.
Also, this test seems to give the correct result but the comparison fails.
Matthias
--
https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980
Your team Zorba Coders is subscribed to branch lp:zorba.
--
Mailing list: https://launchpad.net/~zorba-coders
Post to : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
First of all let me mention that this type of communication in Launchpad
messages is IMHO very hard to follow.
Anyway: this is how you chose to report your issues so here are my remarks
below.
Where did the file fots.xq go? Is it replaced by cli.xq? If so, the README.txt
and the help
in cli.xq should be adapted.
They were updated: are you using the latest version of the driver?
I would be nice to get better error messages if one of the command line
parameters is missing.
For example,
./bin/zorba -f -q ../sandbox/test/fots_driver/cli.xq -e
fotsPath:=/home/mbrantner/w3c/checkout/2011/QT3-test-suite/ -e mode:=run-
test-sets -e testSetPrefixes:=prod-WindowClause
/home/mbrantner/zorba/build/URI_PATH/org/expath/ns/file.xq:359,3: type error
[err:XPTY0004]: empty path; raised at
/home/mbrantner/zorba/sandbox/src/util/fs_util.cpp:196
=
file:read-text#1 http://expath.org/ns/file
/home/mbrantner/zorba/sandbox/test/fots_driver/zorba-fots-driver.xqy at line
175 column 40
=
driver:run-fots#9 http://www.zorba-xquery.com/zorba-fots-driver
/home/mbrantner/zorba/build/../sandbox/test/fots_driver/cli.xq at line 127
column 10
Please open a Launchpad issue with the fots-driver TAG and assign it to me.
Also, I think it became to hard to use. It would be nice if at least the path
to the manifest
file doesn't need to be given on the command line. Maybe a CMake-configured
bash/bat
script in the build directory would help to make it easier to run.
My experience with what you are proposing is this:
- it makes it easier to use, it is true
- everyone will forget what params were set in the file or that a file even
exists. This makes it very hard to debug the reported issues because you also
have to send the file with the configured params.
Also one needs to set different options to see what happens: for instance what
happens when you enable the HOF feature? You can easily create a new
ZorbaManifest_HOF_ENABLED.xml and invoke teh CLI with that file.
You see the difference in a second, you to not need to change any configuration
file.
I would vote against your proposal.
- Rename fots_driver to fots
- Rename zorba-fots-driver.xqy to fots-driver.xq
People must get used to the terminology:
- FOTS means the testsuite from W3C: hence when a test is wrong = file a
bug to W3C
- FOTS driver is the thing that runs the testsuite: hence when an error is
detected in the FOTS driver = file a bug in Launchpad with a fots-driver
TAG
- test case is a single test
- test set is unit (a bunch) of tests grouped together because they test a
certain feature/function/functionality
The renaming you suggest would confuse people.
At least the zorba in zorba-fots-driver is redundant.
No it is not: if we decide to put it in FOTS/drivers together with the Saxon,
BaseX it will make sense right?
- We generally use xq instead of xqy for xquery files. Maybe we should do
this
here to.
If you really insist then I can rename the files, sure...
It's not a strong reject but I love consistency in the repository.
Please open a Launchpad issue with the fots-driver TAG and assign it to me.
- Does the dependency output have to appear in every run?
Do not know what you mean by dependency output: can you please be more
specific?
The output file is very verbose. At the beginning, it contains a huge list
starting with the dependencies-satisfied
file. Does it need to be there.
The output of a failing test is very long. You can't immediately see why
something fails.
For example,
test-case name=WindowingUseCase04 result=fail
environment ref=WindowingUseCases04/
dependency type=spec value=XQ30+/
test
let $MAX_DIFF := 2
for sliding window $w in ./stream/event
start $s_curr at $s_pos previous $s_prev
when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
only end next $e_next
when $e_next/@time - $s_curr/@time gt $MAX_DIFF
return
avg( $w/@temp )
/test
result
assert-xml8 9 17 18/assert-xml
/result
info
environment name=WindowingUseCases04
source role=. file=WindowClause/temp_events.xml
description temp_events.xml from the XQuery 3.0 Use Cases
/description
created by=Matthias Brantner on=2011-07-28/
/source
/environment
queryimport module namespace xqxq = 'http://www.zorba-
xquery.com/modules/xqxq';
variable $queryID := xqxq:prepare-main-module('
xquery version ''3.0'';
let $MAX_DIFF := 2
for sliding window $w in ./stream/event
start $s_curr at $s_pos previous $s_prev
when ($s_curr/@time ne $s_prev/@time) or (empty($s_prev))
only end next $e_next
when $e_next/@time - $s_curr/@time gt $MAX_DIFF
return
avg( $w/@temp )
'
);
variable $contextItem :=
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
@Matthias: Let me ask you if I may: which of the issues you reported are a show stopper for merging the FOTS driver into Zorba? Can you please list them with the appropriate reported Launchpad issues? This will help me a lot in addressing them asap. Thanks for your feedback. -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
The implementation looks very clean. I poked around a bit and ran several random test sets. The feeling I got is that it's not quite ready yet for merging. I would like to avoid waisting time when more people start using it and all run into the same issues. Before we continue, I would like to understand why the following two problems exist. Especially the first one seems to be a candidate to fix many more tests. - assert-serialization doesn't not seem to be implemented. Should it? I poked around and found many tests failing because of this (e.g. CondExpr002) It is implemented, but the name is 'assert-xml' please check the latest version of catalog-schema.xsd: http://dev.w3.org/cvsweb/2011/QT3-test-suite/catalog-schema.xsd?rev=1.25;content-type=text%2Fx-cvsweb-markup look for 'assert-xml' - It seems that all of the prod-OrderByClause tests fail. It's not clear whether this is a bug in my setting, the driver, or Zorba itself. Already reported as lp:956460. Some minor issues are: - I find it very hard to read the output of a test run but maybe this will change if we have the stylesheet - How do I know whether my test run actually ran any tests? Sometimes I had a typo in the prefix and it didn't run anything but it's hard to see. Note: the FOTS driver should be used together with the -o output.xml and --indent of the Zorba CLI. The user will then get trace output in the text window and the output of the tests in the output.xml file. Whenever a test was run, there is a trace output for it's name on the command line. Whenever a test that was run failed, there is a trace message for that pointing also the name of the test case it belongs to for easy identification. I have updated the README.txt to reflect these issues, see http://bazaar.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/revision/11136 - Rename fots_driver to fots - Rename zorba-fots-driver.xqy to fots-driver.xq People must get used to the terminology: - FOTS means the testsuite from W3C: hence when a test is wrong = file a bug to W3C - FOTS driver is the thing that runs the testsuite: hence when an error is detected in the FOTS driver = file a bug in Launchpad with a fots-driver TAG - test case is a single test - test set is unit (a bunch) of tests grouped together because they test a certain feature/function/functionality The renaming you suggest would confuse people. - We generally use xq instead of xqy for xquery files. Maybe we should do this here to. If you really insist then I can rename the files, sure... - Does the dependency output have to appear in every run? Do not know what you mean by dependency output: can you please be more specific? -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
I agree with Sorin that the file naming is OK: I've had to teach myself to say FOTS driver when that's what I mean, rather than FOTS itself. The file naming as it is now is clear and consistent. I have no real opinion about .xqy - .xq. The test case CondExpr002 is passing currently (Matthias, make sure you cvs update your FOTS download). IMHO this code is ready to go. There needs to be another layer on top of it to integrate with ctest/remote queue, which would post-process the output for day-to-day use. But that's a separate problem that will be more easily solved once this code is on the trunk. -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
Review: Needs Fixing The implementation looks very clean. I poked around a bit and ran several random test sets. The feeling I got is that it's not quite ready yet for merging. I would like to avoid waisting time when more people start using it and all run into the same issues. Before we continue, I would like to understand why the following two problems exist. Especially the first one seems to be a candidate to fix many more tests. - assert-serialization doesn't not seem to be implemented. Should it? I poked around and found many tests failing because of this (e.g. CondExpr002) - It seems that all of the prod-OrderByClause tests fail. It's not clear whether this is a bug in my setting, the driver, or Zorba itself. Some minor issues are: - I find it very hard to read the output of a test run but maybe this will change if we have the stylesheet - Rename fots_driver to fots - We generally use xq instead of xqy for xquery files. Maybe we should do this here to. - Rename zorba-fots-driver.xqy to fots-driver.xq - How do I know whether my test run actually ran any tests? Sometimes I had a typo in the prefix and it didn't run anything but it's hard to see. - Does the dependency output have to appear in every run? -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/zorba_FOTS_driver into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/zorba_FOTS_driver/+merge/132980 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
