Bugs item #2893875, was opened at 2009-11-07 17:19 Message generated for change (Comment added) made by stmane You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2893875&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: PF/tests Group: Pathfinder "stable" >Status: Closed >Resolution: Fixed Priority: 9 Private: No Submitted By: Stefan Manegold (stmane) Assigned to: Stefan Manegold (stmane) Summary: PF: 82 tests fail on Auf2009_NFI/Nov2009 but work on Aug2009 Initial Comment: The following 82 tests work(ed) fine with the Aug2009 branch, but appear to fail with the Aug2009 branch, and hence most probaly also with the Nov2009 branch (which is currently even more instable wrt. testing for yet undiscovered reasons): benchmarks/MBench/qa05 benchmarks/MBench/ benchmarks/XBench/DC/MD/q19 benchmarks/XBench/DC/MD/ modules/pftijah/procs modules/pftijah/colltest1 modules/pftijah/colltest2 modules/pftijah/ runtime/xrpc/admin/add_del_doc_noxrpc runtime/xrpc/admin/backup_restore_noxrpc runtime/xrpc/admin/ tests/BugTracker/treat_as.SF-1586454 tests/BugTracker/insert_multiple.SF-1590580 tests/BugTracker/inserting_multiple_elements.SF-1590583 tests/BugTracker/insert_into_transient.SF-1626208 tests/BugTracker/clear_attrs_on_delete.SF-1612739 tests/BugTracker/clear_attrs_on_delete.SF-1612739-b tests/BugTracker/clear_attrs_on_delete.SF-1612739-c tests/BugTracker/corrupt_after_update.SF-1706640 tests/BugTracker/accessing_renamed_inserted_deleted_node.SF-1718622-1718635-1718709 tests/BugTracker/insert_large_doc.SF-1726954 tests/BugTracker/replace-corrupts.SF-1758902 tests/BugTracker/swizzle-bug.SF-1760811 tests/BugTracker/swizzle-bug2.SF-1763495 tests/BugTracker/insert_attribute_gives_ERROR_in_merged_union.SF-1763575 tests/BugTracker/immune_for_updates.SF-1766259 tests/BugTracker/repeated-insert.SF-1814911 tests/BugTracker/insert-new-page.SF-1854215 tests/BugTracker/immune_for_updates.SF-1981852 tests/BugTracker/hang_non_existing_doc.SF-1911209 tests/BugTracker/function_parameter_without_type.SF-1898518 tests/BugTracker/copybug.SF-2642003 tests/BugTracker/update-with-timing.SF-2852928 tests/BugTracker/compilation_error.SF-2860574_1 tests/BugTracker/compilation_error.SF-2860574_2 tests/BugsViaSourgeforce/invisible_error_messages.SF-1409122 tests/BugsViaSourgeforce/ tests/StandOff/update/basic_insert tests/StandOff/update/basic_test tests/StandOff/update/ tests/Update/setattr-1 tests/Update/insert-1 tests/Update/insert-2 tests/Update/symm tests/Update/replacevaluetest tests/Update/replacevaluetest2 tests/Update/update tests/Update/illegal-insert tests/Update/illegal-delete tests/Update/insert_test_order tests/Update/insert_test_order_seq tests/Update/replacenode-corruption.SF-2153245 tests/W3C_use_cases/XQUF/AddressBook/Q1x tests/W3C_use_cases/XQUF/AddressBook/check_docs tests/W3C_use_cases/XQUF/AddressBook/ tests/W3C_use_cases/XQUF/Parts/Q1 tests/W3C_use_cases/XQUF/Parts/Q2 tests/W3C_use_cases/XQUF/Parts/Q3a1 tests/W3C_use_cases/XQUF/Parts/Q3a2 tests/W3C_use_cases/XQUF/Parts/Q4a tests/W3C_use_cases/XQUF/Parts/Q4ax tests/W3C_use_cases/XQUF/Parts/Q4b tests/W3C_use_cases/XQUF/Parts/Q4bx tests/W3C_use_cases/XQUF/Parts/Q4c tests/W3C_use_cases/XQUF/Parts/Q4cx tests/W3C_use_cases/XQUF/Parts/Q6 tests/W3C_use_cases/XQUF/Parts/Q6x tests/W3C_use_cases/XQUF/Parts/ tests/W3C_use_cases/XQUF/R/Q1 tests/W3C_use_cases/XQUF/R/Q2 tests/W3C_use_cases/XQUF/R/Q3 tests/W3C_use_cases/XQUF/R/Q4 tests/W3C_use_cases/XQUF/R/Q4x tests/W3C_use_cases/XQUF/R/Q5b tests/W3C_use_cases/XQUF/R/Q6a tests/W3C_use_cases/XQUF/R/Q6b tests/W3C_use_cases/XQUF/R/Q7 tests/W3C_use_cases/XQUF/R/Q7x tests/W3C_use_cases/XQUF/R/Q8 tests/W3C_use_cases/XQUF/R/Q9 tests/XQuery/is-before4 tests/XQuery/union ---------------------------------------------------------------------- >Comment By: Stefan Manegold (stmane) Date: 2009-11-18 10:43 Message: confirmed. closing. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-17 10:08 Message: forgot to finish: "I already" ... ... approved ALG-specific output of the insert-order test on Lefteris' advice and closed the bug report: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1964365&group_id=56967 ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-17 10:05 Message: Thanks, Peter! Your checkins were "too late" for last night's (this morning's) testing, but local testing shows pathfinder/benchmarks/MBench/Tests/qa05.* and pathfinder/tests/Update/Tests/update.* working fine, again. I already Can be closed once confirmed by nightly testing. (For now, system-specific problems are also documented in the TestWeb, and should be handled later; they're not considered crucial for the upcoming release.) ---------------------------------------------------------------------- Comment By: Peter Boncz (boncz) Date: 2009-11-17 00:32 Message: both problems are fixed now I did spot in tests/Update/insert_test_order_seq another instance of differences accross all platforms. The test is a little vauge, inserting a sequence of values; I am not sure what the XQUF semantics says about this. Is the order implementation-dependent? This this order might also be ok and the test could be approved. ---------------------------------------------------------------------- Comment By: Jan Rittinger (tsheyar) Date: 2009-11-16 18:42 Message: In MBench/qa05 Peter's heuristic rewrite leads to an (unnecessary) cross product. (For all eNest nodes it applies pf:attribute.) I guess that might be the problem. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2009-11-16 15:42 Message: This is a show stopper. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-16 15:28 Message: Peter, the two "crucial" problems remaining (which both started on the Nov2009 branch after propagations of NFI changes from the Aug2009_NFI branch to the Nov2009 branch) are timeout in http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/benchmarks_MBench/qa05.err.00.html (since 2009.10.09) missing updates in http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_Update/update.out.00.html (since 2009.10.08) They might actually be missing, because the query parts of the update queries involving an attribute predicate yield an empty result. The missing updates are for $elem in $testdoc/document/eleme...@attribute = 50] return do rename exactly-one($elem) into "Element" for $elem in $testdoc/document/eleme...@attribute = 10] return (do insert <a/> as first into exactly-one($elem) treat as element(), do insert <b/> as first into exactly-one($elem) treat as element()) for $elem in $testdoc/document/eleme...@attribute = 20] return (do insert <a/> as last into exactly-one($elem) treat as element(), do insert <b/> as last into exactly-one($elem) treat as element()) for $elem in $testdoc/document/eleme...@attribute = 30] return (do insert <a/> before exactly-one($elem), do insert <b/> before exactly-one($elem)) for $elem in $testdoc/document/eleme...@attribute = 40] return (do insert <a/> after exactly-one($elem), do insert <b/> after exactly-one($elem)) ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-15 13:57 Message: below the details what I did so far: > - tests/BugTracker/empty_file.SF-2017862: SunOS specific version only!! approved. > - tests/BugTracker/XML_document_cache_broken.SF-1414720 approved. > - tests/W3C_use_cases/XQ/NS/* (or remove tests!) disabled. > - tests/XQuery/{div.is-before4,union} div: approved MIL error messages iso. XQuery error message is-before4, union: nothing, as I'm not sure, whether the new output is correct. Since the NFI change were propagated to the Nov2009 branch on Oct 7, these tests show again the output as it was before =================================================================== 2009/05/20 - tsheyar: pathfinder/tests/XQuery/Tests/is-before4.stable.out,1.5.30.1 pathfinder/tests/XQuery/Tests/union.stable.out,1.6.12.1 -- Make the error message for missing documents more informative: Include the document name in the error message. -- Adjust new error messages in the testweb. -- Adjust stable output (for tests with a modified inter-document order). -- Fix/Extend a cross product rewrite (for unreferenced inputs). -- Simplify the 'missing document' error message in the algebra for constant document names (which is the default). =================================================================== cf., http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_XQuery/is-before4.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_XQuery/union.out.00.html Peter and/or Jan R., could you please comment? > - tests/BugDay_2005-12-19_0.9.3/attribute_after_non-attribute_content.SF-1351516 disabled. > - tests/BugDay_2005-12-19_0.9.3/shred_doc_with_space.SF-1211372: windows only minor --- either approve or change access rights on Windows machine: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.32.d.1-Windows6.0/tests_BugDay_2005-12-19_0.9.3/shred_doc_with_space.SF-1211372.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/Int.64.64.d.1-Windows6.0/tests_BugDay_2005-12-19_0.9.3/shred_doc_with_space.SF-1211372.out.00.html > - tests/clients/php_monetdb fixed. > - runtime/procs/approve approved. > - modules/pftijah/{load,sigs,procs} approved. > - tests/BugsViaSourgeforce/{invisible_error_messages.SF-1409122,ID.1766173} approved. > (2) remove or disable the following tests, which seem incorrect or outdated: > - tests/BugTracker/xs_untypedAtomic.SF-1509806.mps disabled. > - tests/BugTracker/Zombie_document.SF-2009556 Nothing yet; we get a timeout: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/Zombie_document.SF-2009556.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/Zombie_document.SF-2009556.err.00.html > - tests/BugTracker/function_parameter_without_type.SF-1898518 approved. > - tests/BugTracker/fn-root_fn-id_on_attribute_nodes.SF-1 disabled. > - tests/BugTracker/975028,non-existing_collection.SF-1991726 approved. > - tests/BugTracker/port_busy.SF-1809586 Nothing, yet; output might actually be OK / as expected: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/port_busy.SF-1809586.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/port_busy.SF-1809586.err.00.html > - tests/BugTracker/collection_management Nothing, yet; need to understand what's (supposed to be) going on: http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/collection_management.SF-1726599.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/tests_BugTracker/collection_management.SF-1726599.err.00.html > - tests/BugDay_2005-12-19_0.9.3/multiple_servers_2.SF-1385152 fixed. > - runtime/smack Nothing, yet; smack00 test works fine for MIL, MAL, SQL, but never worked (timout: hangs "forever") for XQuery http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/runtime/smack00.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/runtime/smack00.err.00.html > - benchmarks/mbench/{qa02,qa06} are now only run for Darwin9.8.0 G.32.32.d.1 > -- should also set NOT_ALGEBRA for this platform these are (and have been) skipped with Algebra on all platforms. > (3) ask Jennie to look at the following tests in runtime/xrpc/admin/: > - add_del_doc_norpc: approve? > - backup_restore_xrpc: empty, check this pls?? Since 12.11., these now fail with ERROR = !module import: error loading module from http://$HOST:47966/admin/admin.xq http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/runtime_xrpc_admin/add_del_doc_noxrpc.err.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d.1-Fedora10/runtime_xrpc_admin/backup_restore_noxrpc.err.00.html > (4) ask Jan Flokstra to look at the following tests in modules/pftijah/: > - coltest1 hang > - test_lms-or: order > - test_lms_rmoverlap, test_select_start*: empty fixed by Sjoerd's backport of Hennigs fixes from the head to the Nov2009 branch > (5) adapt or disable these test scripts to work on windows: > - tests/BugDay_2005-12-19_0.9.3/file_locked_after_shredding.SF-1238352 > (windows6 only) - > - tests/BugDay_2005-12-19_0.9.3/shredding_on_the_fly.SF-1377006: idem > - tests/BugTracker/predicate_selects_too_few_nodes.SF-1636588 - windows fails > to parse, because of Mtest substitution failure (for $v becomes for 4) fixed by Sjoerd. > (6) ignore output of tests (possibly for certain platforms only), as these are > system limits related (too deep recursion): > - tests/BugTracker/crash_on_concatenated_query.SF-1730547 - for all systems > - tests/BugTracker/server-side_compilation_crash.SF-1607210 - for all systems > - tests/BugsViaSourgeforce/ID.1015172{a,b,c}: timeout on windows5 (windows6 > works) > - tests/XQuery/step_attr_nametest: Darwin9.8.0 G.32.32.d too deep recursion, > - tests/XQuery/typeswitch3 - too deep recursion on various systems.. > - tests/BugDay_2005-11-09_0.9.3/xquery_crashtest.SF-1207048: pf recursion > check not triggered on Fedora10 G.32.32.d.1 to be done ... didn't get further than here, so far ... Stefan ---------------------------------------------------------------------- Comment By: Peter Boncz (boncz) Date: 2009-11-11 11:45 Message: Thanks. I looked into this and fixed a problem with XQuery updates that indeed was present. This will reduce the amount of problems, removing (almost) all of those tests that are XQuery updates from the list. The update bug was caused by a later modification that succeeded my own check of the testweb that I performed before checking in the code. At that time, already, I found it very hard to work with the test web. Note, this is not a complaint to anyone in particular. However a matter of fact. Simply too many tests fail, such that one is forced to spend an hour just to compare the state of the testweb before and after a check-in, to detect which of the tests that fail were actually already failing before the check-in. Let alone analyze what is happening. my suggestons are as follows: (1) approve output: - tests/BugTracker/empty_file.SF-2017862: SunOS specific version only!! - tests/BugTracker/XML_document_cache_broken.SF-1414720 - tests/W3C_use_cases/XQ/NS/* (or remove tests!) - tests/XQuery/{div.is-before4,union} - tests/BugDay_2005-12-19_0.9.3/attribute_after_non-attribute_content.SF-1351516 - tests/BugDay_2005-12-19_0.9.3/shred_doc_with_space.SF-1211372: windows only - tests/clients/php_monetdb - runtime/procs/approve - modules/pftijah/{load,sigs,procs} - tests/BugsViaSourgeforce/{invisible_error_messages.SF-1409122,ID.1766173} (2) remove or disable the following tests, which seem incorrect or outdated: - tests/BugTracker/xs_untypedAtomic.SF-1509806.mps - tests/BugTracker/Zombie_document.SF-2009556 - tests/BugTracker/function_parameter_without_type.SF-1898518 - tests/BugTracker/fn-root_fn-id_on_attribute_nodes.SF-1 - tests/BugTracker/975028,non-existing_collection.SF-1991726 - tests/BugTracker/port_busy.SF-1809586 - tests/BugTracker/collection_management - tests/BugDay_2005-12-19_0.9.3/multiple_servers_2.SF-1385152 - runtime/smack - benchmarks/mbench/{qa02,qa06} are now only run for Darwin9.8.0 G.32.32.d.1 -- should also set NOT_ALGEBRA for this platform (3) ask Jennie to look at the following tests in runtime/xrpc/admin/: - add_del_doc_norpc: approve? - backup_restore_xrpc: empty, check this pls?? (4) ask Jan Flokstra to look at the following tests in modules/pftijah/: - coltest1 hang - test_lms-or: order - test_lms_rmoverlap, test_select_start*: empty (5) adapt or disable these test scripts to work on windows: - tests/BugDay_2005-12-19_0.9.3/file_locked_after_shredding.SF-1238352 (windows6 only) - - tests/BugDay_2005-12-19_0.9.3/shredding_on_the_fly.SF-1377006: idem - tests/BugTracker/predicate_selects_too_few_nodes.SF-1636588 - windows fails to parse, because of Mtest substitution failure (for $v becomes for 4) (6) ignore output of tests (possibly for certain platforms only), as these are system limits related (too deep recursion): - tests/BugTracker/crash_on_concatenated_query.SF-1730547 - for all systems - tests/BugTracker/server-side_compilation_crash.SF-1607210 - for all systems - tests/BugsViaSourgeforce/ID.1015172{a,b,c}: timeout on windows5 (windows6 works) - tests/XQuery/step_attr_nametest: Darwin9.8.0 G.32.32.d too deep recursion, - tests/XQuery/typeswitch3 - too deep recursion on various systems.. - tests/BugDay_2005-11-09_0.9.3/xquery_crashtest.SF-1207048: pf recursion check not triggered on Fedora10 G.32.32.d.1 what will be left, then ======================= The only problems that will be left after following the above are in fact problems that occur only on a single system. Rather than indicating a platform deficiency, however, it is still very likely that these differences indicate the precense of a bug, that is only triggered under certain circumstances that happen to hold on that platform. so these are interesting to look at; all of them were already present in the last Stable Fedora10.G.64.64.d.0 - merge_union2: non dense head - tests/XQuery/merge_union2 Fedora10 G.64.64.d.0 - crash - tests/XQuery/step_attribute8, Fedora10 I.64.64.d.1: crash - tests/XQuery/orderby6 Fedora10 G.64.64.s.1 - crash - tests/XQuery/step_descendant-or-self3 Fedora10.G.32.32.d.1 -- segmentation fault - xbench/tc/md/q10 - tests/W3C_use_cases/XQ/XMP/Q07 windows - !ERROR: couldn't read name (1000000152_qn_uri) 1 - tests/BugTracker/32_docs.SF-1730617: sunos5 g32.32.d.1 - segmentation fault - mbench/{qs28,31} - tests/XQuery/{step_duplicates,transient_upward_steps} - benchmarks/XPathMark/{Q27,28} - benchmarks/XPath/q13 - benchmarks/XMark/q04 - tests/XQuery/step_attr_nametest (also on Darwin9.8.0 G.32.32.d.1) sunos5 g32.32.d.1 - 4 times inserted nil due to errors at tuples 0, 1, 2, 3 - tests/BugTracker/child-steps-and-replace.SF-2716723: Debian4.0 G.32.32.d.1 - !fatal error: columns referenced in trace message operator not found - tests/BugTracker/fn-trace.SF-2805513 (also Gentoo2.0.1 G.64.64.d.1) ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-08 18:13 Message: The same test that fail with Aug2009_NFI, while working with Aug2009, indeed also fail with Nov2009. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2009-11-08 16:50 Message: For completeness: I checked this on my 64-bit Fedora 10 Linux desktop, using a "testing" build of MonetDB, i.e., configured with --disbale-debug --enable-optimize --enable-assert . I ran Mtest.py (more precisely make check, respectively RunMtest in the build directory) seperately for the Aug2009 and the Aug2009_NFI brach (builds) of pathfinder, collected the console output in files, and run diff over those (as well as over the times.sql files that Mtest.py creates in its <TSTTRGBASE> directory. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2893875&group_id=56967 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
