Bugs item #1588053, was opened at 2006-10-31 18:24 Message generated for change (Comment added) made by stmane You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1588053&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: PF/tests Group: Pathfinder CVS Head Status: Open Resolution: None >Priority: 4 Private: No Submitted By: Stefan Manegold (stmane) Assigned to: Nobody/Anonymous (nobody) Summary: XQ: W3C_use_cases namespace tests fail: Bug or Feature? Initial Comment: With MonetDB/XQuery, the tests in tests/W3C_use_cases/XQ/NS/Tests/ produce output that differs from the "expected results" given in http://www.w3.org/TR/2006/WD-xquery-use-cases-20060608/#ns-queries-results cf. http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q2.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q3.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q4.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q5.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q6.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q7.out.00.html I'm not sure, whether this is indeed a bug, or rather a "feature", i.e., an allowed variation of the output within the XQuery/XML/namespace specifications. Could an expert help, please? ---------------------------------------------------------------------- >Comment By: Stefan Manegold (stmane) Date: 2009-11-11 18:46 Message: =================================================================== 2009/11/11 - stmane: All,1.1.1.1.32.1 Disabled tests that are bound to fail as MonetDB/XQuery does (can) not support correct namespace serialization (yet?). See also ID: 1588053 "XQ: W3C_use_cases namespace tests fail: Bug or Feature?" https://sourceforge.net/tracker/?func=detail&aid=1588053&group_id=56967&atid=482468 =================================================================== ---------------------------------------------------------------------- Comment By: Jan Rittinger (tsheyar) Date: 2007-09-13 10:15 Message: Logged In: YES user_id=993208 Originator: NO @Jennie: I think you refer to the third examples (serializing node d). Node e needs no namespace declarations as these are already made available by node d (nodes inherit the namespaces by their ancestors as long as they do not overwrite them). @Jens: The problem with your idea can be seen best if you look at the second example. If you store a namespace node for namespace 10 (name v) as child of node 1 (name a) and you only want to serialize node 3 (name c) then you would need to apply ancestor::*/namespace::* path steps for each result node to get the correct namespace definitions. This is getting even trickier if a node from a document (with namespace definitions in scope) has been part of a subtree copy due to node constructors (like in the above query Q2). Then namespace nodes at an ancestor are not reachable anymore (or node constructors would get way more complicated). ---------------------------------------------------------------------- Comment By: Jens Teubner (teubner) Date: 2007-09-12 18:54 Message: Logged In: YES user_id=731390 Originator: NO Just a little idea that I haven't thought through at all, but maybe you find it helpful. In XPath 1.0 there was a notion of `namespace nodes', which were part of the XML document tree similar to other nodes. Namespace nodes were even accessible via the special navigation axis `namespace::'. This concept was abandoned in XPath 2.0/XQuery, but it may be an interesting approach to store namespace information. I could think of "namespace nodes" that are implemented analogously to attributes (in fact, during serialization they even become attributes). This might be easier to implement than some new "namespace tree". Just my 2 US-¢ ---------------------------------------------------------------------- Comment By: Ying Zhang (yingying) Date: 2007-09-12 17:58 Message: Logged In: YES user_id=341633 Originator: NO Hi Jan, Why no namespaces need to be generated for the serialization of node e? As node e has very much the same position as node c, I would expect that namespace nodes 15, 11 and 10 should be printed. Jennie (who hasn't read the Serialization spec.) ---------------------------------------------------------------------- Comment By: Jan Rittinger (tsheyar) Date: 2007-09-12 11:22 Message: Logged In: YES user_id=993208 Originator: NO I think Jens refers to a different problem. What is missing for the W3C tests to work fine is the preservation of Namespace declarations during shredding and serialization. (If I'm not 100% wrong) we are currently using the declarations only to assign the namespace URIs to all other nodes -- we however do not store Namespace declarations in the document encoding. (I think that's also what Benjamin wants: '... retreive it "as-was"'.) Jens on the other side proposed to attach node constructors with all Namespaces visible in the XQuery query. ---- Sketch for a solution ---- To solve the 'reported' bug we would probably need to store an additional namespace declaration *tree* for each document. This tree would then maintain the relationship between all namespaces in a document and each node in the encoded document would refer to a node in the namespace tree. Using the node reference then allows to collect all visible namespaces by walking along the path from root to the referred namespace node. The price for this functionality is to build the additional namespace tree during shredding, maintaining two additional columns in the document encoding (ns_id|ns_cont) e.g., during updates or during node constructors in a query, and printing the namespace declarations during the serialization. <a xmlns:v="v" xmlns:w="w"> <b xmlns:x="x"> <c xmlns:y="y" xmlns:w="another_w"/> </b> <d xmlns:z="z"> <e/> </d> </a> Would result in the following document encoding: pre|size|level|kind|name|cont|ns_id|ns_cont 0 5 -1 doc - 1 -1 1 1 3 0 elem a 1 11 1 2 1 1 elem b 1 12 1 3 0 2 elem c 1 14 1 4 1 1 elem d 1 15 1 5 0 2 elem e 1 15 1 and the following namespace tree (encoded with a pre|parent encoding) pre|par|name|value 10 - v v 11 10 w w 12 11 x x 13 12 y y 14 13 w another_w 15 11 z z Depending on the query the serialization now has to do different things: 1.) result is node 2 (name b): During the serialization of node b namespace 12, 11, and 10 have to be printed -- the path from the referrenced namespace node (12) to root -- and during the serialization of node c namespace 14, 13 have to be printed -- the difference to the previously printed nodes. 2.) result is node 3 (name c): During the serialization of node c namespace 14, 13, 12, and 10 have to be printed -- namespace node 11 has to be ignored as node 14 overwrites it. 3.) result is node 4 (name d): During the serialization of node d namespace 15, 11, and 10 have to be printed and during serialization of node e no namespace attributes have to be generated. All in all it looks like quite some work on the runtime part (and a small amount of work in the compiler -- fixing the node constructors). (Due to lack of time) I think I'm not the one implementing such a change (and thus un-assign myself from the bug). Are there any volunteers? ---------------------------------------------------------------------- Comment By: Jens Teubner (teubner) Date: 2007-09-11 19:15 Message: Logged In: YES user_id=731390 Originator: NO Sorry I cannot be of much help on this any more. A long while ago, I did some changes to the namespace resolution that are somewhat preparatory for what is needed here. During namespace resolution, an environment with statically-known namespaces is carried around. The implementation thereby is ready such that the environment could be annotated to every abstract syntax tree expression node (environments are properly copied already, so you don't have to worry about side effects to the environment during namespace resolution). The large chunk of the work still remains to be done: -- Annotate the statically known namespaces to AST nodes. (I think this could be done trivially by annotating the current namespace environment to every AST node.) -- Propagate them to the Core tree. This needs a bit more work, and the environment needs to be correctly propagated to the right Core tree node. I remember this to be a bit tricky because there is no 1:1 relationship between Core and AST. -- Finally propagate the information to the backend code generator. (This also means that the environment needs to be represented as a MonetDB data structure.) Once available on the MIL side, the namespace environment can not only be used to properly implement serialization, but also to implement casts to QNames correctly (i.e., things like `element { "foo:bar" } { "Hello World!" }'). ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2007-09-11 18:40 Message: Logged In: YES user_id=572415 Originator: YES Jan, Teggy, potentially being a "show-stopper" in below mentioned project (which has quite some "publicity potential" for MonetDB/XQuery and hence Pathfinder), I think it's reasonable to increase the priority of this bug, again. Despite Jens comment about the amount of work/effort that will be required to fix this bug, I'd like to raise the question, whether there is a reasonable way to get this one fixed in the foreseeable future --- surely not for the feature freeze later this week. Obviously, the required expertise is in Munich. Maybe, there is a ways to combine forces (TUM+CWI, possibly also +University of Versailles) to get this bug solved say before the end of this year? Thanks in advance! Stefan ---------------------------------------------------------------------- Comment By: Benjamin Nguyen (benjie-nguyen) Date: 2007-09-11 16:10 Message: Logged In: YES user_id=1887942 Originator: NO Greetings, we have read up on the bug, we would like to stress that this bug is important to us : We use MonetDB as XML repository in the French ANR - RNTL "Web Content" project, which includes a large spectrum of industrial partners, such as Thales, EADS, and a number of SME. Here is the context of our application : We want to store documents in the database, and we have defined a UML model to represent these documents. This model is implemented by some partners (EADS) using JAVA classes and objects. These classes are serialized in XML using JAXB, and sent, via a web service that we have developped, (and can share if there is any interest), to the MonetDB server. This XML document is stored, and may be retrieved and sent to other clients, via a web service call. These clients will transform, using JAXB, the XML document into Java objects. The problem : In order to transform the XML document to java classes, JAXB needs a reference to the XSD representing our data model. The reference to this namespace is currently _lost_ when we store the document in MonetDB, therefore once we output the XML document over the web server, nothing can be made of it by the client that wishes to process it, once transformed into java objects. We would rather not have to implement anything specific in order to manage these lost namespaces, so we would like to advocate for a revision of the importance of this bug, since JAXB is now standard with Java 6 and will probably be widely used to generate XML data to be store, and that we do not have much control on the XML that is generated by JAXB. We do however need to store it and retreive it "as-was". Kind regards, Benjamin NGUYEN & Loïc SAINT-GHISLAIN PRiSM Laboratory University of Versailles France ---------------------------------------------------------------------- Comment By: Jens Teubner (teubner) Date: 2007-02-20 09:34 Message: Logged In: YES user_id=731390 Originator: NO My interpretation of the W3C XQuery Serialization specification is that our serialization is wrong here. So it is a bug. Fixing this involves quite some changes to the code. Currently, namespaces are completely handled during the (early) namespace resolution (file ns.c). During namespace resolution, namespace attributes are stripped from all element constructors and we lose the direct correspondence between namespace bindings and element constructors. To fix this corner case of our serializer, we'd have to propagate more details about namespaces all the way through our compiler pipeline. (Propagating namespace bindings through the pipeline would also allow us to interpret computed QNames in element constructors correctly.) If I compare the effort required to fix this issue and the significance of the problem, I would say this is a low-priority bug. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2007-02-19 15:51 Message: Logged In: YES user_id=43607 Originator: NO Assigning to Jens to decide the answer (or to assign to somebody else who has to decide). The difference is that the expected output contains namespace declarations that are presumbaly in the input document byt which are not used in the output of the query. So the question is whether those namespace declarations are optional according to the XQuery standard. ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2006-11-12 12:25 Message: Logged In: YES user_id=572415 for the test results with the latest "Stable" release branch (via `make check` with debugmask 10) see http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q2.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q3.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q4.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q5.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q6.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Stable/pathfinder/.mTests103/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q7.out.00.html for the test results with the latest "Current" development trunk (via `Mtest.py -r -d0` with debugmask 0) see http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests03/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q2.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests03/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q3.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests03/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q4.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests03/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q5.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests03/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q6.out.00.html http://monetdb.cwi.nl/testing/projects/monetdb/Current/pathfinder/.mTests03/GNU.64.64.d-Fedora4/tests_W3C_use_cases_XQ_NS/Q7.out.00.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1588053&group_id=56967 ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
