These are great suggestions and this is the kind of feedback we need to make a better product. In V6 for the custom transforms I focused on transformations which could be predicted without inspecting instance documents. For example the number/string issue. I could "look" at a value and say "that looks like a number" and output it as a JSON number but the next one might "look like" a string ... I felt that would cause significant confusion. A given element should produce either a number or a string every time. I took this to more complex transformations as well. Such as the array case. But the array case is more confusing in some ways as you have found. Not only do you need to enumerate which elements should be arrays but when to choose to put things into arrays and when not to would be dependent on the document instance.\ So some JSON might end up with a food as an object and some as an array ... this was the rationale behind not doing these kinds of "do what we think you really meant" issues. But ... the flip side is that we lose data in the case of repeating elements for custom transforms if the element is not marked as an array. Which is worse ? inconsistent transformations or loss of data ... internally ,recently, we had this discussion and I am leaning towards loss of data is worse.
Now getting rid of the wrapping element is a whole different problem. It could be useful for sure but then it breaks round-trip ... No longer would we know to insert a <food> element on round trip. But if this was configurable it could work. Anyway just touching on the kinds of issues that confound the choices of behavior. Some things would be really great for some use cases and really aweful for others. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected] Phone: +1 812-482-5223 Cell: +1 812-630-7622 www.marklogic.com<http://www.marklogic.com/> From: [email protected] [mailto:[email protected]] On Behalf Of Stu Salsbury Sent: Sunday, April 21, 2013 10:12 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] transform-to-json -- custom config drops elements? Sorry, we cross-emailed. If the parent of the item with multiplicity has any attributes or other elements then the request to unwrap would be incompatible and would fall back to not unwrapping . In other words, the only schema for an element that would be compatible with this parameter is a complexType that has a single sequence with multiplicity. <xs:element name="foods"> <xs:complexType> <xs:sequence maxOccurs="unbounded" minOccurs="0"> <xs:element ref="food"/> </xs:sequence> </xs:complexType> </xs:element> I would submit that even though it would *only* be compatible with such a schema, that it is still worthwhile due to the "usefulness" or being able to group elements in XML in order to get arrays cleanly into json while maintaining decent semantics with pluralization of nouns. Without the feature, one needs to fudge the semantics of pluralization in either the XML or the JSON. But if this issue occurs at the top level of a node tree, then there is no longer a single root element. For example, I could start with the XML I have: <foods> <food/> <food/> </foods> and pre-transform it for purposes of JSON to <foods/> <foods/> ... which, of course, would need to have its own separate schema, all of which is work for the app developer. And we'd be left without a valid document since nothing contains the new foods elements. That this "feature" would be sort of a corner case is admittedly irksome, and yet the benefits seem large. Perhaps with the right parameter name, it would taste better: "unwrap-unitary-sequences" ?? :) On Sun, Apr 21, 2013 at 9:42 AM, Stu Salsbury <[email protected]<mailto:[email protected]>> wrote: Or maybe you meant: <foods > <food id="1" name="celery"> <location name="England"> <currency>pounds</currency> </location> </food> <food id="2" name="beets"> <location name="France"> <currency>francs</currency> </location> </food> <food id="3" name="goat cheese"> <location name="United States"> <currency>dollars</currency> </location> </food> </foods> which, with the appropriate schema, would hypothetically get me: {"foods":[{"id":1, "name":"celery", "location":{"name":"England", "currency":"pounds"}},{"id":2, "name":"beets", "location":{name:"France", "currency":"francs"}}, {"id":3, "name":"goat cheese", "location":{"name":"United States","currency":dollars"}}]} .. since you've already got the functionality of folding attributes and child elements together as subobjects of their parent. On Sun, Apr 21, 2013 at 9:23 AM, Stu Salsbury <[email protected]<mailto:[email protected]>> wrote: I'm not sure I follow. If I had food repeating, and food had a cost attribute and a location element (and I had appropriate schema, and the two hypothetical map parameters) I'd get: {"foods":[{"id":1, "name":"celery", "cost":50.2, "location": "Norway"},{"id":2, "name":"beets", "cost":23.3, "location":"Finland"},{"id":3, "name":"goat cheese", "cost":0.5, "location":"Denmark"}]} But wouldn't the example just be demonstrating what happens if I change the "price" element to a "cost" attribute and add a "location" element? I think I must be misunderstanding your question. On Sun, Apr 21, 2013 at 8:54 AM, David Lee <[email protected]<mailto:[email protected]>> wrote: Q: in your example what if foods had an attribute or more then one element name? ( say food repeating and cost attribute and location element) Sent from my iPad (excuse the terseness) David A Lee [email protected]<mailto:[email protected]> 812-630-7622<tel:812-630-7622> On Apr 21, 2013, at 10:50 AM, "Stu Salsbury" <[email protected]<mailto:[email protected]>> wrote: Thank you, David. Adding "element-namespace" certainly helps. I wish I'd thought of that. I guess that, unless the function is directly told that food is to be an array, it only renders the last element? Also, it would be great if: (a) there was a parameter to indicate that any elements which have multiplicity should be arrays; and (b) there was a parameter to indicate that the array elements should not be in the json (i.e. their content remains but is not wrapped with their name as a key/value). something like: (: I'm making up these parameter names -- this code isn't real!!! :) , $_ := map:put($c,"arrays-for-multiplicity",true); , $_ := map:put($c,"unwrap-array-elements",true); so that I'd get: {"foods":[{"id":1, "name":"celery", "price":50.2},{"id":2, "name":"beets", "price":23.3},{"id":3, "name":"goat cheese", "price":0.5}]} On Sun, Apr 21, 2013 at 4:49 AM, David Lee <[email protected]<mailto:[email protected]>> wrote: You need to indicate that food is an array element. Since you have now put food in a namespace your array-element-names isnt finding it. You either need to use an xs:QName() to give the full name of "food" or to declare the default namespace for the configuration using e.g. this should work , $_ := map:put($c,"array-element-names", xs:QName("agt:food") ) alternatively , $_ := map:put($c,"element-namespace","http://example.com/agt";) which has the nice side effect of making the transformation reversable. ----------------------------------------------------------------------------- David Lee Lead Engineer MarkLogic Corporation [email protected]<mailto:[email protected]> Phone: +1 812-482-5224<tel:%2B1%20812-482-5224> Cell: +1 812-630-7622<tel:%2B1%20812-630-7622> www.marklogic.com<http://www.marklogic.com/> From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Stu Salsbury Sent: Sunday, April 21, 2013 1:18 AM To: [email protected]<mailto:[email protected]> Subject: [MarkLogic Dev General] transform-to-json -- custom config drops elements? I have the following xml <foods xmlns="http://example.com/agt"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > <food id="1" name="celery"> <price>50.2</price> </food> <food id="2" name="beets"> <price>23.3</price> </food> <food id="3" name="goat cheese"> <price>0.5</price> </food> </foods> With the following xsd: <xs:schema targetNamespace="http://example.com/agt"; attributeFormDefault="unqualified" elementFormDefault="unqualified" xmlns:xs="http://www.w3.org/2001/XMLSchema"; > <xs:element name="price" type="xs:decimal"/> <xs:element name="food"> <xs:complexType> <xs:sequence> <xs:element ref="price"/> </xs:sequence> <xs:attribute name="id" type="xs:integer"/> <xs:attribute name="name" type="xs:string"/> </xs:complexType> </xs:element> <xs:element name="foods"> <xs:complexType> <xs:sequence> <xs:element ref="food" maxOccurs="unbounded" minOccurs="0"/> </xs:sequence> </xs:complexType> </xs:element> </xs:schema> When I execute the following query in the console: xquery version "1.0-ml"; import module namespace json = "http://marklogic.com/xdmp/json"; at "/MarkLogic/json/json.xqy"; declare namespace agt = "http://example.com/agt";; let $c := json:config("custom") , $_ := map:put($c,"array-element-names", "food") return json:transform-to-json( doc("foods.xml") , $c ) ... I get: {"foods":{"food":{"id":3, "name":"goat cheese", "price":0.5}}} Something isn't clicking for me. What happened to the food elements? No array, and only the last one made it into the output. Note that taking out the array-element-names does *not* solve the missing food elements. Is this a bug or am I doing it wrong? I just started with MarkLogic Server, so I could very well be doing it wrong. Any advice appreciated. Regards, Stu _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
