Hi David,
Thank you for explaining all the things. I went through the documentation
that you have shared with me again.
What I understand, my code could be fixed in following two ways,
1. First convert JOSN object to string format however the jsonLib:parse()
is not accepting the value that returns from xdmp:to-json($params)/node().
It says, this is not a string and If I apply xdmp:quote(), this starts
working with existing code that requires xs:string as input. The
json:transform-to-json()
is not working however logically this should as based on the documentation,
this also returns xs:string and take json as object. I want input json as
xs:string for my function.
let $params := xdmp:get-request-body()
let $json := xdmp:to-json($params)/node()
let $json := try{ jsonLib:parse(xdmp:quote($json)) } catch ($e)
{xdmp:log($json)} (: doing xdmp:quote() is strange here as
xdmp:to-json()/node() returns string itself.:)
2. Using ML8 function and wrapping the result with in <json> that requires
for my API code.
let $params := xdmp:get-request-body()
let $custom :=
let $config := json:config("custom")
return
(
map:put($config, "element-namespace", "http://marklogic.com/json";),
$config
)
let $json := try{ (json:transform-from-json($params, $custom)) } catch
($e) {()}
let $json := <json type="object" xmlns="http://marklogic.com/json
">{$json}</json>
The json functions those requires xs:string to convert into my APP specific
output.
declare function json:parse(
$json as xs:string,
$enableExtensions as xs:boolean
) as element(json:json)
{
let $test := if($isSupported) then () else
error(xs:QName("json:UNSUPPORTED"), "The JSON library isn't supported under
this version of MarkLogic, upgrade to 4.2 or later")
let $tokens := json:tokenize($json)
let $value := json:parseValue($tokens, 1, (), $enableExtensions)
let $test :=
if(xs:integer($value/@position) != count($tokens) + 1)
then json:outputError($tokens, xs:integer($value/@position),
"Unhandled tokens")
else ()
return <json:json>{ $value/(@type, @boolean), $value/node()
}</json:json>
};
declare function json:parse(
$json as xs:string
) as element(json:json)
{
json:parse($json, true())
};
Please advise if I am doing wrong thing.
Regards,
Indrajeet
On Tue, May 5, 2015 at 4:17 AM, David Lee <[email protected]> wrote:
> I will start you in the right direction ...
>
> XDMP-AS: (err:XPTY0004) $json as element(json:json) -- Invalid coercion:
> <error:error xsi:schemaLocation="http://marklogic.com/xdmp/error
> error.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:error="http://marklogic.com/xdmp/error"><error:code>XDMP-AS</error:code><error:name>err:XPTY0004</error:...</error:error>
> as element(json:json)
>
>
>
>
>
> This is explained in the compatibilty doc, I suggest you start there.
> Most of the internal JSON API's have changed signatures to use the native
> JSON nodes - this requires app level changes.
>
>
>
> https://docs.marklogic.com/guide/relnotes/chap4
>
>
>
> See:
>
> https://docs.marklogic.com/guide/relnotes/chap4#id_94947
>
> " The json:transform-to-json
> <https://docs.marklogic.com/json:transform-to-json> function uses
> xdmp:to-json <https://docs.marklogic.com/xdmp:to-json>, so it also
> returns a document-node() in MarkLogic 8"
>
>
>
> The error below are to be expected as a side effect of this.
>
>
>
> Any pre V8 code that uses the marklogic JSON functions is likely to need
> changing - That change may percolate up the API Layer - or it can be
> issolated within the function/api to expose the same signatures as before -
> which is best depends on your use case - but in any case its going to take
> some work. The actual changes can be very simple in most cases - it is
> quite easy to convert between representations, most of the work is design -
> where do you want to make the changes, is it better to hide them as much as
> possible or to convert to the native JSON data model. There are advantages
> to both.
>
> But it starts by reading the above docs, and then looking at your code and
> any dependent code to make the right choices for you.
>
>
>
>
>
>
> -----------------------------------------------------------------------------
>
> David Lee
> Lead Engineer
> *Mark**Logic* Corporation
> [email protected]
> Phone: +1 812-482-5224
>
> Cell: +1 812-630-7622
> www.marklogic.com
>
>
>
> *From:* [email protected] [mailto:
> [email protected]] *On Behalf Of *Indrajeet Verma
> *Sent:* Monday, May 04, 2015 1:35 PM
> *To:* MarkLogic Developer Discussion
> *Subject:* Re: [MarkLogic Dev General] json namespace changes in
> json:transform-from-json($input)
>
>
>
> Hi David,
>
>
>
> Thank you for your explanation and help!
>
>
>
> 1. Could you explain what issues you are having ?
>
>
>
> XDMP-AS: (err:XPTY0004) $json as element(json:json) -- Invalid coercion:
> <error:error xsi:schemaLocation="http://marklogic.com/xdmp/error
> error.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xmlns:error="http://marklogic.com/xdmp/error"><error:code>XDMP-AS</error:code><error:name>err:XPTY0004</error:...</error:error>
> as element(json:json)
>
>
>
> Code in save()
>
> let $json := try{ jsonLib:parse(map:get($req-map, "req-body") ) } catch
> ($e) {($e)}
>
>
>
> 2. Where is this document used ?
>
> This saved document is being used to show user history.
>
>
>
> 3. Could you show the XQuery code that uses it?
>
> Please refer the code below.
>
>
>
> 4. What is not working, what you expected vs what you are getting etc.
>
> let $json := try{ jsonLib:parse(map:get($req-map, "req-body") ) } catch
> ($e) {($e)}
>
>
>
> I am expecting json:json however this is not coming. see below error.
>
> <error:format-string>XDMP-AS: (err:XPTY0004) $json as xs:string -- Invalid
> coercion: json:object(<json:object xmlns:xs="
> http://www.w3.org/2001/XMLSchema"; xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance"; xmlns:json="
> http://marklogic.com/xdmp/json"><json:entry
> key="docTitle"><json:value>histories: Search for
> app...</json:object>) as xs:string</error:format-string>
>
>
>
> One thing if you see, this seems a namespace mismatch issue. The namespace
> in the converted output is "http://marklogic.com/xdmp/json"; however this
> expects http://marklogic.com/json. I thing roxy is playing major role
> here in upgrading ML8. I am going to open a ticket in roxy as well for this.
>
>
>
> This "http://marklogic.com/json"; is being used by roxy to handle
> json.This namespace is in data model as well and index are also with same
> namespace. So if I will modify this namespace, Might be, the complete code
> repository require changes:).
>
>
>
> Actually I need to upgrade ML6 to ML8 without changing in UI & data model.
>
>
>
> I am sending you the code for your reference however what I have observed
> while upgrading, wherever the JSON is being used in the request body for
> creating files in the DB, everywhere the issues are coming.
>
>
>
> So the JSON format that is being sent in the request body in ML6 is not
> compatible with ML8 any more. I am not sure , is this roxy issue or the
> logic that has been written in the modules.
>
>
>
>
>
> declare function res:save( $type, $prefix)
>
> {
>
> try {
>
>
>
> invoke:invoke(
>
> "save",
>
> "http://marklogic.com/roxy/models/user/resources";,
>
> "/app/lib/user-resources.xqy",
>
> (
>
> auth:userName(),
>
> auth:userDirectory(),
>
> $type,
>
> $prefix,
>
> $req:request
>
> ),
>
> fn:false(),
>
> $cfg:RESOURCE-DB
>
> )
>
> } catch ($e) {xdmp:log($e)}
>
> };
>
>
>
> declare function res:save( $user-id, $root, $type, $prefix, $map)
>
> {
>
> let $doc :=
>
> element resource {
>
> attribute wk-pid {fn:concat($prefix,"-",
> tools:generate-uuid-v4())},
>
> element docType {$type},
>
> element ownerId {$user-id},
>
> element created {fn:current-dateTime()}
>
> }
>
> return
>
> (
>
> res:_save(
>
> fn:concat($root,"/", $doc/docType, "/", $doc/@wk-pid,".xml"),
>
> $doc,
>
> $map
>
> )
>
> )
>
> };
>
>
>
> (: ERROR IS COMING BELOW LINES
>
> let $json := try{ jsonLib:parse(map:get($req-map, "req-body") ) } catch
> ($e) {()}
>
> let $searchParameter := srch:json-search-params($json)
>
> :)
>
> declare function res:_save( $uri, $doc, $map )
>
> {
>
> (:Working in ML6:)
>
> let $req-map := map:map($map)
>
> let $json := try{ jsonLib:parse(map:get($req-map, "req-body") ) } catch
> ($e) {()}
>
> let $searchParameter := srch:json-search-params($json)
>
> (:
>
> I am writing to support ML8
>
> let $params := xdmp:get-request-body()
>
> let $custom :=
>
> let $config := jsonl:config("custom")
>
> return
>
> (
>
> map:put($config, "element-namespace", "
> http://marklogic.com/json";),
>
> $config
>
> )
>
> let $json := try{ (jsonl:transform-from-json($params, $custom)) } catch
> ($e) {()}
>
> let $json := <json type="object" xmlns="http://marklogic.com/json
> ">{$json}</json>
>
> :)
>
>
>
> let $update :=
>
> element resource {
>
> $doc/@*,
>
> $doc/docType,
>
> element docTitle {xs:string((map:get($req-map,"docTitle"),
> $json/json:docTitle,
> fn:concat($doc/docType,"-",fn:current-dateTime()))[1])},
>
> $doc/ownerId,
>
> $doc/created,
>
> element modified {fn:current-dateTime()},
>
> element shared {},
>
> element parameter {
>
> for $node in $json/element()[fn:local-name(.) =
> ('type','contentColletion','source')]
>
> return
>
> element {fn:local-name($node)} {
>
> $node/fn:string()
>
> }
>
> },
>
> if ($searchParameter) then
>
> (
>
> (: $searchParameter :)
>
> (: element cts-query
> {search:parse($searchParameter/queryText/fn:string(), $cfg:SEARCH-OPTIONS)}
> :)
>
>
>
> (:ticket 4885: save-query process extract contentCollectionId
> :)
>
> if($searchParameter/element()[fn:local-name(.)
> ='contentCollection'])
>
> then (
>
> element contentCollectionId
> {res:extractCollectionId($doc/ownerId,$searchParameter/element()[fn:local-name(.)
> ='contentCollection']/fn:string())}
>
> )
>
> else ()
>
> )
>
> else (),
>
> element {fn:node-name($json)} {
>
> $json/@*,
>
> $json/node()[fn:not(fn:local-name(.) =
> ("docUrl","docId","docTitle","created","modified"))]
>
> }
>
> }
>
> return (
>
> $update,
>
> tools:invoke("document-insert",
>
> (
>
> $uri,
>
> $update
>
> ),
>
> $cfg:RESOURCE-DB
>
> )
>
> )[1]
>
> };
>
>
>
> (:~
>
> : Converts incoming JSON structure into internal search parameter
> structure.
>
> : It will be stored into user resources and used for query construction.
>
> :)
>
> declare function m:json-search-params( $json as element(json:json))
>
> as element (searchParameter)?
>
> {
>
> if ($json/json:searchParameter) then
>
> element searchParameter {
>
> for $item in $json/json:searchParameter/*
>
> where $item/@type != 'object'
>
> return
>
> element {fn:local-name($item)} {
>
> if ($item/@type = 'array') then (
>
> for $value in $item/json:item[fn:not(@type =
> ('object','array'))]
>
> return
>
> element value {
>
> attribute type {if ($value/@type = "number") then
> "xs:unsignedLong" else "xs:string"},
>
> $value/text()
>
> }
>
> ) else (
>
> attribute type {if ($item/@type = "number") then
> "xs:unsignedLong" else "xs:string"},
>
> $item/text()
>
> )
>
> }
>
> }
>
> else
>
> ()
>
> };
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Mon, May 4, 2015 at 9:44 PM, David Lee <[email protected]> wrote:
>
> If those modules are custom modules (written by you or your team, not
> by marklogic or roxy), then yes this is the right place to ask.
>
>
>
> If you goal is to minimize client side changes - then you will need to
> modify your XQuery code wherever V6 vs V8 compatibility has changed.
>
>
>
> Here is a good place to start in order to identify potential changes
> needed.
>
>
>
> https://docs.marklogic.com/guide/relnotes/chap4
>
>
>
> I am not familiar with where this element would be used by a ML API
>
> <json type="object" xmlns="http://marklogic.com/json";>
>
>
>
> That is neither directly compatible with V6,7 or 8 'basic' JSON format,
>
> nor is it the namespace or format for a ML server side API that I am aware
> of.
>
>
>
> In the original question regarding use of xml:transform-to-xml , in
> general
>
> what you are doing is correct - the JSON is being translated to XML to the
> extent
>
> you describe you wanted.
>
> Since you don't reference exactly what it is that you are having problems
> with, its hard to guess ...
>
> Could you explain what issues you are having ?
>
> Where is this document used ?
>
> could you show the XQuery code that uses it?
>
> What is not working, what you expected vs what you are getting etc.
>
>
>
>
>
>
> -----------------------------------------------------------------------------
>
> David Lee
> Lead Engineer
> *Mark**Logic* Corporation
> [email protected]
> Phone: +1 812-482-5224
>
> Cell: +1 812-630-7622
> www.marklogic.com
>
>
>
> *From:* [email protected] [mailto:
> [email protected]] *On Behalf Of *Indrajeet Verma
> *Sent:* Monday, May 04, 2015 11:48 AM
>
>
> *To:* MarkLogic Developer Discussion
> *Subject:* Re: [MarkLogic Dev General] json namespace changes in
> json:transform-from-json($input)
>
>
>
> Hi David,
>
>
>
> Might be I am wrong here and I will raise a ticket in roxy as well however
> in this case roxy is sending the request body same as it receives from the
> request.
>
>
>
> The manipulation and transformation logic is happening inside modules and
> that by using ML server functions.
>
>
>
> Regards,
>
> Indy
>
>
>
>
>
> On Mon, May 4, 2015 at 9:12 PM, David Lee <[email protected]> wrote:
>
> You mention Roxy. That adds another layer. I suggests the Roxy forum
> for discussing this.
>
>
>
> https://github.com/marklogic/roxy/issues
>
>
>
> Roxy is a community developed project, Not part of the core MarkLogic
> product.
>
> I don't know to what extent Roxy has been modified to work with ML8 or
> what changes to suggest to code using it.
>
>
>
> Many of the low level JSON related API's changed in ML8 and not all the
> community developed libraries have incorporated those yet. The Application
> APIs provided by MarkLogic (like the Java Client API, XCC etc) are updated
> in-sync with the server update releases, externally managed products and
> libraries are independently managed by their authors and/or the community
> (for OS projects).
>
>
>
>
>
>
>
>
>
>
> -----------------------------------------------------------------------------
>
> David Lee
> Lead Engineer
> *Mark**Logic* Corporation
> [email protected]
> Phone: +1 812-482-5224
>
> Cell: +1 812-630-7622
> www.marklogic.com
>
>
>
> *From:* [email protected] [mailto:
> [email protected]] *On Behalf Of *Indrajeet Verma
> *Sent:* Monday, May 04, 2015 11:27 AM
>
>
> *To:* MarkLogic Developer Discussion
> *Subject:* Re: [MarkLogic Dev General] json namespace changes in
> json:transform-from-json($input)
>
>
>
> Thank you Eric and David for your response however our system is working
> fine in ML6 and details are,
>
>
>
> Same I wanted to achieve in ML8 without changing in UI and other search
> API that are being used for retrieval.
>
>
>
> Input request in body via roxy,
>
>
>
> {
>
> "docTitle": "histories: Search for apple",
>
> "type": "Search",
>
> "searchParameter": {
>
> "page": 1,
>
> "pageLength": 10,
>
> "queryText": "apple AND microsoft",
>
> "contentCollection": "All Content"
>
> }
>
> }
>
>
>
> Output that is being saved in ML,
>
>
>
> XML document <http://localhost:8000/qconsole/>
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <resource wk-pid="HIS-954e3f8f-2bbb-e79b-3747-7480b73db65b">
>
> <docType>
>
> Histories
>
> </docType>
>
> <docTitle>
>
> histories: Search for apple
>
> </docTitle>
>
> <ownerId>
>
> USR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00
>
> </ownerId>
>
> <created>
>
> 2015-05-04T17:13:40.126+05:30
>
> </created>
>
> <modified>
>
> 2015-05-04T17:13:40.126+05:30
>
> </modified>
>
> <shared>
>
> </shared>
>
> <parameter>
>
> <type>
>
> Search
>
> </type>
>
> </parameter>
>
> <json type="object" xmlns="http://marklogic.com/json";>
>
> <type>
>
> Search
>
> </type>
>
> <searchParameter>
>
> <page>
>
> 1
>
> </page>
>
> <pageLength>
>
> 10
>
> </pageLength>
>
> <queryText>
>
> Apple AND microsoft
>
> </queryText>
>
> <contentCollection>
>
> All Content
>
> </contentCollection>
>
> </searchParameter>
>
> </json>
>
> </resource>
>
>
>
>
>
>
>
> On Mon, May 4, 2015 at 7:36 PM, David Lee <[email protected]> wrote:
>
> Erik addressed the most likely issue/question.
>
> As a follow-up , for clarification -
>
> your 2 examples are fundamentally different use cases of
> json:transform-to-xml()
>
> The first use (without the $config) is intended *only* for the pre-V8
> use case of transforming
>
> arbitrary JSON into a 'black box' XML format with 100% fidelity. While
> it is a fairly readable format,
>
> Its not intended to be used directly - it is an implementation used by V7
> before we had native JSON data types in the database.
>
> It is maintained for backwards compatibility for that use case only (if
> you have existing data in V7 format).
>
>
>
> The second example (with the 'custom' strategy and a $config) is not
> intended as a refinement or otherwise
>
> replacement for the first case. Its designed to allow for custom
> transformations between JSON and XML -
>
> possibly bi-directionally - at the expense of losing 100% fidelity.
> Many of these use cases are no longer necessary in V8
>
> due to native support for JSON directly.
>
>
>
> It *can* be used as you have done - it should not be confused with the
> first case ( making any modifications
>
> at all to the 'basic' strategy transformation produces XML which will
> break any code expecting the V7 internal format),
>
> and I don't recommend it for API's that have a JSON interface - its much
> better to use that directly if possible.
> For API's that accept either JSON or XML - the formats for both are well
> defined - but the transformation between those formats
> can be very complex to get 100% right. So if you have JSON and using an
> API that accepts JSON - its much easier and better
>
> (and more efficient) to not attempt to transform it to XML.
>
>
>
>
>
>
>
>
> -----------------------------------------------------------------------------
>
> David Lee
> Lead Engineer
> *Mark**Logic* Corporation
> [email protected]
> Phone: +1 812-482-5224
>
> Cell: +1 812-630-7622
> www.marklogic.com
>
>
>
> *From:* [email protected] [mailto:
> [email protected]] *On Behalf Of *Erik Hennum
> *Sent:* Monday, May 04, 2015 9:16 AM
> *To:* MarkLogic Developer Discussion
> *Subject:* Re: [MarkLogic Dev General] json namespace changes in
> json:transform-from-json($input)
>
>
>
> Hi, Indrajeet:
>
> In MarkLogic 8, you would typically store JSON directly as native JSON and
> not converted to XML. The direct approach will improve performance and
> maintainability.
>
> In the particular case, xdmp:unquote() will turn the input string into a
> JSON document that can be inserted into the database.
>
> The particular case looks like the input to a search request. Is there is
> a specific reason it must be encoded as XML?
>
>
>
> Erik Hennum
> ------------------------------
>
> *From:* [email protected] [
> [email protected]] on behalf of Indrajeet Verma [
> [email protected]]
> *Sent:* Monday, May 04, 2015 6:00 AM
> *To:* MarkLogic Developer Discussion
> *Subject:* [MarkLogic Dev General] json namespace changes in
> json:transform-from-json($input)
>
> Hi,
>
>
>
> While upgrading ML6 to ML8, I needed to modify the JSON format and I did
> it based on my understanding on ML8 JSON handling however my senior is not
> convince with this solution so wanted to know if my approach is not best
> fit and could be better solution.
>
>
>
> Please someone take a look on my code and suggest any better solution?
>
>
>
> let $input := '{"docTitle":"histories: Search for apple", "type":"Search",
> "searchParameter":{"page":1, "pageLength":10}}'
>
>
>
> When I am using json:transform-from-json($input), getting below output
> with namespace "http://marklogic.com/xdmp/json/basic";
>
> <json type="object" xmlns="http://marklogic.com/xdmp/json/basic";>
>
> <docTitle type="string">
>
> histories: Search for apple
>
> </docTitle>
>
> <type type="string">
>
> Search
>
> </type>
>
> <searchParameter type="object">
>
> <page type="number">
>
> 1
>
> </page>
>
> <pageLength type="number">
>
> 10
>
> </pageLength>
>
> </searchParameter>
>
> </json>
>
>
>
> However I wanted the results with the namespace "http://marklogic.com/json";
> so I have coded like below,
>
>
>
> let $custom :=
>
> let $config := jsonl:config("custom")
>
> return
>
> (
>
> map:put($config, "element-namespace", "
> http://marklogic.com/json";),
>
> $config
>
> )
>
> let $json := try{ (jsonl:transform-from-json($input, $custom)) } catch
> ($e) {()}
>
> let $json := <json type="object" xmlns="http://marklogic.com/json
> ">{$json}</json>
>
> return $json
>
>
>
> *output:*
>
> <json type="object" xmlns="http://marklogic.com/json";>
>
> <docTitle>
>
> histories: Search for apple
>
> </docTitle>
>
> <type>
>
> Search
>
> </type>
>
> <searchParameter>
>
> <page>
>
> 1
>
> </page>
>
> <pageLength>
>
> 10
>
> </pageLength>
>
> </searchParameter>
>
> </json>
>
>
>
> Regards,
>
> Indy
>
>
> _______________________________________________
> General mailing list
> [email protected]
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
>
>
>
> _______________________________________________
> General mailing list
> [email protected]
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
>
>
>
> _______________________________________________
> General mailing list
> [email protected]
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
>
>
> _______________________________________________
> General mailing list
> [email protected]
> Manage your subscription at:
> http://developer.marklogic.com/mailman/listinfo/general
>
>
_______________________________________________
General mailing list
[email protected]
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
