Re: [MarkLogic Dev General] JSON sending in the request body is not being converted into map:map in ML8 using roxy however working fine with ML6

2015-04-28 Thread David Ennis 
HI.

The issue here may be is that json is a first-class citizen in MarkLogic 8
and is happy staying as json.

If all of your code expects a map, then perhaps you can test the type of
the document and if its json, transform it to the map on the fly.

Kind Regards,
David Ennis



Kind Regards,
David Ennis


David Ennis
*Content Engineer*

[image: HintTech]  http://www.hinttech.com/
Mastering the value of content
creative | technology | content

Delftechpark 37i
2628 XJ Delft
The Netherlands
T: +31 88 268 25 00
M: +31 63 091 72 80

[image: http://www.hinttech.com] http://www.hinttech.com
https://twitter.com/HintTech  http://www.facebook.com/HintTech
http://www.linkedin.com/company/HintTech

On 28 April 2015 at 16:14, Indrajeet Verma indrajeet.ve...@gmail.com
wrote:

 Hi,

 Please help us if you have encountered same kind of problem.

 We are upgrading ML6 to ML8 and using roxy as well. The roxy is customized
 based on the customer requirement.


 For one of the REST request /setcache, we are sending below data to
 cache and that is being converted into map (key value pair)

 ML6-

 Request in Body-
 ---

 {USERSESSION:a57e7e98cae8a7857285cd9a66de813f|4/28/2015 9:48:14
 PM:{user:user-profile
 wk-pid=\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39\docTypeUsers/docTypeusername
 type=\string\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernameuserId
 type=\string\lrunne...@wk.com/userIdemail type=\string\
 lrunne...@wk.com/emailuserStatus
 type=\string\Active/userStatusdefaultContentCollectionALL/defaultContentCollectionrole
 type=\string\user/roletrialEndDate type=\string\/activationDate
 type=\string\2013-12-25/activationDatephoneNumber
 type=\string\/sendWelcomeEmail type=\string\/additionalEmail
 type=\string\/filingContent
 type=\string\Allowed/filingContentfirstName
 type=\string\John/firstNamelastName
 type=\string\Doe/lastNameorgCode
 type=\string\WK001/orgCodeorgName
 type=\string\LoadRunner/orgNamesalesTerritory
 type=\string\/password5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2014-07-11T11:05:11.589-05:00/createdcreatedBy
 type=\string\System/createdBymodified2014-07-11T11:05:11.589-05:00/modifiedmodifiedBy
 yun.y...@wolterskluwer.com/modifiedBy/user-profile,
 session:session wk-pid=\a57e7e98cae8a7857285cd9a66de813f|4/28/2015
 9:48:14
 PM\usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernamecreated2015-04-28T17:48:14.661+05:30/createdexpiration2015-04-28T21:48:14.661+05:30/expirationcurrenttime2015-04-28T17:48:14.661+05:30/currenttimevalidfalse/valid/session}}

 Converted into below map using ML6-
 --

 map:map(map:map xmlns:xs=http://www.w3.org/2001/XMLSchema; xmlns:xsi=
 http://www.w3.org/2001/XMLSchema-instance;
 xmlns:map=http://marklogic.com/xdmp/map;map:entry
 key=USERSESSION:97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24
 PMmap:valuemap:mapmap:entry key=usermap:valueuser-profile
 wk-pid=USR-59fb4f7d-de03-1e37-a23c-5c3d71552eaausernameUSR-59fb4f7d-de03-1e37-a23c-5c3d71552eaa/usernamedocTypeUsers/docTypeuserId
 type=stringsjauh...@innodata.com/userIdfirstName
 type=stringshruti/firstNamelastName
 type=stringjauhari/lastNameorgCode
 type=stringFORD-101/orgCodeorgName type=stringFord Motor
 Company/orgNamefilingContent
 type=stringAllowed/filingContentuserStatus
 type=stringActive/userStatustrialEndDate
 type=string2018-12-31/trialEndDaterole
 type=stringuser/roletelephoneNumber 
 type=string212-555-1212/telephoneNumbersalesTerritory
 type=stringNortheast/salesTerritorysendWelcomeEmail
 type=stringFalse/sendWelcomeEmaildefaultContentCollection
 type=stringdefault/defaultContentCollectionadditionalEmail
 type=stringskh...@innodata.com/additionalEmailpassword5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2015-04-22T15:37:32.151+05:30/createdcreatedByUSR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00/createdBymodified2015-04-22T15:37:32.151+05:30/modifiedmodifiedByUSR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00/modifiedBy/user-profile/map:value/map:entrymap:entry
 key=sessionmap:valuesession
 wk-pid=97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24
 PMusernameUSR-59fb4f7d-de03-1e37-a23c-5c3d71552eaa/usernamecreated2015-04-28T17:57:24.34+05:30/createdexpiration2015-04-28T21:57:24.34+05:30/expirationcurrenttime2015-04-28T17:57:24.34+05:30/currenttimevalidfalse/valid/session/map:value/map:entry/map:map/map:value/map:entry/map:map)

 ML8 -
 ---
 Input and converted output is same. i.e. without map:

 {USERSESSION:a57e7e98cae8a7857285cd9a66de813f|4/28/2015 9:48:14
 PM:{user:user-profile
 wk-pid=\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39\docTypeUsers/docTypeusername
 type=\string\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernameuserId
 type=\string\lrunne...@wk.com/userIdemail type=\string\
 lrunne...@wk.com/emailuserStatus
 type=\string\Active/userStatusdefaultContentCollectionALL/defaultContentCollectionrole
 type=\string\user/roletrialEndDate

Re: [MarkLogic Dev General] SQL Query Performance and Tableau Connection

2015-04-28 Thread Brad Rix 
One comment about your specific query.You have a left outer join against 
the patientaddress table.  This can actually be written as an inner join with 
the same results since you are restricting the right side of the outer join 
with the where clause.   You can try your query as an inner join and see if 
that improves your performance.The relational database engine probably 
optimized the query by re-writing it as an inner join.If you did not have 
the a.State='LA', then it would have to remain an outer join.   The left 
outer join might be greatly effecting performance.Since you are also using 
group by and order by, the total amount of results might also effect the 
performance.

select c.Patientid, ClaimNumber, FirstName, LastName, sum (PharmacyTotalCost) 
as PharmacyTotalCost
  from claim c inner join patient p on c.patientid = p.patientid
  left outer join patientaddress a on p.patientid = a.patientid
  where a.[State] = 'LA'
  group by  c.PatientId, ClaimNumber, FirstName, LastName  order by 
SUM(PharmacyTotalCost) desc limit 10

Note:  You can see a reference in mysql as an example about when outer joins 
can be re-written to inner joins:  
https://dev.mysql.com/doc/refman/5.0/en/outer-join-simplification.html

Brad Rix | Technical Lead
O: +1 303 542 2172 | M: +1 303 915 2771
Skype: Brad.Rix | Google Talk: 
bradford@gmail.commailto:bradford@gmail.com

From: 
general-boun...@developer.marklogic.commailto:general-boun...@developer.marklogic.com
 [mailto:general-boun...@developer.marklogic.com] On Behalf Of Sara Mazer
Sent: Monday, April 27, 2015 6:30 PM
To: MarkLogic Developer Discussion; ji...@rdacorp.commailto:ji...@rdacorp.com
Cc: Scott Fowler; Walt Rolle
Subject: Re: [MarkLogic Dev General] SQL Query Performance and Tableau 
Connection

Hi Cynthia,
My name is Sara Mazer, and I am a solutions consultant in the DC area.  I've 
worked a lot with Tableau and MarkLogic and would like to set up some time to 
discuss your evaluation and give you some best practices so that you can get 
the best performance possible.  I was the person who worked with Tableau to add 
MarkLogic as a named connection and supported their certification of MarkLogic, 
and I think I can help you.

I have a too many best practices to go over by email, so I'd like to speak with 
you and walk you through setting up databases for successful analytics through 
WebEx.  Would that be possible?  I have a meeting from 9:30-11:00 but can move 
any other meeting I have tomorrow to suit your schedule, just let me know a 
good time for you.

In general, I plan on:
Recommending the best data model for MarkLogic of this type of data (hint: it 
doesn't look like what would be in an RDBMs)
Showing you a demo of patient and claims data with Tableau and review how it 
was done
Discussing the best practices of working with Tableau (minimize the data coming 
back from MarkLogic by using custom SQL for charts, using MATCH keyword, having 
Tableau do grouping/sorting instead of in SQL)
Reviewing common connection issues between MarkLogic and Tableau and how to 
avoid them (most likely you are corrupting your SQL views:a best practice is to 
have a unique schema database for each MarkLogic database)
Reviewing performance of XQuery/JavaScript vs. SQL in MarkLogic

Best regards,
Sara Mazer
sara dot mazer at marklogic dot com
This e-mail and any accompanying attachments are confidential. The information 
is intended solely for the use of the individual to whom it is addressed. Any 
review, disclosure, copying, distribution, or use of this e-mail communication 
by others is strictly prohibited. If you are not the intended recipient, please 
notify us immediately by returning this message to the sender and delete all 
copies. Thank you for your cooperation.



From: 
general-boun...@developer.marklogic.commailto:general-boun...@developer.marklogic.com
 [mailto:general-boun...@developer.marklogic.com] On Behalf Of Cynthia Jiang
Sent: Monday, April 27, 2015 4:56 PM
To: General@developer.marklogic.commailto:General@developer.marklogic.com
Cc: Walt Rolle
Subject: [MarkLogic Dev General] SQL Query Performance and Tableau Connection

Hello,

We have been testing MarkLogic 8 downloaded from your website for a POC.

Our environment is very straight forward.

A single VM (windows 7 Enterprise - 64bit) with 16GM RAM and 2 Intel (E5-2690 
v2) 3GHz processors.

We have around 4 SQL tables that we modeled in Json format so each row ends up 
becoming a document, and here are some of the stats:

Patient : 8 fields / 20,924 documents
PatientAddress: 5 fields / 36,193 documents
Claim: less than 30 fields / 335,349 documents
ClaimDiagnosis: 2 fields / 8,676 documents

We created element range index for all the fields, and created 4 views that 
each represent a SQL table.

Then we were able to use SQL to query the data and get the result in the query 
console.

But the performance is not nearly as what we expected, we thought it would

[MarkLogic Dev General] JSON sending in the request body is not being converted into map:map in ML8 using roxy however working fine with ML6

2015-04-28 Thread Indrajeet Verma 
Hi,

Please help us if you have encountered same kind of problem.

We are upgrading ML6 to ML8 and using roxy as well. The roxy is customized
based on the customer requirement.


For one of the REST request /setcache, we are sending below data to cache
and that is being converted into map (key value pair)

ML6-

Request in Body-
---

{USERSESSION:a57e7e98cae8a7857285cd9a66de813f|4/28/2015 9:48:14
PM:{user:user-profile
wk-pid=\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39\docTypeUsers/docTypeusername
type=\string\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernameuserId
type=\string\lrunne...@wk.com/userIdemail type=\string\
lrunne...@wk.com/emailuserStatus
type=\string\Active/userStatusdefaultContentCollectionALL/defaultContentCollectionrole
type=\string\user/roletrialEndDate type=\string\/activationDate
type=\string\2013-12-25/activationDatephoneNumber
type=\string\/sendWelcomeEmail type=\string\/additionalEmail
type=\string\/filingContent
type=\string\Allowed/filingContentfirstName
type=\string\John/firstNamelastName
type=\string\Doe/lastNameorgCode
type=\string\WK001/orgCodeorgName
type=\string\LoadRunner/orgNamesalesTerritory
type=\string\/password5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2014-07-11T11:05:11.589-05:00/createdcreatedBy
type=\string\System/createdBymodified2014-07-11T11:05:11.589-05:00/modifiedmodifiedBy
yun.y...@wolterskluwer.com/modifiedBy/user-profile,
session:session wk-pid=\a57e7e98cae8a7857285cd9a66de813f|4/28/2015
9:48:14
PM\usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernamecreated2015-04-28T17:48:14.661+05:30/createdexpiration2015-04-28T21:48:14.661+05:30/expirationcurrenttime2015-04-28T17:48:14.661+05:30/currenttimevalidfalse/valid/session}}

Converted into below map using ML6-
--

map:map(map:map xmlns:xs=http://www.w3.org/2001/XMLSchema; xmlns:xsi=
http://www.w3.org/2001/XMLSchema-instance;
xmlns:map=http://marklogic.com/xdmp/map;map:entry
key=USERSESSION:97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24
PMmap:valuemap:mapmap:entry key=usermap:valueuser-profile
wk-pid=USR-59fb4f7d-de03-1e37-a23c-5c3d71552eaausernameUSR-59fb4f7d-de03-1e37-a23c-5c3d71552eaa/usernamedocTypeUsers/docTypeuserId
type=stringsjauh...@innodata.com/userIdfirstName
type=stringshruti/firstNamelastName
type=stringjauhari/lastNameorgCode
type=stringFORD-101/orgCodeorgName type=stringFord Motor
Company/orgNamefilingContent
type=stringAllowed/filingContentuserStatus
type=stringActive/userStatustrialEndDate
type=string2018-12-31/trialEndDaterole
type=stringuser/roletelephoneNumber
type=string212-555-1212/telephoneNumbersalesTerritory
type=stringNortheast/salesTerritorysendWelcomeEmail
type=stringFalse/sendWelcomeEmaildefaultContentCollection
type=stringdefault/defaultContentCollectionadditionalEmail
type=stringskh...@innodata.com/additionalEmailpassword5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2015-04-22T15:37:32.151+05:30/createdcreatedByUSR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00/createdBymodified2015-04-22T15:37:32.151+05:30/modifiedmodifiedByUSR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00/modifiedBy/user-profile/map:value/map:entrymap:entry
key=sessionmap:valuesession
wk-pid=97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24
PMusernameUSR-59fb4f7d-de03-1e37-a23c-5c3d71552eaa/usernamecreated2015-04-28T17:57:24.34+05:30/createdexpiration2015-04-28T21:57:24.34+05:30/expirationcurrenttime2015-04-28T17:57:24.34+05:30/currenttimevalidfalse/valid/session/map:value/map:entry/map:map/map:value/map:entry/map:map)

ML8 -
---
Input and converted output is same. i.e. without map:

{USERSESSION:a57e7e98cae8a7857285cd9a66de813f|4/28/2015 9:48:14
PM:{user:user-profile
wk-pid=\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39\docTypeUsers/docTypeusername
type=\string\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernameuserId
type=\string\lrunne...@wk.com/userIdemail type=\string\
lrunne...@wk.com/emailuserStatus
type=\string\Active/userStatusdefaultContentCollectionALL/defaultContentCollectionrole
type=\string\user/roletrialEndDate type=\string\/activationDate
type=\string\2013-12-25/activationDatephoneNumber
type=\string\/sendWelcomeEmail type=\string\/additionalEmail
type=\string\/filingContent
type=\string\Allowed/filingContentfirstName
type=\string\John/firstNamelastName
type=\string\Doe/lastNameorgCode
type=\string\WK001/orgCodeorgName
type=\string\LoadRunner/orgNamesalesTerritory
type=\string\/password5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2014-07-11T11:05:11.589-05:00/createdcreatedBy
type=\string\System/createdBymodified2014-07-11T11:05:11.589-05:00/modifiedmodifiedBy
yun.y...@wolterskluwer.com/modifiedBy/user-profile,
session:session wk-pid=\a57e7e98cae8a7857285cd9a66de813f|4/28/2015
9:48:14

Re: [MarkLogic Dev General] JSON sending in the request body is not being converted into map:map in ML8 using roxy however working fine with ML6

2015-04-28 Thread David Lee 
This is probably best addressed by filing an issue on the Roxy github issue 
tracker.

https://github.com/marklogic/roxy

Roxy is a community/developer supported product.

I suspect the issue is due to the (not backwards compatible, but documented) 
change to many JSON related API's in MarkLogic.

The JSON API's have been converted to work directly with the native JSON data 
types in ML8.

All of the internal MarkLogic libraries and apps have been updated as well to 
adjust to this change.   Exactly where in Roxy this is breaking I don't know, 
but the errors suggest is likely due to the ML8 changes in JSON.

Happy to help with any specific questions on the changes to ML APIs.

-David




-
David Lee
Lead Engineer
MarkLogic Corporation
d...@marklogic.com
Phone: +1 812-482-5224
Cell:  +1 812-630-7622
www.marklogic.comhttp://www.marklogic.com/

From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Indrajeet Verma
Sent: Tuesday, April 28, 2015 10:15 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] JSON sending in the request body is not being 
converted into map:map in ML8 using roxy however working fine with ML6

Hi,

Please help us if you have encountered same kind of problem.

We are upgrading ML6 to ML8 and using roxy as well. The roxy is customized 
based on the customer requirement.


For one of the REST request /setcache, we are sending below data to cache and 
that is being converted into map (key value pair)

ML6-

Request in Body-
---

{USERSESSION:a57e7e98cae8a7857285cd9a66de813f|4/28/2015 9:48:14 
PM:{user:user-profile 
wk-pid=\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39\docTypeUsers/docTypeusername
 type=\string\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernameuserId 
type=\string\lrunne...@wk.commailto:lrunne...@wk.com/userIdemail 
type=\string\lrunne...@wk.commailto:lrunne...@wk.com/emailuserStatus 
type=\string\Active/userStatusdefaultContentCollectionALL/defaultContentCollectionrole
 type=\string\user/roletrialEndDate type=\string\/activationDate 
type=\string\2013-12-25/activationDatephoneNumber 
type=\string\/sendWelcomeEmail type=\string\/additionalEmail 
type=\string\/filingContent 
type=\string\Allowed/filingContentfirstName 
type=\string\John/firstNamelastName 
type=\string\Doe/lastNameorgCode type=\string\WK001/orgCodeorgName 
type=\string\LoadRunner/orgNamesalesTerritory 
type=\string\/password5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2014-07-11T11:05:11.589-05:00/createdcreatedBy
 
type=\string\System/createdBymodified2014-07-11T11:05:11.589-05:00/modifiedmodifiedByyun.y...@wolterskluwer.commailto:yun.y...@wolterskluwer.com/modifiedBy/user-profile,
 session:session wk-pid=\a57e7e98cae8a7857285cd9a66de813f|4/28/2015 
9:48:14 
PM\usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernamecreated2015-04-28T17:48:14.661+05:30/createdexpiration2015-04-28T21:48:14.661+05:30/expirationcurrenttime2015-04-28T17:48:14.661+05:30/currenttimevalidfalse/valid/session}}

Converted into below map using ML6-
--

map:map(map:map xmlns:xs=http://www.w3.org/2001/XMLSchema; 
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xmlns:map=http://marklogic.com/xdmp/map;map:entry 
key=USERSESSION:97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24 
PMmap:valuemap:mapmap:entry key=usermap:valueuser-profile 
wk-pid=USR-59fb4f7d-de03-1e37-a23c-5c3d71552eaausernameUSR-59fb4f7d-de03-1e37-a23c-5c3d71552eaa/usernamedocTypeUsers/docTypeuserId
 
type=stringsjauh...@innodata.commailto:sjauh...@innodata.com/userIdfirstName
 type=stringshruti/firstNamelastName 
type=stringjauhari/lastNameorgCode 
type=stringFORD-101/orgCodeorgName type=stringFord Motor 
Company/orgNamefilingContent 
type=stringAllowed/filingContentuserStatus 
type=stringActive/userStatustrialEndDate 
type=string2018-12-31/trialEndDaterole 
type=stringuser/roletelephoneNumber 
type=string212-555-1212/telephoneNumbersalesTerritory 
type=stringNortheast/salesTerritorysendWelcomeEmail 
type=stringFalse/sendWelcomeEmaildefaultContentCollection 
type=stringdefault/defaultContentCollectionadditionalEmail 
type=stringskh...@innodata.commailto:skh...@innodata.com/additionalEmailpassword5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2015-04-22T15:37:32.151+05:30/createdcreatedByUSR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00/createdBymodified2015-04-22T15:37:32.151+05:30/modifiedmodifiedByUSR-2fe944cc-013e-5b5a-0dba-7e98c3ffab00/modifiedBy/user-profile/map:value/map:entrymap:entry
 key=sessionmap:valuesession 
wk-pid=97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24

Re: [MarkLogic Dev General] JSON sending in the request body is not being converted into map:map in ML8 using roxy however working fine with ML6

2015-04-28 Thread Indrajeet Verma 
Thank you David for your explanation. I also have same understanding
however not able to find exact method/module which is not converting into
map.

I identified router.xqy (line 67) and request.xqy where the conversion is
happening and setting map to the request.

Might be good to open a ticket in the github/roxy and see if I get some
resolution.

However I am attaching the router.xqy and request.xqy for your reference.
if possible please take a look and suggest some resolution.

Could you please help me to get the link of the APIs that have modified as
you said in your comments? OR Should we look into release notes?

Thank you for your time and help.

Regards,
Indrajeet

On Tue, Apr 28, 2015 at 8:16 PM, David Lee david@marklogic.com wrote:

  This is probably best addressed by filing an issue on the Roxy github
 issue tracker.



 https://github.com/marklogic/roxy



 Roxy is a community/developer supported product.



 I suspect the issue is due to the (not backwards compatible, but
 documented) change to many JSON related API's in MarkLogic.



 The JSON API's have been converted to work directly with the native JSON
 data types in ML8.



 All of the internal MarkLogic libraries and apps have been updated as well
 to adjust to this change.   Exactly where in Roxy this is breaking I don't
 know, but the errors suggest is likely due to the ML8 changes in JSON.



 Happy to help with any specific questions on the changes to ML APIs.



 -David










 -

 David Lee
 Lead Engineer
 *Mark**Logic* Corporation
 d...@marklogic.com
 Phone: +1 812-482-5224

 Cell:  +1 812-630-7622
 www.marklogic.com



 *From:* general-boun...@developer.marklogic.com [mailto:
 general-boun...@developer.marklogic.com] *On Behalf Of *Indrajeet Verma
 *Sent:* Tuesday, April 28, 2015 10:15 AM
 *To:* MarkLogic Developer Discussion
 *Subject:* [MarkLogic Dev General] JSON sending in the request body is
 not being converted into map:map in ML8 using roxy however working fine
 with ML6



 Hi,



 Please help us if you have encountered same kind of problem.



 We are upgrading ML6 to ML8 and using roxy as well. The roxy is customized
 based on the customer requirement.


 For one of the REST request /setcache, we are sending below data to
 cache and that is being converted into map (key value pair)

 ML6-

 Request in Body-
 ---

 {USERSESSION:a57e7e98cae8a7857285cd9a66de813f|4/28/2015 9:48:14
 PM:{user:user-profile
 wk-pid=\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39\docTypeUsers/docTypeusername
 type=\string\USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernameuserId
 type=\string\lrunne...@wk.com/userIdemail type=\string\
 lrunne...@wk.com/emailuserStatus
 type=\string\Active/userStatusdefaultContentCollectionALL/defaultContentCollectionrole
 type=\string\user/roletrialEndDate type=\string\/activationDate
 type=\string\2013-12-25/activationDatephoneNumber
 type=\string\/sendWelcomeEmail type=\string\/additionalEmail
 type=\string\/filingContent
 type=\string\Allowed/filingContentfirstName
 type=\string\John/firstNamelastName
 type=\string\Doe/lastNameorgCode
 type=\string\WK001/orgCodeorgName
 type=\string\LoadRunner/orgNamesalesTerritory
 type=\string\/password5f4dcc3b5aa765d61d8327deb882cf99/passwordpwpassword/pwcountOfLoggedIn11/countOfLoggedIncreated2014-07-11T11:05:11.589-05:00/createdcreatedBy
 type=\string\System/createdBymodified2014-07-11T11:05:11.589-05:00/modifiedmodifiedBy
 yun.y...@wolterskluwer.com/modifiedBy/user-profile,
 session:session wk-pid=\a57e7e98cae8a7857285cd9a66de813f|4/28/2015
 9:48:14
 PM\usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/usernamecreated2015-04-28T17:48:14.661+05:30/createdexpiration2015-04-28T21:48:14.661+05:30/expirationcurrenttime2015-04-28T17:48:14.661+05:30/currenttimevalidfalse/valid/session}}

 Converted into below map using ML6-
 --

 map:map(map:map xmlns:xs=http://www.w3.org/2001/XMLSchema; xmlns:xsi=
 http://www.w3.org/2001/XMLSchema-instance;

 xmlns:map=http://marklogic.com/xdmp/map;map:entry
 key=USERSESSION:97f7d8ec0b5bba928cb025bd1d4d28d4|4/28/2015 9:57:24
 PMmap:valuemap:mapmap:entry key=usermap:valueuser-profile
 wk-pid=USR-59fb4f7d-de03-1e37-a23c-5c3d71552eaausernameUSR-59fb4f7d-de03-1e37-a23c-5c3d71552eaa/usernamedocTypeUsers/docTypeuserId
 type=stringsjauh...@innodata.com/userIdfirstName
 type=stringshruti/firstNamelastName
 type=stringjauhari/lastNameorgCode
 type=stringFORD-101/orgCodeorgName type=stringFord Motor
 Company/orgNamefilingContent
 type=stringAllowed/filingContentuserStatus
 type=stringActive/userStatustrialEndDate
 type=string2018-12-31/trialEndDaterole
 type=stringuser/roletelephoneNumber 
 type=string212-555-1212/telephoneNumbersalesTerritory
 type=stringNortheast/salesTerritorysendWelcomeEmail
 type=stringFalse/sendWelcomeEmaildefaultContentCollection

Re: [MarkLogic Dev General] JSON sending in the request body is not being converted into map:map in ML8 using roxy however working fine with ML6

2015-04-28 Thread Christopher Hamlin 
On Tue, Apr 28, 2015 at 11:42 AM, Indrajeet Verma
indrajeet.ve...@gmail.com wrote:
 ...

 Could you please help me to get the link of the APIs that have modified as
 you said in your comments? OR Should we look into release notes?



Since JSON is now native, there is much change in v8.  The relevant
section of the release notes has a lot of information:

https://docs.marklogic.com/guide/relnotes/chap4#id_92312

/ch
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] SQL Query Performance and Tableau Connection

2015-04-28 Thread Walt Rolle 
Sorry for the duplicate here, I had to subscribe first...

From: Walt Rolle
Sent: Tuesday, April 28, 2015 11:24 AM
To: 'Brad Rix'; MarkLogic Developer Discussion; Cynthia Jiang
Cc: Scott Fowler; 'sara.ma...@marklogic.com'
Subject: RE: [MarkLogic Dev General] SQL Query Performance and Tableau 
Connection

Hi Brad,
Thanks for the tip. Also it looks like it's going to make sense for us just to 
store the primary address at the patient level anyway (so no joins).

I wanted to mention some conclusions we arrived at after talking to Sara.


*If we want to use SQL at all, then we can't have repeating rows in our 
document structure, or at least need to name the first one in the collection 
something different to separate it.

*Getting SQL to work in our situation is about supporting ODBC 
connectivity options such as ODBC to Tableau. We can simplify the data model by 
de-normalizing a bit, such as sticking to just primary address and active 
eligibility. After that, we supply Tableau with our own specific SQL statement 
to limit the columns and number of tables. Group by / order by / functions 
are best left to Tableau afterwards, so our source queries are simple.

*MarkLogic lends itself better to document search, which would use 
things like XQuery or Javascript to screen the documents and return patient 
documents that match the criteria. Because we broke apart our model to support 
SQL, we take a performance hit here in having to do joins or store data in 
duplicate locations. The other option would be to keep the large hierarchical 
documents and ETL a subset off to a data mart, if the customer wants to do 
relational things against it.

Because the customer has a lot of existing code and internal capability around 
the traditional SQL Server stack, we're thinking of having our model driven 
towards supporting SQL and NoSQL techniques simultaneously - at least as the 
first step of the roadmap. In the future, it would not be much effort to 
generate new documents that are complete hierarchies of patient history. To do 
this we need to separate objects by grain.


1.  Patient (with primary address and active eligibility)

2.  Claim Header (consider storing diagnosis codes as repeating rows in one 
field, it's essentially factless)

3.  Claim Revenue (this is line item detail with dollar amounts)

4.  Claim Procedure (another type of line item detail with dollar amounts)

5.  Care Provider (specialty may need to be a field level collection like 
diagnosis)

6.  Facility

We'll have to revisit these decisions when we get more time to spend on looking 
at the analytics use cases to make sure we always choose the right tool for the 
job. As far as deep amounts of storage for terabytes of data, MarkLogic seems 
to be a good landing place for the variety of incoming claims files, either 
before or after we standardize them or both.

Thanks,
Walt

Walter Rolle, PMP
Project Manager
ro...@rdacorp.commailto:ro...@rdacorp.com
o: (770) 884.4512 | m: (678) 722.1549 | f: (770) 884.4512
2475 NorthWinds Parkway, Suite 200
Alpharetta, GA 30009-4807
[in]http://www.linkedin.com/in/wrolle[cid:image004.png@01CDD2D0.8A8D29B0]http://twitter.com/rdadevteam[cid:image003.jpg@01CDD2D0.8A8D29B0]http://www.facebook.com/pages/RDA-Corporation/9054123
www.rdacorp.comhttp://www.rdacorp.com/

From: Brad Rix [mailto:brad@flatironssolutions.com]
Sent: Tuesday, April 28, 2015 10:45 AM
To: MarkLogic Developer Discussion; Cynthia Jiang
Cc: Scott Fowler; Walt Rolle
Subject: RE: [MarkLogic Dev General] SQL Query Performance and Tableau 
Connection

One comment about your specific query.You have a left outer join against 
the patientaddress table.  This can actually be written as an inner join with 
the same results since you are restricting the right side of the outer join 
with the where clause.   You can try your query as an inner join and see if 
that improves your performance.The relational database engine probably 
optimized the query by re-writing it as an inner join.If you did not have 
the a.State='LA', then it would have to remain an outer join.   The left 
outer join might be greatly effecting performance.Since you are also using 
group by and order by, the total amount of results might also effect the 
performance.

select c.Patientid, ClaimNumber, FirstName, LastName, sum (PharmacyTotalCost) 
as PharmacyTotalCost
  from claim c inner join patient p on c.patientid = p.patientid
  left outer join patientaddress a on p.patientid = a.patientid
  where a.[State] = 'LA'
  group by  c.PatientId, ClaimNumber, FirstName, LastName  order by 
SUM(PharmacyTotalCost) desc limit 10

Note:  You can see a reference in mysql as an example about when outer joins 
can be re-written to inner joins:  
https://dev.mysql.com/doc/refman/5.0/en/outer-join-simplification.html

Brad Rix | Technical Lead
O: +1 303 542 2172 | M: +1 303 915 2771
Skype: Brad.Rix | Google Talk:

Re: [MarkLogic Dev General] Error in getting map keys in ML8

2015-04-28 Thread Indrajeet Verma 
Thank you Bryan for your help!

Regards,
Indy

On Wed, Apr 29, 2015 at 12:38 AM, Joe Bryan joe.br...@marklogic.com wrote:

  Hi Indrajeect,

  Your $params variable is an element (the serialization of a map), not a
 map. You can use the map constructor function to convert the XML
 serialization of a map to map:

  return map:keys(map:map($params))

  Thanks.

  -jb

   From: Indrajeet Verma indrajeet.ve...@gmail.com
 Reply-To: MarkLogic Developer Discussion general@developer.marklogic.com
 Date: Tuesday, April 28, 2015 at 3:05 PM
 To: MarkLogic Developer Discussion general@developer.marklogic.com
 Subject: [MarkLogic Dev General] Error in getting map keys in ML8

   Hi,

  Could somebody help me what I am missing in below query. I am using this
 code in ML8

  let $params := map:map xmlns:map=http://marklogic.com/xdmp/map;
 xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
 xmlns:xs=http://www.w3.org/2001/XMLSchema;
 map:entry key=USERSESSION:483d4d2f1a54a7f0eb42f6f1dfa6518c|4/29/2015
 3:41:5 AM
 map:value
 map:map
 map:entry key=user
 map:value
 user-profile
 wk-pid=USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39
 docTypeUsers/docType
 username type=string

 USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/username

 /user-profile
 /map:value
 /map:entry
 map:entry key=session
 map:value
 session
 wk-pid=483d4d2f1a54a7f0eb42f6f1dfa6518c|4/29/2015 3:41:5 AM

 usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/username

 created2015-04-28T23:41:05.172+05:30/created
 /session
 /map:value
 /map:entry
 /map:map
 /map:value
 /map:entry
 /map:map
 return map:keys($params)

  Error that I am seeing,

   [1.0-ml] XDMP-ARGTYPE: (err:XPTY0004) map:keys(map:map xmlns:xs=
 http://www.w3.org/2001/XMLSchema; xmlns:xsi=
 http://www.w3.org/2001/XMLSchema-instance; xmlns:map=
 http://marklogic.com/xdmp/map;map:entry
 key=USERSESSION:483d4d2f1a54a7f0eb42f6f1dfa6518c|4/2.../map:map) -- arg1
 is not of type map:map


 ___
 General mailing list
 General@developer.marklogic.com
 Manage your subscription at:
 http://developer.marklogic.com/mailman/listinfo/general


___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] SQL Query Performance and Tableau

2015-04-28 Thread Freddy Holwerda 
: 
 http://developer.marklogic.com/pipermail/general/attachments/20150428/c22d795f/attachment-0001.html
  
 
 --
 
 Message: 3
 Date: Tue, 28 Apr 2015 11:22:56 +0530
 From: Shashidhar Rao raoshashidhar...@gmail.com
 Subject: [MarkLogic Dev General] Is data also store in forest
 To: general@developer.marklogic.com
 Message-ID:
   CAFY8jidOTxR7PnKAPQ8=u+powg2n6qjs_4rdo6945xonegd...@mail.gmail.com
 Content-Type: text/plain; charset=utf-8
 
 Hi,
 
 We are running out of disk space due to the indexing .
 
 Could some answer a basic question.
 
 I have loaded data to Marklogic server from /home/data/data1 . Now the
 loading is finished and re-indexing and merging started.
 
 Does Marklogic server still require data to be in /home/data/data1 or can
 this data be deleted .
 
 I have always assumed that even after loading the data from the path above
 Marklogic still requires data to be in this folder.
 
 Basically I want the data to be deleted once the loading is complete as
 space is running out due to heavy re-indexing.
 
 Thanks
 -- next part --
 An HTML attachment was scrubbed...
 URL: 
 http://developer.marklogic.com/pipermail/general/attachments/20150428/67f143ed/attachment.html
  
 
 --
 
 ___
 General mailing list
 General@developer.marklogic.com
 Manage your subscription at: 
 http://developer.marklogic.com/mailman/listinfo/general
 
 
 End of General Digest, Vol 130, Issue 106
 *
  ___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

[MarkLogic Dev General] Error in getting map keys in ML8

2015-04-28 Thread Indrajeet Verma 
Hi,

Could somebody help me what I am missing in below query. I am using this
code in ML8

let $params := map:map xmlns:map=http://marklogic.com/xdmp/map;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xmlns:xs=http://www.w3.org/2001/XMLSchema;
map:entry key=USERSESSION:483d4d2f1a54a7f0eb42f6f1dfa6518c|4/29/2015
3:41:5 AM
map:value
map:map
map:entry key=user
map:value
user-profile
wk-pid=USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39
docTypeUsers/docType
username type=string

USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/username

/user-profile
/map:value
/map:entry
map:entry key=session
map:value
session
wk-pid=483d4d2f1a54a7f0eb42f6f1dfa6518c|4/29/2015 3:41:5 AM

usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/username

created2015-04-28T23:41:05.172+05:30/created
/session
/map:value
/map:entry
/map:map
/map:value
/map:entry
/map:map
return map:keys($params)

Error that I am seeing,

[1.0-ml] XDMP-ARGTYPE: (err:XPTY0004) map:keys(map:map xmlns:xs=
http://www.w3.org/2001/XMLSchema; xmlns:xsi=
http://www.w3.org/2001/XMLSchema-instance; xmlns:map=
http://marklogic.com/xdmp/map;map:entry
key=USERSESSION:483d4d2f1a54a7f0eb42f6f1dfa6518c|4/2.../map:map) -- arg1
is not of type map:map
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Error in getting map keys in ML8

2015-04-28 Thread Joe Bryan 
Hi Indrajeect,

Your $params variable is an element (the serialization of a map), not a map. 
You can use the map constructor function to convert the XML serialization of a 
map to map:

return map:keys(map:map($params))

Thanks.

-jb

From: Indrajeet Verma 
indrajeet.ve...@gmail.commailto:indrajeet.ve...@gmail.com
Reply-To: MarkLogic Developer Discussion 
general@developer.marklogic.commailto:general@developer.marklogic.com
Date: Tuesday, April 28, 2015 at 3:05 PM
To: MarkLogic Developer Discussion 
general@developer.marklogic.commailto:general@developer.marklogic.com
Subject: [MarkLogic Dev General] Error in getting map keys in ML8

Hi,

Could somebody help me what I am missing in below query. I am using this code 
in ML8

let $params := map:map xmlns:map=http://marklogic.com/xdmp/map;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xmlns:xs=http://www.w3.org/2001/XMLSchema;
map:entry key=USERSESSION:483d4d2f1a54a7f0eb42f6f1dfa6518c|4/29/2015 
3:41:5 AM
map:value
map:map
map:entry key=user
map:value
user-profile 
wk-pid=USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39
docTypeUsers/docType
username type=string

USR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/username
/user-profile
/map:value
/map:entry
map:entry key=session
map:value
session 
wk-pid=483d4d2f1a54a7f0eb42f6f1dfa6518c|4/29/2015 3:41:5 AM

usernameUSR-b740cbc5-5e27-8b1c-d97a-1329625f6b39/username
created2015-04-28T23:41:05.172+05:30/created
/session
/map:value
/map:entry
/map:map
/map:value
/map:entry
/map:map
return map:keys($params)

Error that I am seeing,

[1.0-ml] XDMP-ARGTYPE: (err:XPTY0004) map:keys(map:map 
xmlns:xs=http://www.w3.org/2001/XMLSchema; 
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; 
xmlns:map=http://marklogic.com/xdmp/map;map:entry 
key=USERSESSION:483d4d2f1a54a7f0eb42f6f1dfa6518c|4/2.../map:map) -- arg1 is 
not of type map:map

___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Is data also store in forest

2015-04-28 Thread Shashidhar Rao 
Thanks all for the info.

On Tue, Apr 28, 2015 at 11:22 AM, Shashidhar Rao raoshashidhar...@gmail.com
 wrote:

 Hi,

 We are running out of disk space due to the indexing .

 Could some answer a basic question.

 I have loaded data to Marklogic server from /home/data/data1 . Now the
 loading is finished and re-indexing and merging started.

 Does Marklogic server still require data to be in /home/data/data1 or can
 this data be deleted .

 I have always assumed that even after loading the data from the path above
 Marklogic still requires data to be in this folder.

 Basically I want the data to be deleted once the loading is complete as
 space is running out due to heavy re-indexing.

 Thanks



___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Is data also store in forest

2015-04-28 Thread Rajamani.Marimuthu 
Hi Shashi,

Yes, You can delete the data once the ML ingestion s done. And always configure 
the great disk space for the forest more than the actual content size 
recommended by ML .

By
Raja 

From: general-boun...@developer.marklogic.com 
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Shashidhar Rao
Sent: Tuesday, April 28, 2015 11:23 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] Is data also store in forest

Hi,
We are running out of disk space due to the indexing .
Could some answer a basic question.
I have loaded data to Marklogic server from /home/data/data1 . Now the loading 
is finished and re-indexing and merging started.

Does Marklogic server still require data to be in /home/data/data1 or can this 
data be deleted .
I have always assumed that even after loading the data from the path above 
Marklogic still requires data to be in this folder.
Basically I want the data to be deleted once the loading is complete as space 
is running out due to heavy re-indexing.
Thanks

This e-mail and any files transmitted with it are for the sole use of the 
intended recipient(s) and may contain confidential and privileged information. 
If you are not the intended recipient(s), please reply to the sender and 
destroy all copies of the original message. Any unauthorized review, use, 
disclosure, dissemination, forwarding, printing or copying of this email, 
and/or any action taken in reliance on the contents of this e-mail is strictly 
prohibited and may be unlawful. Where permitted by applicable law, this e-mail 
and other e-mail communications sent to and from Cognizant e-mail addresses may 
be monitored.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general

Re: [MarkLogic Dev General] Is data also store in forest

2015-04-28 Thread Geert Josten 
It is wise to keep the free space twice or maybe even three times as large as 
the total forest sizes as a general rule.

All data, including binary is normally inserted into the forest itself, though 
large binaries are stored outside the search scope, but within MarkLogic data 
folder. The only exception to this is external binaries, but those are loaded 
in a separate way..

Kind regards,
Geert

From: 
rajamani.marimu...@cognizant.commailto:rajamani.marimu...@cognizant.com 
rajamani.marimu...@cognizant.commailto:rajamani.marimu...@cognizant.com
Reply-To: MarkLogic Developer Discussion 
general@developer.marklogic.commailto:general@developer.marklogic.com
Date: Tuesday, April 28, 2015 at 8:58 AM
To: general@developer.marklogic.commailto:general@developer.marklogic.com 
general@developer.marklogic.commailto:general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Is data also store in forest

Hi Shashi,

Yes, You can delete the data once the ML ingestion s done. And always configure 
the great disk space for the forest more than the actual content size 
recommended by ML .

By
Raja 

From: 
general-boun...@developer.marklogic.commailto:general-boun...@developer.marklogic.com
 [mailto:general-boun...@developer.marklogic.com] On Behalf Of Shashidhar Rao
Sent: Tuesday, April 28, 2015 11:23 AM
To: general@developer.marklogic.commailto:general@developer.marklogic.com
Subject: [MarkLogic Dev General] Is data also store in forest

Hi,
We are running out of disk space due to the indexing .
Could some answer a basic question.
I have loaded data to Marklogic server from /home/data/data1 . Now the loading 
is finished and re-indexing and merging started.

Does Marklogic server still require data to be in /home/data/data1 or can this 
data be deleted .
I have always assumed that even after loading the data from the path above 
Marklogic still requires data to be in this folder.
Basically I want the data to be deleted once the loading is complete as space 
is running out due to heavy re-indexing.
Thanks

This e-mail and any files transmitted with it are for the sole use of the 
intended recipient(s) and may contain confidential and privileged information. 
If you are not the intended recipient(s), please reply to the sender and 
destroy all copies of the original message. Any unauthorized review, use, 
disclosure, dissemination, forwarding, printing or copying of this email, 
and/or any action taken in reliance on the contents of this e-mail is strictly 
prohibited and may be unlawful. Where permitted by applicable law, this e-mail 
and other e-mail communications sent to and from Cognizant e-mail addresses may 
be monitored.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at: 
http://developer.marklogic.com/mailman/listinfo/general