Re: [MarkLogic Dev General] collection function searching
Hi, Paul:
Syntactically, the XPath expressions
* retrieve a sequence of documents from the database
* extract a sequence of nodes from the sequence of documents (in case 2)
* filter to produce a final sequence by applying the predicate to each item
The engine tries to optimize XPath expressions by executing as much as of
the expression as possible as a query against indexes. Not all expressions are
possible to optimize as a query. The optimizer may also miss some cases that
are possible to optimize.
The best practice is to use explicit queries instead of XPath expressions when
retrieving documents from the database. That way, the use of indexes is
unambiguous. In addition, you have access to index mechanisms (such as fields)
that aren't available in predicates.
To put it the other way, the best practice is to use XPath expressions only to
traverse nodes after they have been retrieved from the database.
In the particular case, the equivalent query for XPath expression 2 would
resemble the following:
let $ait := cts:search((), cts:and-query((
cts:collection-query($mycol),
cts:element-query(xs:QName("aaa"),
cts:element-value-query(xs:QName("myelem"), "myval"))
)
)))
While the query is more verbose, it declares a carefully considered use of the
indexes. That's a good thing for scalability and maintainability with most
production databases.
For more information, see:
http://docs.marklogic.com/cts:collection-query
http://docs.marklogic.com/cts:element-query
http://docs.marklogic.com/cts:element-value-query
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Paul M <pjm...@yahoo.com>
Sent: Tuesday, May 8, 2018 11:47:35 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] collection function searching
I have the following three queries I am comparing
declare variable $my:col as xs:string...
(:
let $ait := collection()[.//myelem="myval"]
:)
(:
let $ait := collection($mycol)/aaa[.//myelem="myval"]
:)
let $ait := collection($mycol)[.//myelem=" myval"]
return $ait//someelem
There may be 10mil total documents in the repository.
There may be 1mil documents that have root element of aaa.
Note:myelem can/should only be in documents with root element of aaa
There may be 3 mil total documents that are in mycol
There may be at most 100k documents that have root element of aaa which are in
mycol
The following are the query-trace for the above three statement:
1st one: roughly 3000 fragments to filter. Seems reasonable aaa documents that
have myelem = myval
2nd statement: roughly 1000 fragments to filter. Seems reasonable - narrowing
the search
Last statement: roughly 8 mil fragments to filter. Not certain why this occurs.
Any explanation to shed some light?
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster
Hi, Pooja: Glad to hear you found the problem. Thanks for following up. Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Pooja Rajput <pooja.raj...@healthbi.com> Sent: Tuesday, May 1, 2018 10:32:45 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi Erik, Issue got resolved now. Below property in app server was false: "Rewrite resolves globally", after making it true it worked fine. Thanks, Pooja -Original Message- From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> On Behalf Of Erik Hennum Sent: Tuesday, May 1, 2018 10:24 AM To: MarkLogic Developer Discussion <general@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi, Pooja: It's strange that one REST API appserver is working but another one isn't. How did you create the faulty appserver? Did you use the usual request: http://docs.marklogic.com/REST/POST/v1/rest-apis Is it possible the configuration of the appserver was changed after creation? Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Pooja Rajput <pooja.raj...@healthbi.com> Sent: Tuesday, May 1, 2018 9:45:53 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi, For the port we are trying to connect we can see the rest api on this link: http://host:8002/LATEST/rest-apis Also, we have another database on different port on same cluster which works well. This one is newly created. Hence doesn't seems like it is different api version. Thanks Pooja The information contained in this transmission may contain privileged and confidential information, including patient information protected by federal and state privacy laws. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution, or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy the original message. -Original Message- From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> On Behalf Of Erik Hennum Sent: Tuesday, May 1, 2018 9:01 AM To: MarkLogic Developer Discussion <general@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi, Pooja: >From the error message, I'm guessing the application is calling >ExtensionLibrariesManager.list() That method doesn't take any parameters. So, the only input from the application consists of the connection parameters. Is it possible that the application is connecting to a port that hasn't been created as a Client REST API server? (Port 8002 is the builtin Management REST API server.) Alternatively, is it possible there is a mismatch between the client API version and the server version? Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Pooja Rajput <pooja.raj...@healthbi.com> Sent: Tuesday, May 1, 2018 12:03 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi All, We are trying to set up new ML database on clustered ML server. Due to some issue we are unable to run extensions, it is giving below error. I can see REST-api for that server is available http://host:8002/LATEST/rest-apis What is it that is missing? Any pointers are appreciated. Caused by: com.marklogic.client.ResourceNotFoundException: Local message: Could not read resource at /ext. Server Message: Server (not a REST instance?) did not respond with an expected REST Error message. at com.marklogic.client.impl.JerseyServices.checkStatus(JerseyServices.java:4297) at com.marklogic.client.impl.JerseyServices.getResource(JerseyServices.java:2985) at com.marklogic.client.impl.ExtensionLibrariesManagerImpl.list(ExtensionLibrariesManagerImpl.java:68) at com.marklogic.client.impl.ExtensionLibrariesManagerImpl.list(ExtensionLibrariesManagerImpl.java:62) Thanks ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mail
Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster
Hi, Pooja: It's strange that one REST API appserver is working but another one isn't. How did you create the faulty appserver? Did you use the usual request: http://docs.marklogic.com/REST/POST/v1/rest-apis Is it possible the configuration of the appserver was changed after creation? Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Pooja Rajput <pooja.raj...@healthbi.com> Sent: Tuesday, May 1, 2018 9:45:53 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi, For the port we are trying to connect we can see the rest api on this link: http://host:8002/LATEST/rest-apis Also, we have another database on different port on same cluster which works well. This one is newly created. Hence doesn't seems like it is different api version. Thanks Pooja The information contained in this transmission may contain privileged and confidential information, including patient information protected by federal and state privacy laws. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution, or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy the original message. -Original Message- From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> On Behalf Of Erik Hennum Sent: Tuesday, May 1, 2018 9:01 AM To: MarkLogic Developer Discussion <general@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi, Pooja: >From the error message, I'm guessing the application is calling >ExtensionLibrariesManager.list() That method doesn't take any parameters. So, the only input from the application consists of the connection parameters. Is it possible that the application is connecting to a port that hasn't been created as a Client REST API server? (Port 8002 is the builtin Management REST API server.) Alternatively, is it possible there is a mismatch between the client API version and the server version? Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Pooja Rajput <pooja.raj...@healthbi.com> Sent: Tuesday, May 1, 2018 12:03 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi All, We are trying to set up new ML database on clustered ML server. Due to some issue we are unable to run extensions, it is giving below error. I can see REST-api for that server is available http://host:8002/LATEST/rest-apis What is it that is missing? Any pointers are appreciated. Caused by: com.marklogic.client.ResourceNotFoundException: Local message: Could not read resource at /ext. Server Message: Server (not a REST instance?) did not respond with an expected REST Error message. at com.marklogic.client.impl.JerseyServices.checkStatus(JerseyServices.java:4297) at com.marklogic.client.impl.JerseyServices.getResource(JerseyServices.java:2985) at com.marklogic.client.impl.ExtensionLibrariesManagerImpl.list(ExtensionLibrariesManagerImpl.java:68) at com.marklogic.client.impl.ExtensionLibrariesManagerImpl.list(ExtensionLibrariesManagerImpl.java:62) Thanks ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Extension installation not working on marklogic cluster
Hi, Pooja: >From the error message, I'm guessing the application is calling >ExtensionLibrariesManager.list() That method doesn't take any parameters. So, the only input from the application consists of the connection parameters. Is it possible that the application is connecting to a port that hasn't been created as a Client REST API server? (Port 8002 is the builtin Management REST API server.) Alternatively, is it possible there is a mismatch between the client API version and the server version? Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Pooja Rajput <pooja.raj...@healthbi.com> Sent: Tuesday, May 1, 2018 12:03 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Extension installation not working on marklogic cluster Hi All, We are trying to set up new ML database on clustered ML server. Due to some issue we are unable to run extensions, it is giving below error. I can see REST-api for that server is available http://host:8002/LATEST/rest-apis What is it that is missing? Any pointers are appreciated. Caused by: com.marklogic.client.ResourceNotFoundException: Local message: Could not read resource at /ext. Server Message: Server (not a REST instance?) did not respond with an expected REST Error message. at com.marklogic.client.impl.JerseyServices.checkStatus(JerseyServices.java:4297) at com.marklogic.client.impl.JerseyServices.getResource(JerseyServices.java:2985) at com.marklogic.client.impl.ExtensionLibrariesManagerImpl.list(ExtensionLibrariesManagerImpl.java:68) at com.marklogic.client.impl.ExtensionLibrariesManagerImpl.list(ExtensionLibrariesManagerImpl.java:62) Thanks ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Loading a text doc into the database
Hi, Simón: Does it help if you specify a contentType property of "text/html; charset=windows-1252" on the document descriptor when passing in the document? Hoping that's useful, Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Simón Carbajal <simoncarba...@gmail.com> Sent: Friday, April 27, 2018 9:58:10 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Loading a text doc into the database Hi, I'm trying to upload a "text/html" document with the Node.js client API but the document is encoded as "windows 1252" and not utf-8 as required, this is causing the following error: "XDMP-DOCUTF8SEQ: Invalid UTF-8 escape sequence at line 39 -- document is not UTF-8 encoded" Is there a way to solve this with Marklogic? or I need to do the conversion myself in Nodejs? Thanks in advance! Simon ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Field Word Queries using Search API
Hi, Alan:
Please work with your support representative to create a ticket to investigate
the issue.
You might experiment with a custom constraint to see whether that comes closer
to what you need.
Alternatively, you might find that cts:parse() provides a faster solution than
search:parse() for parsing query text.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Alan Darnell
<alan.darn...@utoronto.ca>
Sent: Tuesday, February 27, 2018 3:54:17 PM
To: General Developer Discussion
Subject: [MarkLogic Dev General] Field Word Queries using Search API
We’ve noticed that when using a Field constraint with the Search API, that
search strings such as:
KW:(war peace)
are parsed in such a way that the two words are treated as a phrase. The
resulting query looks like this:
(cts:search(/book, cts:field-word-query("keywords", "war peace",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1), ("score-logtfidf",cts:score-order("descending")), 1))[1 to 10]
This is different than how a none-constrained search is treated, where the two
words are passed to two separate cts:word-query functions, which in turn are
AND’d together.
(cts:search(/book, cts:and-query((cts:word-query("war",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1), cts:word-query("peace",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1)), ()), ("score-logtfidf",cts:score-order("descending")), 1))[1 to 10]
It is also not the same way that non-Field based constraints are handled. For
example, we have an element constraint (SE) defined on the element
“collection-meta” that produces an AND’d query like this
(cts:search(/book, cts:element-query(fn:QName("","collection-meta"),
cts:and-query((cts:word-query("war",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1), cts:word-query("peace",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1)), ()), ()), ("score-logtfidf",cts:score-order("descending")), 1))[1 to 10]
in response to this input SE:(war peace)
What we need is a way to produce the following:
(cts:search(/book, cts:and-query((cts:field-word-query("keywords", "war",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1), cts:field-word-query("keywords", "peace",
("case-insensitive","diacritic-insensitive","punctuation-insensitive","wildcarded","lang=en"),
1)), ()), ("score-logtfidf",cts:score-order("descending")), 1))[1 to 10]
We can do this by passing this string to search:
KW:war AND KW:peace
but that’s not practical from a user perspective. We’d also like to avoid
either editing the Search query module code or having to parse out the user
entered string and tokenize for words and prefix each of these with the
constraint name. The first doesn’t seem wise and the second seems too much
like a hack.
Question:
Is there a way to override the default behaviour of Field based constraints in
the Search API (using options) so that words passed to the constraint are
parsed as individual words and used to construct an AND query in the same way
non-Field based constraints and unconstrained searches operate?
Thanks for any help.
Alan
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Rewriter - match-path - regex sub-matches not set
Hi, Florent: You might need to capture the matches within the same rule. Does it work to use add-query-param or set-var? as in: http://docs.marklogic.com/guide/app-dev/XMLrewriter#id_80469 Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Florent Georges <li...@fgeorges.org> Sent: Monday, December 11, 2017 10:11:19 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Rewriter - match-path - regex sub-matches not set Hi, I am using a declarative XML rewriter. I have a problem with a rule, the matches from the regex in match-path seem to be empty: 0: $0, 1: $1, 2: $2 /web/$1/$2 And indeed, in the logs, I get the following: 2017-12-11 18:22:13.681 Info: [Event:id=debug] 0: , 1: , 2: I feel like I am missing the obvious here. Any idea? Regards, -- Florent Georges H2O Consulting http://h2o.consulting/ ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Best Approach to Manage "Flags" That Might Change Within a Single Transaction
Hi, Eliot: Have you considered a server field -- where any code that changes the status also updates the server field and the iterator checks the server field? The server fields are local to the host, so there's no concern about a separate iterator running on a different host. If multiple iterators run on the same host, each would need to distinguish its status by an id, which the iterator could generate from a random id when it starts. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Eliot Kimber <ekim...@contrext.com> Sent: Thursday, December 7, 2017 7:48:44 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Best Approach to Manage "Flags" That Might Change Within a Single Transaction In the context of my remote processing management system, where my client server is sending many tasks to a set of remote servers through a set of spawned tasks running in parallel, I need to be able to pause the client so that it stops sending new tasks to the remote servers. So far I've been using a single document stored in ML as my mechanism for indicating that a job is in progress and capturing the job details (job ID, start time, servers in use, etc.). This works fine because it was only updated at the start and end of the job. But for the pause/resume use case I need to have a flag that indicates that the job is paused and have other processes (e.g., my task-submission code) immediately respond to a change. For example, if I'm looping over 100 tasks to load up a remote task queue and the job is paused, I want that loop to end immediately. So basically, in this loop, for every iteration, check the "is paused" status, which requires reading the job doc to see if a @paused attribute is present (the @paused attribute captures the time the pause was requested and serves as the "is paused" flag). However, because the loop is a single transaction, it will see the same version of the job doc for every iteration, even if it's changed. I tried using xdmp:eval() to read the job doc but that didn't seem to change the behavior. E.g., doing this in query console: return (er:is-job-paused(), er:pause-job(), er:is-job-paused()) Results in (false, false) So this isn't going to work. So my question: what's the best way to manage this kind of dynamic flag in ML? I could use file system files instead of docs in the database, which would avoid the ML transaction behavior but that seems a little hackier than I'd like. What I'd really like is some kind of "shared memory" mechanism where I can set and reset variables at will across different modules running in parallel but I haven't seen anything like that in my study of the ML API. Is there such a mechanism that I've missed? Or am I just thinking about the problem the wrong way? Thanks, Eliot -- Eliot Kimber http://contrext.com ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] How to query metadata
Hi, Florent:
Document metadata values are not indexed by default. First, define a field for
the metadata value that you want to search:
http://docs.marklogic.com/guide/search-dev/query-options#id_50523
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Florent Georges
<li...@fgeorges.org>
Sent: Monday, December 4, 2017 9:45:11 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] How to query metadata
Hi,
I have documents with metadata (the metadata values you can retrieve with
xdmp:document-get-metadata()).
I would like to search documents with a metadata with a specific value, but I
cannot find any CTS query function for that purpose. Something like:
cts:search(/, cts:metadata('foo', 'bar'))
Have I missed something?
Regards,
--
Florent Georges
H2O Consulting
http://h2o.consulting/
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Regarding limiting no of co-occurrences returned.
Hi, Shabana: If you are on MarkLogic 9, you might consider using TDE to project the rows you want from the documents. You can then use the Optic API to retrieve rows from documents qualified by a cts.query(). Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Christopher Hamlin <cbham...@gmail.com> Sent: Tuesday, November 28, 2017 8:05:14 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Regarding limiting no of co-occurrences returned. You'll get tuples for the cross product over the document (that is, the co-occurrences within the whole document). So, is it the case that you have more than one occurrence of the two values you are returning, in the same document? If so, sometimes you can do some work by using the 'proximity' and 'ordered' options. The query you pass restricts the fragments. Unless you have fragmentation, this equates to co-occurrences across all documents matching the query. /ch On Tue, Nov 28, 2017 at 7:53 AM, shabana khan <shabana.0...@gmail.com> wrote: > Hi All, > > I am using cts:value-co-occurrences() for fetching pair of values for data > set in my project.I have a query too for restricting resultant pair only > from specific section. > But the problem I am facing is that duplicate results are being returned for > each occurrence of values. > Basically, I am looking for a way by which each pair of value should occur > only once in result. I looked at the available options and tried using > "limit=N" but that seems to have no effect on this. > Filtering at later stage is not possible since it slows down the query if I > go for selecting unique records based on some value. > > Is there some way to achieve it. If anyone has faced this problem please do > share what you did exactly to get unique pair of values. > Any heads-up will be appreciated. > > Thanks and Regards, > Shabana Khan > LinkedIn > > > > > ___ > General mailing list > General@developer.marklogic.com > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Data Masking on Import/Export
Hi, Ric: I'm not aware of any builtin feature to import masked data. At a guess, you might implement a dictionary approach, providing a transform to replace the masked data by looking up the equivalent real data in the dictionary. Stating the obvious, updating based on masked data introduces some security risks, so you would want to restrict narrowly what can be replaced. Caveat: I've never tried to implement import for masked data. Perhaps others on the distribution have better suggestions. Erik Hennum From: Shmennen <shmen...@yahoo.com> Sent: Monday, October 30, 2017 2:24:09 PM To: Erik Hennum; General@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Data Masking on Import/Export Any thoughts about how to import masked data in MarkLogic? Some built in features? Thanks Ric On Mon, Oct 30, 2017 at 18:34, Shmennen <shmen...@yahoo.com> wrote: Thanks Erik, indeed I could not find anything in the docs ... BTW, I found this thread http://developer.marklogic.com/pipermail/general/2015-March/016633.html which mentions about data masking in context of flexible replication. I have configured one Master and one Slave based on https://docs.marklogic.com/guide/flexrep/quick_start#id_50142 but the info is not masked on Slave. Do you know what I can configure in addition to mask data on flexible replication? -Ric On Monday, October 30, 2017 6:24 PM, Erik Hennum <erik.hen...@marklogic.com> wrote: Hi, Ric: So far as I know, redaction is purely an export feature. Erik Hennum From: Shmennen <shmen...@yahoo.com<mailto:shmen...@yahoo.com>> Sent: Monday, October 30, 2017 2:51:20 AM To: Erik Hennum; General@developer.marklogic.com<mailto:General@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Data Masking on Import/Export Thanks Erik. Is there any way to import the masked data (e.g. the masked fields + updated not-masked fields) and MarkLogic to do the correlation on the fly for masked data? I could not find this in the Redaction part... Thanks Ric On Saturday, October 28, 2017 2:34 AM, Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Hi, Richard: These are two separate features, which were introduced in MarkLogic 9: * Element Level Security prevents access to unpermitted branches of the document http://docs.marklogic.com/guide/security/element * Redaction masks data on export http://docs.marklogic.com/guide/app-dev/redaction A redacted export of the data can only be generated by a user with a role that has permission to access to the data. The user who generates the redacted export can, of course, make the redacted data available to users without permission to access the data. Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com><mailto:general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com><mailto:general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>>> on behalf of Shmennen <shmen...@yahoo.com<mailto:shmen...@yahoo.com><mailto:shmen...@yahoo.com<mailto:shmen...@yahoo.com>>> Sent: Thursday, October 26, 2017 1:26:50 PM To: General@developer.marklogic.com<mailto:General@developer.marklogic.com><mailto:General@developer.marklogic.com<mailto:General@developer.marklogic.com>> Subject: [MarkLogic Dev General] Data Masking on Import/Export Hello All, Could you please point me to some useful documentation in order to configure data masking in MarkLogic? e.g. If a user cannot see field A inside an XML (e.g. user is not granted to permissions for that field), on export that field A should be masked. Then, if user modifies the exported data (fields he has access to, not the one masked) and wants to re-import it, then the field masked should be correlated to the initial value from ML. Thanks -Richard ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Data Masking on Import/Export
Hi, Ric: So far as I know, redaction is purely an export feature. Erik Hennum From: Shmennen <shmen...@yahoo.com> Sent: Monday, October 30, 2017 2:51:20 AM To: Erik Hennum; General@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Data Masking on Import/Export Thanks Erik. Is there any way to import the masked data (e.g. the masked fields + updated not-masked fields) and MarkLogic to do the correlation on the fly for masked data? I could not find this in the Redaction part... Thanks Ric On Saturday, October 28, 2017 2:34 AM, Erik Hennum <erik.hen...@marklogic.com> wrote: Hi, Richard: These are two separate features, which were introduced in MarkLogic 9: * Element Level Security prevents access to unpermitted branches of the document http://docs.marklogic.com/guide/security/element * Redaction masks data on export http://docs.marklogic.com/guide/app-dev/redaction A redacted export of the data can only be generated by a user with a role that has permission to access to the data. The user who generates the redacted export can, of course, make the redacted data available to users without permission to access the data. Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of Shmennen <shmen...@yahoo.com<mailto:shmen...@yahoo.com>> Sent: Thursday, October 26, 2017 1:26:50 PM To: General@developer.marklogic.com<mailto:General@developer.marklogic.com> Subject: [MarkLogic Dev General] Data Masking on Import/Export Hello All, Could you please point me to some useful documentation in order to configure data masking in MarkLogic? e.g. If a user cannot see field A inside an XML (e.g. user is not granted to permissions for that field), on export that field A should be masked. Then, if user modifies the exported data (fields he has access to, not the one masked) and wants to re-import it, then the field masked should be correlated to the initial value from ML. Thanks -Richard ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Temporal data implementation
Hi, Grant: One footnote on Subhadra's good guidance ... > Lets say my data is represented as documents with about 1000 unique JSON > paths... > Is the Marklogic temporal data feature implemented at the document level or > the individual path/value level? If the former, it would seem like a lot of > redundant storage of copies of data that does not change. The best way to modelin MarkLogic is to represent each record in a separate document and assemble records as needed using collections or queries. In relational terms, think of a document as a row instead of as a table. Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Subhadra Thirumavalavan <subi.thirumavala...@marklogic.com> Sent: Saturday, October 28, 2017 9:26:30 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Temporal data implementation Hi, It is implemented at the document level. When a temporal document is updated, the original version of the document is preserved and a copy of the document with the updated data is inserted as a new document. So even if the change Is minimal there will be two documents in the database. Thanks Subhadra From: <general-boun...@developer.marklogic.com> on behalf of Grant McKenzie <mcken...@optonline.net> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com> Date: Saturday, October 28, 2017 at 6:34 PM To: "general@developer.marklogic.com" <general@developer.marklogic.com> Subject: [MarkLogic Dev General] Temporal data implementation Hi, I have a question regarding your bi-temporal data feature documented here: http://www.marklogic.com/what-is-marklogic/whats-new/bitemporal/<mailto:general@developer.marklogic.com> Lets say my data is represented as documents with about 1000 unique JSON paths (ie paths to leaf values). The typical change is a single leaf value for one path i.e. 999 values stay the same, one changes. Is the Marklogic temporal data feature implemented at the document level or the individual path/value level? If the former, it would seem like a lot of redundant storage of copies of data that does not change. Thanks. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Xquery to procure part no's, effectivity, units
Hi, Eric:
The appropriate code depends on the purpose, data model, and release.
Alternatives include:
* Using a cts:search() to find the appropriate documents and using XPath to
extract the element values
* Using range indexes and cts:value-tuples() to construct dynamic rows based
on document co-occurrence in data typed value indexes
* Using TDE to project rows from documents and using Optic to retrieve rows
from indexes (in MarkLogic 9)
Can you expand about the context for the question?
Erik Hennum
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Eric Shevchuk
<eric.shevc...@bristowgroup.com>
Sent: Thursday, October 26, 2017 10:46:57 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Xquery to procure part no's, effectivity,
units
Hello,
I am trying to construct an xquery to procure the below elements as variables
as need to procure all part no’s, effectivity and only numeric units?. Any
assistance you could provide would be greatly appreciated.
Examples – Part No, Effectivity, Units
92103-15000-042
006-084
1
Thanks,
Eric Shevchuk
Sr. Production Support/Applications Consultant
Infosys/Bristow Group, Inc.
2103 City West Blvd., 2nd Floor
Houston, TX 77042
eric.shevc...@bristowgroup.com
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Oleksii Segeda
Sent: Thursday, August 03, 2017 3:05 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] fitness score
Hi everyone,
I calculate relevance of a single document to some query by doing this:
cts:fitness(
cts:search(
fn:doc(),
cts:and-query((
cts:document-query(... document uri ...),
... some query ...
)),
("score-logtf","unfiltered")
)
)
I want to do the same thing, but for an element, without ingesting this element
to my database.
Any ideas?
Oleksii Segeda
IT Analyst
Information and Technology Solutions
[http://siteresources.worldbank.org/NEWS/Images/spacer.png]
[http://siteresources.worldbank.org/NEWS/Images/WBG_Information_and_Technology_Solutions.png]
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Data Masking on Import/Export
Hi, Richard: These are two separate features, which were introduced in MarkLogic 9: * Element Level Security prevents access to unpermitted branches of the document http://docs.marklogic.com/guide/security/element * Redaction masks data on export http://docs.marklogic.com/guide/app-dev/redaction A redacted export of the data can only be generated by a user with a role that has permission to access to the data. The user who generates the redacted export can, of course, make the redacted data available to users without permission to access the data. Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Shmennen <shmen...@yahoo.com> Sent: Thursday, October 26, 2017 1:26:50 PM To: General@developer.marklogic.com Subject: [MarkLogic Dev General] Data Masking on Import/Export Hello All, Could you please point me to some useful documentation in order to configure data masking in MarkLogic? e.g. If a user cannot see field A inside an XML (e.g. user is not granted to permissions for that field), on export that field A should be masked. Then, if user modifies the exported data (fields he has access to, not the one masked) and wants to re-import it, then the field masked should be correlated to the initial value from ML. Thanks -Richard ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] REST-API: Optimistic Locking - Feature Request
Hi, Andreas: exists() wraps the HEAD request and returns a DocumentDescriptor, which have the version if optimistic locking is enabled: https://docs.marklogic.com/javadoc/client/com/marklogic/client/document/DocumentManager.html#exists-java.lang.String- Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Andreas Hubmer <andreas.hub...@ebcont.com> Sent: Thursday, September 28, 2017 6:40:03 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] REST-API: Optimistic Locking - Feature Request Hi, Do you know whether the Java API contains a method to perform such a HEAD request? Thanks, Andreas 2017-09-28 15:34 GMT+02:00 Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>>: Hi, Andreas: Where a gap in the product functionality affect your ability to deliver on requirements, it's best to raise an issue with Support. On the particular issue, if you don't need the document content, HEAD should also return the ETag timestamp for use in optimistic locking. Of course, the HTTP request still requires a network roundtrip. Depending on network efficiency, it might not make that much difference. Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of Andreas Hubmer <andreas.hub...@ebcont.com<mailto:andreas.hub...@ebcont.com>> Sent: Thursday, September 28, 2017 5:16:47 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] REST-API: Optimistic Locking - Feature Request Hi, We use optimistic locking. The document-get request returns the Etag header which we send in the If-Match header of a subsequent document-put request. We have a use case where a user performs several updates (put requests) in a row. Currently, we have to perform a get request after each update to retrieve the new etag. The request pattern looks like this: GET PUT, immediately followed by GET PUT, immediately followed by GET PUT, immediately followed by GET The additional get requests could be avoid, if the REST API (and the Java API) would return the new etag in the response of the PUT request. Similarly, the etag of a document should be returned with the response of a POST request, where the server-side generates the URI (https://docs.marklogic.com/8.0/REST/POST/v1/documents@extension=[ext]). Cheers, Andreas -- Andreas Hubmer Senior IT Consultant EBCONT enterprise technologies GmbH ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] REST-API: Optimistic Locking - Feature Request
Hi, Andreas: Where a gap in the product functionality affect your ability to deliver on requirements, it's best to raise an issue with Support. On the particular issue, if you don't need the document content, HEAD should also return the ETag timestamp for use in optimistic locking. Of course, the HTTP request still requires a network roundtrip. Depending on network efficiency, it might not make that much difference. Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Andreas Hubmer <andreas.hub...@ebcont.com> Sent: Thursday, September 28, 2017 5:16:47 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] REST-API: Optimistic Locking - Feature Request Hi, We use optimistic locking. The document-get request returns the Etag header which we send in the If-Match header of a subsequent document-put request. We have a use case where a user performs several updates (put requests) in a row. Currently, we have to perform a get request after each update to retrieve the new etag. The request pattern looks like this: GET PUT, immediately followed by GET PUT, immediately followed by GET PUT, immediately followed by GET The additional get requests could be avoid, if the REST API (and the Java API) would return the new etag in the response of the PUT request. Similarly, the etag of a document should be returned with the response of a POST request, where the server-side generates the URI (https://docs.marklogic.com/8.0/REST/POST/v1/documents@extension=[ext]). Cheers, Andreas -- Andreas Hubmer Senior IT Consultant EBCONT enterprise technologies GmbH ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Multiple Inputs for Custom XQuery Extension
Hi, Bushan: Yes, a resource service extension can take any number of parameters. You prefix each parameter with rs: when constructing the equivalent HTTP URI parameter. Please see: http://docs.marklogic.com/REST/GET/v1/resources/[name] Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Bhushan Suryawanshi <bsuryawan...@xenomorph.com> Sent: Friday, September 15, 2017 9:28:35 AM To: General@developer.marklogic.com Subject: [MarkLogic Dev General] Multiple Inputs for Custom XQuery Extension Hello Team, Need some help here. I am using MarkLogic 8.0, suing REST API. I have created a custom XQuery extension which expecting a document as a Input and would give back the requested Node which is required. Now I want to add an another input parameter to the same extension is it possible? and How multiple parameters would be used In a URI. Below is the snippet of get function of my XQuery Resource extension. Thanks in advance for the help. [cid:image001.jpg@01D32E1D.E6045810] Thanks, Bhushan Suryawanshi Developer t: +1-212-401-7894 www.xenomorph.com<http://www.xenomorph.com/?mail> | blog<http://xenomorph.typepad.com/?mail> | twitter<http://goo.gl/oaBWO> | linkedin<http://goo.gl/rdi8W> This email is confidential and is intended only for the addressee. If you are not the intended recipient, please note that any dissemination, distribution or copying of this email is strictly prohibited. Any and all estimates, proposals, quotes and fees contained herein are for discussion purposes only and are not contractually binding unless executed in accordance with the standard practices of the Company. Attachments to this message have been virus checked but no guarantee can be made that any attachment is virus free. Please notify us immediately of any problem. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Noob query question..
Hi, Eric:
In MarkLogic 9, you could create a TDE template to project rows into the index
with columns for the subject and date_taken.
Then, using SQL, SPARQL, or Optic, you could group on the subject and count the
rows as filtered by the date.
In MarkLogic 8, you could set up a range indexes on the subject and date_taken
and use a values query on the subject range index, counting document frequency
and filtering on the date.
Hoping that's orienting,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Ladner, Eric
(Eric.Ladner) [eric.lad...@chevron.com]
Sent: Thursday, August 24, 2017 2:30 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Noob query question..
Thank you. I will play with this in my development environment tomorrow. I
don’t quote see how it’s getting the counts per subject, though.
For reference.. the structure is similar to this:
Test Subject
2017-04-01T15:32:00
Blah, blah
There would be many notes, obviously and the output would ideally be something
like (not married to that output, but some output showing the counts for each
subject over that time range).
Test Subject
2
Subject 2
4
...
Eric Ladner
Systems Analyst
eric.lad...@chevron.com<mailto:eric.lad...@chevron.com>
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Sam Mefford
Sent: August 24, 2017 15:59
To: MarkLogic Developer Discussion <general@developer.marklogic.com>
Subject: [**EXTERNAL**] Re: [MarkLogic Dev General] Noob query question..
I should point out that this is not the fastest way to do it. A faster way
would be to index "date-taken" as a dateTime element range index and use
cts:search with cts:element-range-query.
Sam Mefford
Senior Engineer
MarkLogic Corporation
sam.meff...@marklogic.com<mailto:sam.meff...@marklogic.com>
Cell: +1 801 706 9731
www.marklogic.com<http://www.marklogic.com>
This e-mail and any accompanying attachments are confidential. The information
is intended solely for the use of the individual to whom it is addressed. Any
review, disclosure, copying, distribution, or use of this e-mail communication
by others is strictly prohibited. If you are not the intended recipient, please
notify us immediately by returning this message to the sender and delete all
copies. Thank you for your cooperation.
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com] on behalf of Sam Mefford
[sam.meff...@marklogic.com]
Sent: Thursday, August 24, 2017 2:56 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Noob query question..
XQuery is an extension of XPath. Here's an example in XPath. These things are
easiest to understand if we know the structure of your docs. Suppose I insert:
xdmp:document-insert("test.xml",
2015-01-01)
I could find the count of docs more than two years old like this:
count(/note[fn:days-from-duration(fn:current-date() - date-taken) > (365 * 2)])
Sam Mefford
Senior Engineer
MarkLogic Corporation
sam.meff...@marklogic.com<mailto:sam.meff...@marklogic.com>
Cell: +1 801 706 9731
www.marklogic.com<http://www.marklogic.com>
This e-mail and any accompanying attachments are confidential. The information
is intended solely for the use of the individual to whom it is addressed. Any
review, disclosure, copying, distribution, or use of this e-mail communication
by others is strictly prohibited. If you are not the intended recipient, please
notify us immediately by returning this message to the sender and delete all
copies. Thank you for your cooperation.
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com] on behalf of Ladner, Eric
(Eric.Ladner) [eric.lad...@chevron.com]
Sent: Thursday, August 24, 2017 2:24 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Noob query question..
I’m still rather new to MarkLogic and apparently have a lot to learn. When
doing research on a proof of concept, I ran across a situation that would be
trivial to solve in SQL, but I’m having problems wrapping my head around how to
do that in XQuery. Or, is XQuery even the right place for this?
Basically, the number of notes per subject for any note that’s less than two
years old. If I was to do this in SQL, it’d look something like:
select subject, count(*) from notes where date_taken > sysdate-(365*2) group
by subject;
There’s some additional WHERE clause stuff for filtering, but on average, the
number of results shouldn’t be large.
Any guidance on building up more complex queries like this
Re: [MarkLogic Dev General] Optimizing XPath Equals Performance
Hi, Basu:
Instead of putting many articles and contributors in a single document
and using XPath to extract the articles and contributors from the
document, have you explored putting each entity in a separate
document and relying on search to give you just the entities you need?
(For those coming from a relational background, the usual guidance
is treat a document as a row instead of as a table.)
The benefit is not only in getting the full benefit of the indexes
in accelerating retrieval and minimizing load but also in avoiding
contention. (Updates apply to the same document much less
frequently if documents are small.)
Retrieving large data sets to the application to filter on the client is
likely to be quite inefficient and increase load.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Basavaraj Kalloli
[basavarajkall...@gmail.com]
Sent: Monday, August 14, 2017 4:34 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Optimizing XPath Equals Performance
Hi,
We have been following up with our on going performance problems and we found
that we are looking up a bunch of documents in Marklogic and then doing an
XPath equals on each of them to find all the nodes that match a contributor,
the query is:
declare namespace npg="";
declare namespace rdf="";
let $uri := cts:uri-match('*blah*')
let $record := doc($uri)
let $article := $record//semantic-metadata/graph/rdf:RDF/npg:Article
let $contributors := $article/npg:hasContributor/@rdf:resource
return $record//semantic-metadata/graph/rdf:RDF/npg:Contributor[@rdf:about eq
$article/npg:hasContributor/@rdf:resource]
The number of contributors in the above case where 3054 so I am looking up all
the contributor nodes (which could be more) in the document and matching any of
the 3054.
My understanding is that we are not doing a search but instead loading the
whole document and doing a comparison on all the contributor nodes hence adding
any indexes is not going to help? Or is there a smarter way of doing this that
I don't know about? Any help would appreciated, these sort of queries are
really killing our MarkLogic cluster.
One other option would be potentially move this outside of xquery and do in the
app rather than using up Marklogic resources which could be used for search
queries instead.
Thanks,
Basu
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Tracking Spawned Tasks?
Hi, Eliot and Ron: The return option is explained with the rest of the options in the eval article: http://docs.marklogic.com/xdmp:eval The second example under spawn uses the promise: http://docs.marklogic.com/xdmp:spawn As Ron notes, the server field is only useful if the polling requests go back to the same host. To allow for restarts, the polling logic should check for the persisted final status document if the server field is empty. (That's the motivation for persisting a final status document even when using server fields.) Thanks for the feedback on the documentation -- I'll pass that along. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Ron Hitchens [r...@ronsoft.com] Sent: Monday, August 14, 2017 7:55 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Tracking Spawned Tasks? Proceed with caution when using server fields. They exist only on a single machine, they are not propagated across nodes in a cluster. If you have a cluster behind a load balancer (as most are) and you stash something in a server field to be checked later, the next request may be vectored to a different cluster node, where your stashed value will not be present. Likewise, if you put something in a field to be picked up by a spawned task, the spawned task may run on a different node. Ron Hitchens r...@overstory.co.uk<mailto:r...@overstory.co.uk>, +44 7879 358212 On August 14, 2017 at 3:24:32 PM, Eliot Kimber (ekim...@contrext.com<mailto:ekim...@contrext.com>) wrote: I like using set-server-field: my requirement feels like just what server fields were intended for. Cheers, E. -- Eliot Kimber http://contrext.com On 8/14/17, 8:32 AM, "general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> on behalf of Erik Hennum" <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> on behalf of erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Hi, Eliot: xdmp:spawn() doesn't return an identifier because, if it is used as a future via the result option, it is obligated to return the result. The approach you sketch below -- passing in an identifier and writing tickets to a status database -- is pretty much what InfoStudio did. One refinement would be to log status in a server field via xdmp:set-server-field() and, on completion, write final status to a database (for durability in the case of a restart). Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Eliot Kimber [ekim...@contrext.com<mailto:ekim...@contrext.com>] Sent: Saturday, August 12, 2017 10:15 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Tracking Spawned Tasks? Using ML 8 I’m refining a profiling application that spawns a number of tasks and then, eventually, reports on the results once all the tasks have completed. Right now I just fire off the tasks and then refresh my app, which looks for results. It would be nice to be able to show the status of the spawned tasks but it looks like xdmp:spawn() doesn’t return anything (sort of expected to get some sort of task ID or something) and so there’s no obvious way to track spawned tasks from the spawning application. I could do something like generate private task IDs and pass those as parameters to the spawned tasks and then maintain a set of task status docs, but I was hoping there was some something easier. It seems like it would be a common requirement but I couldn’t find anything useful in the ML 8 docs or searching the web. Thanks, Eliot -- Eliot Kimber http://contrext.com ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Tracking Spawned Tasks?
Hi, Eliot: xdmp:spawn() doesn't return an identifier because, if it is used as a future via the result option, it is obligated to return the result. The approach you sketch below -- passing in an identifier and writing tickets to a status database -- is pretty much what InfoStudio did. One refinement would be to log status in a server field via xdmp:set-server-field() and, on completion, write final status to a database (for durability in the case of a restart). Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Eliot Kimber [ekim...@contrext.com] Sent: Saturday, August 12, 2017 10:15 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Tracking Spawned Tasks? Using ML 8 I’m refining a profiling application that spawns a number of tasks and then, eventually, reports on the results once all the tasks have completed. Right now I just fire off the tasks and then refresh my app, which looks for results. It would be nice to be able to show the status of the spawned tasks but it looks like xdmp:spawn() doesn’t return anything (sort of expected to get some sort of task ID or something) and so there’s no obvious way to track spawned tasks from the spawning application. I could do something like generate private task IDs and pass those as parameters to the spawned tasks and then maintain a set of task status docs, but I was hoping there was some something easier. It seems like it would be a common requirement but I couldn’t find anything useful in the ML 8 docs or searching the web. Thanks, Eliot -- Eliot Kimber http://contrext.com ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Trying to Get ML Data Visualization Widgets Working
Hi, Eliot:
The AppBuilder has been superseded by initiatives in the JavaScript ecosystem
and is deprecated in MarkLogic 8 and removed in 9.
I've heard good things about the D3 (versatile) and vis.js (easy) Open Source
JavaScript visualization libraries.
Hoping that's useful,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Eliot Kimber
[ekim...@contrext.com]
Sent: Monday, August 07, 2017 12:24 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Trying to Get ML Data Visualization Widgets
Working
Using ML 8, I’m setting up a little profiling web application and I need to do
visualization on the recorded data, e.g., durations reported by query meters
for a large number operations.
I’m following the guidance in the Search Developer's Guide — Chapter 31, Data
Visualization Widgets, in the context of my own simple Web app (that is, I did
not use the application builder to initially create my app, I just created a
simple HTTP app from scratch.
I’m generating an HTML page that includes all the Javascript for visualization:
var durationBarChartConfig = {
title: "Duration Distributions",
dataLabel: "Durations",
dataType: "int"
}
ML.controller.init();
ML.chartWidget('duration-bar-chart-1', 'bar', durationBarChartConfig);
ML.chartWidget('duration-bar-chart-2', 'bar', durationBarChartConfig);
ML.chartWidget('duration-bar-chart-3', 'bar', durationBarChartConfig);
ML.controller.loadData();
And in the main HTML I’m generating the corresponding widget-containing divs:
However, when I load the page I get this result in the console:
chart.js:82 Uncaught Chart widget container ID "duration-bar-chart-1" does not
exist
The element exists and there are no other errors in the JS console.
I assume I must be missing something basic here but as I’m not at all versed in
JavaScript I’m hoping someone can point me in the right direction.
I didn’t see anything in the ML guide or the underlying JavaScript code that
suggested I’m missing some setup.
Thanks,
Eliot
--
Eliot Kimber
http://contrext.com
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] JSearch - collections
HI, Florent: Au contraire, thank you for identifying these issues and for persisting. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Florent Georges [li...@fgeorges.org] Sent: Tuesday, August 01, 2017 6:20 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] JSearch - collections Thank you, Erik! I hoped there was a ready-to-use function for that. Thank you for confirming there is none. I definitely owe you a drink of your choice after the past few days :-) Regards, -- Florent Georges H2O Consulting http://h2o.consulting/ On 1 August 2017 at 15:10, Erik Hennum wrote: Hi, Florent: The collections() call is just a shortcut / convenience. A full query can have a cts.andQuery() over a cts.collectionQuery() list. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Florent Georges [li...@fgeorges.org<mailto:li...@fgeorges.org>] Sent: Tuesday, August 01, 2017 3:46 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] JSearch - collections Hi, The jsearch.collections() function uses the union of the collections. Is there another function for the intersection instead? That is, to select all documents tagged with ALL the given collections (instead of ANY of them). Regards, -- Florent Georges H2O Consulting http://h2o.consulting/ ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] JSearch - collections
Hi, Florent: The collections() call is just a shortcut / convenience. A full query can have a cts.andQuery() over a cts.collectionQuery() list. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Florent Georges [li...@fgeorges.org] Sent: Tuesday, August 01, 2017 3:46 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] JSearch - collections Hi, The jsearch.collections() function uses the union of the collections. Is there another function for the intersection instead? That is, to select all documents tagged with ALL the given collections (instead of ANY of them). Regards, -- Florent Georges H2O Consulting http://h2o.consulting/ ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Search results are same w/ or w/o punctuation with unfiltered search option
Hi, Indrajeet: Mary Holstege provides a concise and authoritative explanation here: http://markmail.org/message/r2sdkmjzrawdjpjn In essence, enable the fast-diacritic-sensitive-searches index (either for the entire database or a field on the element or property) and pass the punctuation sensitive (or exact) option on a value query. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Indrajeet Verma [indrajeet.ve...@gmail.com] Sent: Monday, July 31, 2017 4:38 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Search results are same w/ or w/o punctuation with unfiltered search option Hi - The search:search() is giving same results for keyword 10, 10%, 10#%@ and $10 when I use "unfiltered" search option. Though when I use "filtered" search-option, the results are accurate and punctuation is considered in the query text. Is there any way if I can get correct results using "unfiltered" option for the query text with punctuation as if I use filtered option the response time drastically increases. Thank You! Regards, Indrajeet ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Client API - Multipart doc ingest - Set each part type
Hi, Florent:
The mime type mapping for the extension takes precedence over the content type
header. To put it the other way, the header is used only if the extension is
not mapped.
The rationale is that the header is ephemeral but the mapping is persistent.
The REST API encourages the best practice of using extensions that reflect the
content type. When that best practice is not followed, the document content
has to be inspected to determine the content type, reducing the manageability
of a database. The recommendation is to use extensions that are consistent with
the content type.
Hoping that clarifies,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Florent Georges
[li...@fgeorges.org]
Sent: Sunday, July 30, 2017 10:00 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Client API - Multipart doc ingest - Set each
part type
Hi,
I am using the POST /v1/documents endpoint of the Client API on 8000, to ingest
several documents in one multipart request. The section "Controlling Document
Type" [1], in chapter "Reading and Writing Multiple Documents" in the REST
Guide, says:
"You can create JSON, XML, Binary, and Text documents in a bulk write. The MIME
type of the part Content-Type header determines the database document type."
And indeed, if I create a multipart to create documents with the following URI,
with the following Content-Type, then it works fine (it creates four documents,
of type resp. text, XML, JSON and binary):
- /test/file-text.xxx - text/plain
- /test/file-xml.xxx - application/xml
- /test/file-json.xxx - application/json
- /test/file-bin.xxx - application/octet-stream
But if I replace the URI to use the extension ".xqy" instead of ".xxx" (the
latter not being in the MIME table), then it uses text for all of them. Like
if the "extension" of the URI was used and took precedence if it is in the MIME
table.
I can provide code to reproduce the multipart invocation, but I think it would
be rather noise here. The code below is to retrieve the type of each document
created.
Is it a documentation bug? Or a product bug? Or just something I did wrong?
Regards,
[1]https://docs.marklogic.com/guide/rest-dev/bulk#id_75274
declare function local:doc-type($uri as xs:string)
{
let $root := fn:doc($uri)/node()
return
if ( $root instance of text() )then 'text'
else if ( $root instance of element() ) then 'xml'
else if ( $root instance of object-node() ) then 'json'
else 'binary'
};
('/test/file-text.xqy',
'/test/file-xml.xqy',
'/test/file-json.xqy',
'/test/file-bin.xqy',
'/test/file-text.xxx',
'/test/file-xml.xxx',
'/test/file-json.xxx',
'/test/file-bin.xxx')
! (. || ' - ' || local:doc-type(.))
Result is:
/test/file-text.xqy - text
/test/file-xml.xqy - text
/test/file-json.xqy - text
/test/file-bin.xqy - text
/test/file-text.xxx - text
/test/file-xml.xxx - xml
/test/file-json.xxx - json
/test/file-bin.xxx - binary
--
Florent Georges
H2O Consulting
http://h2o.consulting/
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Client API - 8000 - POST /v1/documents - Unexpected 400
Hi, Florent:
I'm glad that you resolved the issue.
One footnote: the Node.js API in Node.js and xdmp.multipartEncode() in SJS
can handle construction of a multipart/mixed payload on behalf of applications.
That said, one way or another, the 500 error is a bug.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Florent Georges
[li...@fgeorges.org]
Sent: Sunday, July 30, 2017 9:21 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Client API - 8000 - POST /v1/documents -
Unexpected 400
Hi Erik,
Thank you for your response! After I read your email, the first thing I tried
was to remove the CRLF before the first boundary. I must admit I didn't
believe it would help, but it seems it was the problem. Removing it made the
request pass, in both the example I posted (ran in QConsole) and in a totally
separate example written for Node.
So to the REST API devs... I am not an expert in the HTTP grammar (like if if
anyone could claim so), but it looks to me the CRLF should be ignored as part
of the preamble. In any case, the 500 could be avoided (either to ignore the
CRLF or to return a 400 instead with a meaningful message).
But as far as I am concerned, problem solved ;-) Thank you, Erik!
Regards,
--
Florent Georges
H2O Consulting
http://h2o.consulting/
On 30 July 2017 at 16:09, Erik Hennum wrote:
Hi, Florent:
I'm wondering whether a CRLF is required before the initial boundary. That
seems to be implied but not explicitly stated by:
https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
A couple of suggestions for investigating:
* Confirm that the constructed multipart/mixed is parseable using
xdmp.multipartDecode()
https://docs.marklogic.com/xdmp.multipartDecode
* Write the two documents individually, read both document back in a single
request that accepts multipart/mixed, and compare the REST API response to the
constructed multipart/mixed
https://docs.marklogic.com/REST/GET/v1/documents
If that doesn't discover the issue, maybe setting the content-length in the
part might help (though the standard explicitly requires no headers in the
part).
Hoping that's useful,
Erik Hennum
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>]
on behalf of Florent Georges [li...@fgeorges.org<mailto:li...@fgeorges.org>]
Sent: Sunday, July 30, 2017 5:56 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Client API - 8000 - POST /v1/documents -
Unexpected 400
Hi,
Actually, I made a mistake in the Content-Type of the overall request, I used
"multipart/related" instead of "multipart/mixed" as mentioned in the doc. But
when I fix it, I get a 500 with the following error message:
XDMP-AS: (err:XPTY0004) $meta-type as xs:string -- Invalid coercion: () as
xs:string . See the MarkLogic server error log for further detail.
It seems it all goes down to $content-type being the empty sequence in the
following call in document-model-update.xqy (in ML 9.0-1.1):
let $document-next :=
if ($part-type ne "document-metadata") then ()
else docmodupd:parse-metadata-map($curr-uri,$content-type,$body)
But then, it is beyond my knowledge...
Regards,
--
Florent Georges
H2O Consulting
http://h2o.consulting/
On 30 July 2017 at 11:41, Florent Georges wrote:
Hi,
I am trying to use the POST /v1/documents endpoint on 8000 to insert several
documents at once. From the chapter "Reading and Writing Multiple Documents"
in the REST Guide, in section "Specifying an Explicit Document URI" [1], the
following pseudo header in each part should be enough to set the URI for each
document:
Content-Disposition: attachment;filename=/your/uri
But I get a 400 with the message: "REST-REQUIREDPARAM: (err:FOER) Required
parameter: missing required extension parameter".
Any idea why?
[1]https://docs.marklogic.com/guide/rest-dev/bulk#id_84006
Below a complete repro that can be played in QConsole (and later an example
using PUT instead, which does work):
'use strict';
const rn = '\r\n';
const bnd = '.ml.rockzzz.';
var body = rn;
// first part
body += '--' + bnd + rn;
body += 'Content-Type: application/json' + rn;
body += 'Content-Disposition: attachment; filename="/test/multi-one.json"' + rn;
body += rn;
body += JSON.stringify({ hello: 'one!' });
body += rn;
// second part
body += '--' + bnd + rn;
body += 'Content-Type: application/json' + rn;
body += 'Content-Disposition: attachment; filename="/test/multi-two.json"' + rn;
body += rn;
body += JSON.stringify({ hello: 'two!' });
body += rn;
// after the last part
body += '--' + bnd + '--' + rn;
xdmp.httpPost(
'htt
Re: [MarkLogic Dev General] Client API - 8000 - POST /v1/documents - Unexpected 400
Hi, Florent:
I'm wondering whether a CRLF is required before the initial boundary. That
seems to be implied but not explicitly stated by:
https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
A couple of suggestions for investigating:
* Confirm that the constructed multipart/mixed is parseable using
xdmp.multipartDecode()
https://docs.marklogic.com/xdmp.multipartDecode
* Write the two documents individually, read both document back in a single
request that accepts multipart/mixed, and compare the REST API response to the
constructed multipart/mixed
https://docs.marklogic.com/REST/GET/v1/documents
If that doesn't discover the issue, maybe setting the content-length in the
part might help (though the standard explicitly requires no headers in the
part).
Hoping that's useful,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Florent Georges
[li...@fgeorges.org]
Sent: Sunday, July 30, 2017 5:56 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Client API - 8000 - POST /v1/documents -
Unexpected 400
Hi,
Actually, I made a mistake in the Content-Type of the overall request, I used
"multipart/related" instead of "multipart/mixed" as mentioned in the doc. But
when I fix it, I get a 500 with the following error message:
XDMP-AS: (err:XPTY0004) $meta-type as xs:string -- Invalid coercion: () as
xs:string . See the MarkLogic server error log for further detail.
It seems it all goes down to $content-type being the empty sequence in the
following call in document-model-update.xqy (in ML 9.0-1.1):
let $document-next :=
if ($part-type ne "document-metadata") then ()
else docmodupd:parse-metadata-map($curr-uri,$content-type,$body)
But then, it is beyond my knowledge...
Regards,
--
Florent Georges
H2O Consulting
http://h2o.consulting/
On 30 July 2017 at 11:41, Florent Georges wrote:
Hi,
I am trying to use the POST /v1/documents endpoint on 8000 to insert several
documents at once. From the chapter "Reading and Writing Multiple Documents"
in the REST Guide, in section "Specifying an Explicit Document URI" [1], the
following pseudo header in each part should be enough to set the URI for each
document:
Content-Disposition: attachment;filename=/your/uri
But I get a 400 with the message: "REST-REQUIREDPARAM: (err:FOER) Required
parameter: missing required extension parameter".
Any idea why?
[1]https://docs.marklogic.com/guide/rest-dev/bulk#id_84006
Below a complete repro that can be played in QConsole (and later an example
using PUT instead, which does work):
'use strict';
const rn = '\r\n';
const bnd = '.ml.rockzzz.';
var body = rn;
// first part
body += '--' + bnd + rn;
body += 'Content-Type: application/json' + rn;
body += 'Content-Disposition: attachment; filename="/test/multi-one.json"' + rn;
body += rn;
body += JSON.stringify({ hello: 'one!' });
body += rn;
// second part
body += '--' + bnd + rn;
body += 'Content-Type: application/json' + rn;
body += 'Content-Disposition: attachment; filename="/test/multi-two.json"' + rn;
body += rn;
body += JSON.stringify({ hello: 'two!' });
body += rn;
// after the last part
body += '--' + bnd + '--' + rn;
xdmp.httpPost(
'http://localhost:8000/v1/documents', {
authentication : {
method : 'digest',
username : 'admin',
password : 'admin'
},
headers: {
"Content-type": 'multipart/related; boundary=' + bnd,
Accept: "application/json"
},
data: body
});
Here is the example using PUT, and indeed working fine:
'use strict';
xdmp.httpPut(
'http://localhost:8000/v1/documents?uri=/test/single.json', {
authentication : {
method : 'digest',
username : 'admin',
password : 'admin'
},
headers: {
"Content-type": 'application/json',
Accept: "application/json"
},
data: JSON.stringify({ hello: 'world!' })
});
Any idea what I did wrong?
Regards,
--
Florent Georges
H2O Consulting
http://h2o.consulting/
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] SPARQL 'SERVICE' ?
Hi, Tony: SPARQL SERVICE is an extension to SPARQL 1.1 for federated query: "The SERVICE keyword instructs a federated query processor to invoke a portion of a SPARQL query against a remote SPARQL endpoint." https://www.w3.org/TR/2013/REC-sparql11-federated-query-20130321/#service As an extension, SERVICE is not part of core SPARQL 1.1 Federated queries are at risk for fundamental architecture issues with respect to performance, transactional consistency, resilience, and so on. For that reason, MarkLogic does not federate queries across other data sources. It should be possible to use MarkLogic as one data source for a SPARQL/RDF tool that implements federation. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Tony Greaves [tony.grea...@hill-informatics.co.nz] Sent: Tuesday, July 25, 2017 4:29 PM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] SPARQL 'SERVICE' ? The MarkLogic documentation states "SPARQL queries are executed natively in MarkLogic to query either in-memory triples or triples stored in a database. When querying triples stored in a database, SPARQL queries execute entirely against the triple index". This would appear to imply that the SPARQL *SERVICE* function is not supported. Experimentation seems to confirm this, but I have not found any explicit comment to this effect in the documentation. Can someone confirm this one way or the other please? thanks Tony Attention: The information in this email and in any attachments is confidential. If you are not the intended recipient then please do not distribute, copy or use this information. Please notify us immediately and then delete the message from your computer. Any views or opinions presented are solely those of the author. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Issue with Appservices after ML upgrade to ML9
Hi, Margare: Modern JavaScript frameworks such as Angular and React have more functionality and vastly more adoption than the custom AppBuilder framework. Three tier architectures are common in production environments. MarkLogic 8 announced that AppBuilder would be removed: http://docs.marklogic.com/8.0/guide/app-builder/intro One common approach is to use a three tier architecture with a modern JavaScript framework in the front end and Java or Nodes.js in the middle tier. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Margare Cundiff [mcundi...@comcast.net] Sent: Tuesday, July 25, 2017 4:09 PM To: Dave Cassel; MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Issue with Appservices after ML upgrade to ML9 Just curious - why was Application Builder junked? Is there anything in ML9 that is a replacement? Thanks, Morgan On July 23, 2017 at 12:44 PM Dave Cassel <david.cas...@marklogic.com> wrote: /Appservices was deprecated in ML8 and removed in ML9. See the Release Notes: http://docs.marklogic.com/guide/relnotes/chap4#id_54314. -- Dave Cassel, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of amit gope <amitgo...@gmail.com<mailto:amitgo...@gmail.com>> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Date: Sunday, July 23, 2017 at 9:47 AM To: "general@developer.marklogic.com<mailto:general@developer.marklogic.com>" <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Subject: [MarkLogic Dev General] Issue with Appservices after ML upgrade to ML9 Hi All, I am facing issue while trying to access the appservices URL http://localhost:8000/appservices. All other things working fine, please suggest what might be going wrong here. The Error that I am seeing is : Error An unexpected error has occurred. Check the error log or contact MarkLogic support<mailto:supp...@marklogic.com>. Message http://marklogic.com/xdmp/error error.xsd" xmlns:error="http://marklogic.com/xdmp/error; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;> REST-INVALIDREQUEST 1.0-ml Invalid request REST-INVALIDREQUEST: Invalid request: AppBuilder request no longer supported: /appservices false AppBuilder request no longer supported: /appservices /MarkLogic/rest-api/8000-rewriter.xml 1.0-ml Best Regards Amit ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Understanding the syntax
Hi, Prakash:
The % calls before the functions are XQuery function annotations (similar in
spirit to @ annotations in Java).
An example of a function reference would be cim:up#0 or cim:build#
If you provide the code that's reporting an error with the error, it should be
possible to identify what to tweak.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Prakashkumar Patel
[prakashkumar.pa...@tcs.com]
Sent: Wednesday, July 19, 2017 4:59 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Understanding the syntax
Thanks a lot Erik for the response.
I tried to see how that works from qconsole but it errored out. Could you
please create a quick example of its usage so that I can run from qconsole
directly and see how it works? This will be really helpful.
Additionally, I had one more spot that amazed me. In the code snippet below,
can you let me know if this is right syntax for functin declaration (of cours
eit is right coz it works in qconsole) and what is significance of
%cim:description.
declare %cim:description(1,"call")
function cim:build() as xs:string {
"Some string"
};
declare %cim:description(2,"One more call")
function cim:up() as xs:string {
"Different string"
};
Thanks,
Prakash
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Erik Hennum
<erik.hen...@marklogic.com>
Sent: Thursday, July 13, 2017 6:40 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Understanding the syntax
Hi, Prakash:
mo:cleanup-table#1 is a reference to the cleanup-table function in the
namespace for the mo prefix that takes a single parameter.
The code is putting the function reference in a map. Presumably, later code
gets the 'table' function reference from the map and calls the function,
passing a single parameter.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Prakashkumar Patel
[prakashkumar.pa...@tcs.com]
Sent: Thursday, July 13, 2017 5:38 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Understanding the syntax
Hi,
I was looking out into a codebase that needs enhancement and I came across
something as below. I am not much clear on what 'mo:cleanup-table#1' does. I
checked there is a function 'mo:cleanup-table' in the repository. Can someone
explain what does mo:cleanup-table#1 do?
let $_:=map:put($ht:override,'table',mo:cleanup-table#1)
Thanks,
Prakash
=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Docs REST api: Content Length and Keep Alive headers ommited on bigger files
Hi, Arunas: To confirm, no, there is no way to override the threshhold on closing the connection for large reads. The recommendation is to get large documents as single requests and to page over smaller documents in batches smaller than the threshhold. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Arunas [aruna...@gmail.com] Sent: Monday, July 17, 2017 10:05 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Docs REST api: Content Length and Keep Alive headers ommited on bigger files Dear All, We are using the documents management /v1/documents REST endpoint to retrieve documents in multipart/mixed content type. It works as expected with a retrieval of smaller documents and returns headers with a response like so: HTTP/1.1 200 OK vnd.marklogic.document-format: binary Content-type: multipart/mixed; boundary=ML_BOUNDARY_11510717316849072801 ML-Effective-Timestamp: 15003098405343020 Content-Length: 15684 Server: MarkLogic Connection: Keep-Alive Keep-Alive: timeout=500 However, on a retrieval of bigger size documents Content-Length & Keep-Alive headers are ommited, Connection header is set to "Close" like so: HTTP/1.1 200 OK vnd.marklogic.document-format: binary Content-type: multipart/mixed; boundary=ML_BOUNDARY_4647702192025548700 ML-Effective-Timestamp: 15003103202328660 Server: MarkLogic Connection: close I obviously done the homework and found somewhat helpful KB article here: https://help.marklogic.com/knowledgebase/article/View/465/0/content-length-keepalive-and-connection-close, mentioning that this is situational depending on the response size. However, it does not necessarilly tell me if there is any way to override the 1MB threshold. Is there any way to override this threshold as I fail to find one on admin interface or use a clever request headers to raise it? We are on v8.0-6.3 but I can also reproduce this on v9.0-1.1 MarkLogic Kind reagards, Arunas Vaitkus ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Understanding the syntax
Hi, Prakash: mo:cleanup-table#1 is a reference to the cleanup-table function in the namespace for the mo prefix that takes a single parameter. The code is putting the function reference in a map. Presumably, later code gets the 'table' function reference from the map and calls the function, passing a single parameter. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Prakashkumar Patel [prakashkumar.pa...@tcs.com] Sent: Thursday, July 13, 2017 5:38 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Understanding the syntax Hi, I was looking out into a codebase that needs enhancement and I came across something as below. I am not much clear on what 'mo:cleanup-table#1' does. I checked there is a function 'mo:cleanup-table' in the repository. Can someone explain what does mo:cleanup-table#1 do? let $_:=map:put($ht:override,'table',mo:cleanup-table#1) Thanks, Prakash =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] ML9 rest-api does not honor 'collection' parameter in search?
Hi, David: The collections should be OR related with one another. The list of collections as a whole should be AND related with the structured query. If that's not what you're seeing, please file a bug with a reproduction case through Support so it can be tracked and prioritized appropriately. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of David Mathei [david.mat...@lambdawerk.com] Sent: Monday, July 10, 2017 6:39 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] ML9 rest-api does not honor 'collection' parameter in search? Good afternoon, I observed that GETting /v1/search?collection== does not filter the results to the collection-members in ML9. /v1/search?collection= does what I'd expect from it, and moving the collection constraint into structuredQuery also works as advertised. In ML8 the first version above also observes teh query and the collection filter. The two databases are constructed with teh same properties. Is there an explanation for this behaviour or is this a bug? I've tested this with a minimal example which I have attached below. Thanks, David ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Using CURL to Test ML HTTP Processing
Hi, Eliot:
Try specifying the content-type.
I believe that, if a POST request doesn't specify the content-type,
curl defaults the content-type to application/x-www-form-urlencoded
(This convenience may or may not be seen as a feature.)
Regards,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Eliot Kimber
[ekim...@contrext.com]
Sent: Thursday, June 22, 2017 3:02 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Using CURL to Test ML HTTP Processing
I’m trying to understand the ML support for handling HTTP requests and I’m
trying to use CURL to test things just to learn. I’m getting an odd behavior
and I haven’t been able to figure out what I’m doing wrong from either the curl
info I can find or from the relevant ML docs.
Here’s my module:
xquery version "1.0-ml";
let $type:= xdmp:get-request-header('Content-Type')
let $field-names := xdmp:get-request-field-names()
return
This is test remote access
{$type}
{
for $name in $field-names
return {$name}
}
{
for $name in $field-names
return xdmp:get-request-field($name)}
I’m trying to use POST to send the data in a file to this module using the
–data-binary parameter:
curl -X POST --data-binary "file=@testfile.txt" --user ekimber:ekimber
http://anglia.corp.mitchellrepair.com:11984/test-remote-access.xqy
However, the response I get is:
This is test remote
accessapplication/x-www-form-urlencodedfile@testfile.txt
Note that the field value is the string “@testfile.txt”, not the content of the
file. This is the form of call that appears to be the correct way to associate
a field name with the data from a file.
If I leave off “file=” then the contents of testfile.txt become the field name:
curl -X POST --data-binary "@testfile.txt" --user ekimber:ekimber
http://anglia.corp.mitchellrepair.com:11984/test-remote-access.xqy
This is test remote
accessapplication/x-www-form-urlencodedThis
is the test
File. More text.
Which also seems wrong.
I must be doing something wrong, either on the CURL side or on the ML side but
I can’t figure out what it is. All the examples I could find in the ML docs use
direct form submission rather than CURL.
Thanks,
Eliot
--
Eliot Kimber
http://contrext.com
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Accessing properties of in-memory JS object in XQuery
Hi, Florent:
Much the way JavaScript objects turn into json:object values in XQuery,
JavaScript arrays turn into json:array values in XQuery.
When appropriate, you can use json:array-values() to convert a json:array
value to a sequence.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Florent Georges
[li...@fgeorges.org]
Sent: Friday, June 16, 2017 2:40 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Accessing properties of in-memory JS
object in XQuery
Hi Geert,
I was mot expecting Europe to still answer at this time, but that was counting
without you ;-) Thank you, it works indeed as a charm.
But it seems I oversimplified my example. The problem actually comes from the
fact that the value is an array of objecst, and just iterating over it like a
sequence of items does not work.
This modified example gives this extra complexity:
xdmp.xqueryEval(
`declare variable $obj external;
$obj ! map:get(., 'name')`,
{ obj: [{ name: 'name', title: 'title' }] });
Regards,
--
Florent Georges
H2O Consulting
http://h2o.consulting/
On 16 June 2017 at 23:18, Geert Josten wrote:
It is a json:object (the map:map specialization)..
Try:
xdmp.xqueryEval(
'declare variable $obj external; map:get($obj, "name")',
{ obj: {name: 'name', title: 'title' }});
Cheers,
Geert
On 6/16/17, 9:27 PM,
"general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
on behalf of
Florent Georges"
<general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
on behalf of
li...@fgeorges.org<mailto:li...@fgeorges.org>> wrote:
>Hi,
>
>I have an SJS script that calls a function from an XQuery library. It
>passes a JS object to the function. The function needs to access the
>value of one property of the object (in this case, a string).
>
>I can't find in the documentation how XQuery code can navigate through
>the properties of an in-memory JS object. Any idea?
>
>A self-contained example (my code require() an XQuery library and
>calls a function instead of using code evaluation, but the issue is
>the same):
>
>xdmp.eval(
> `declare variable $obj as external;
> $obj ! xs:string(name)`,
> { name: 'name', title: 'title' });
>
>I am using ML 9.
>
>Regards,
>
>--
>Florent Georges
>H2O Consulting
>http://h2o.consulting/
>___
>General mailing list
>General@developer.marklogic.com<mailto:General@developer.marklogic.com>
>Manage your subscription at:
>http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com<mailto:General@developer.marklogic.com>
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Optimising XQuery Timeouts
Hi, Basavaraj:
I suspect some of the xdmp:value() call got lost along the way, given the
string concatenation operator || with no strings.
It might speed things up to execute fn:subsequence(cts:search(...), $start,
$end - $start + 1), specifying the sort order with cts:index-order() arguments
to cts:search() instead of evaluating a for iteration.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Basavaraj Kalloli
[basavarajkall...@gmail.com]
Sent: Monday, May 29, 2017 11:25 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Optimising XQuery Timeouts
Hi Erik,
Here is the full extract of the xdmp:value() call:
xdmp:value((for $i in cts:search(/record[.//npg:Article], $final-query,
('unfiltered' ), 0.0) order by || $o1 || $o2 || return $i)[$start to $end])
Looks like that query is bringing in everything from the database and then
ordering. As I am typing I think it will be good for it to be ordered as part
of the search query and return only that many results as from $start to $end.
I will also look into the empty not-queries and post more details. The problem
we have been having is that this is legacy code which is making it difficult to
investigate.
Thanks for the pointers, hope the above code can possibly be the reason the
queries are running slow with concurrent requests?
Cheers,
Basavaraj
On Fri, May 26, 2017 at 3:58 PM, Erik Hennum
<erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote:
Hi, Basavaraj:
Can you show the full xdmp:value() call?
It looks like the FLWOR expression is ordering based on an XPath into each
retrieved document. It would be more efficient to order within the
cts:search() call based on range indexes.
The $final-query reported by query trace has some odd subqueries of the form:
cts:not-query(cts:or-query((), ()), 1)
An empty cts:or-query() is always false, making the negation is always true, so
these subqueries contribute no selectivity.
You might look at expressing the searchable expression criteria as a subquery
with the $final-query. Also, you could try to call cts:search() directly
instead of evalling it with xdmp:value().
Finally, if you're retrieving a large number of documents, the best practice is
to page over the result set.
Hoping that helps,
Erik Hennum
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>]
on behalf of Basavaraj Kalloli
[basavarajkall...@gmail.com<mailto:basavarajkall...@gmail.com>]
Sent: Friday, May 26, 2017 4:13 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Optimising XQuery Timeouts
Hi All,
I have been trying to resolve a couple of our queries which time out every
couple of hours. I believe this is down to the number of concurrent requests.
Things evaluated/investigated:
* I turned on the debug logs to see if there are any deadlocks
* I couldn't find any traces of locks
* Next I profiled my query and looks like there are 234 expressions and
normally it returns under 0.00738 secs
* Next I did try query-trace the output looks like:
xdmp:value("(for $i in cts:search(/record[.//npg:Article], $final-query,
('u...")
Analyzing path for search: fn:collection()/record[descendant::npg:Article]
Step 1 is searchable: fn:collection()
Step 2 is searchable: record[descendant::npg:Article]
Path is fully searchable.
Gathering constraints.
Step 2 predicate 1 contributed 1 constraint: descendant::npg:Article
Step 2 predicate 1 contributed 2 constraints: descendant::npg:Article
Step 2 contributed 3 constraints: record[descendant::npg:Article]
Comparison contributed string scatter value constraint: xdmp:collection =
("http://ns.nature.com/graphs/articles-labanimal;, "lab_animal",
"http://ns.nature.com/graphs/articles-nature;, ...)
Search query contributed 1 constraint:
cts:and-query((cts:not-query(cts:or-query((), ()), 1),
cts:collection-query(("http://ns.nature.com/graphs/articles-labanimal;,
"lab_animal", "http://ns.nature.com/graphs/articles-nature;, "journals_nature",
"http://ns.nature.com/graphs/articles-palgrave;, "journals_palgrave")),
cts:or-query(cts:field-value-query("doi", "10.1038/212441a0", ("lang=en"), 0),
()), cts:not-query(cts:or-query((), ()), 1), cts:not-query(cts:or-query((),
()), 1), cts:not-query(cts:or-query((), ()), 1)), ())
Order by clause contributed 1 range ordering constraint for $i: order by
xs:date($i/descendant::prism:publicationDate) descending
Ordering can be unfiltered.
Executing search.
Selected 1 fragment.
I dont see anything unsual with the output - no traces of unsearchable
expressions and la
Re: [MarkLogic Dev General] Optimising XQuery Timeouts
Hi, Basavaraj:
Can you show the full xdmp:value() call?
It looks like the FLWOR expression is ordering based on an XPath into each
retrieved document. It would be more efficient to order within the
cts:search() call based on range indexes.
The $final-query reported by query trace has some odd subqueries of the form:
cts:not-query(cts:or-query((), ()), 1)
An empty cts:or-query() is always false, making the negation is always true, so
these subqueries contribute no selectivity.
You might look at expressing the searchable expression criteria as a subquery
with the $final-query. Also, you could try to call cts:search() directly
instead of evalling it with xdmp:value().
Finally, if you're retrieving a large number of documents, the best practice is
to page over the result set.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Basavaraj Kalloli
[basavarajkall...@gmail.com]
Sent: Friday, May 26, 2017 4:13 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Optimising XQuery Timeouts
Hi All,
I have been trying to resolve a couple of our queries which time out every
couple of hours. I believe this is down to the number of concurrent requests.
Things evaluated/investigated:
* I turned on the debug logs to see if there are any deadlocks
* I couldn't find any traces of locks
* Next I profiled my query and looks like there are 234 expressions and
normally it returns under 0.00738 secs
* Next I did try query-trace the output looks like:
xdmp:value("(for $i in cts:search(/record[.//npg:Article], $final-query,
('u...")
Analyzing path for search: fn:collection()/record[descendant::npg:Article]
Step 1 is searchable: fn:collection()
Step 2 is searchable: record[descendant::npg:Article]
Path is fully searchable.
Gathering constraints.
Step 2 predicate 1 contributed 1 constraint: descendant::npg:Article
Step 2 predicate 1 contributed 2 constraints: descendant::npg:Article
Step 2 contributed 3 constraints: record[descendant::npg:Article]
Comparison contributed string scatter value constraint: xdmp:collection =
("http://ns.nature.com/graphs/articles-labanimal;, "lab_animal",
"http://ns.nature.com/graphs/articles-nature;, ...)
Search query contributed 1 constraint:
cts:and-query((cts:not-query(cts:or-query((), ()), 1),
cts:collection-query(("http://ns.nature.com/graphs/articles-labanimal;,
"lab_animal", "http://ns.nature.com/graphs/articles-nature;, "journals_nature",
"http://ns.nature.com/graphs/articles-palgrave;, "journals_palgrave")),
cts:or-query(cts:field-value-query("doi", "10.1038/212441a0", ("lang=en"), 0),
()), cts:not-query(cts:or-query((), ()), 1), cts:not-query(cts:or-query((),
()), 1), cts:not-query(cts:or-query((), ()), 1)), ())
Order by clause contributed 1 range ordering constraint for $i: order by
xs:date($i/descendant::prism:publicationDate) descending
Ordering can be unfiltered.
Executing search.
Selected 1 fragment.
I dont see anything unsual with the output - no traces of unsearchable
expressions and lack of indexes.
* I did try query-meters
http://marklogic.com/xdmp/query-meters
query-meters.xsd" xmlns:qm="http://marklogic.com/xdmp/query-meters;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
PT0.003874S
0
150
0
0
0
0
0
0
1
0
0
0
0
3
35
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
record
1
0
/n5061/xml/212441a0.xml
1
0
#removed#
1
PT0.000921S
#removed#
1
PT0.000856S
The only thing that strikes me is that there are value cache misses, I dont
know if I can do anything for it or anything else I could try. I am running out
of ideas so it would be great if anyone can share some thoughts/pointers.
Thanks,
Basavaraj Kalloli
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Processing Large Number of Docs to Get Statistics
Hi, Eliot:
On reflection, let me retract the range index suggestion. I wasn't considering
the domain implied by the element names -- it would never make sense
to blow out a range index with the value of all of the paragraphs.
The TDE suggestion for MarkLogic 9 would still work, however, because you
could have an xs:short column with a value of 1 for every paragraph.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Erik Hennum
[erik.hen...@marklogic.com]
Sent: Tuesday, May 23, 2017 6:21 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Processing Large Number of Docs to Get
Statistics
Hi, Eliot:
One alternative to Geert's good suggestion -- if and only if the number
of element names is small and you can create range indexes on them:
* add an element attribute range index on Article/@id
* add an element range index on p
* execute a cts:value-tuples() call with the constraining element query and
directory query
* iterate over the tuples, incrementing the value of the id in a map
* remove the range index on p
In MarkLogic 9, that approach gets simpler. You can just use TDE
to project rows with columns for the id and element, group on
the id column, and count the rows in the group.
Hoping that's useful (and salutations in passing),
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Geert Josten
[geert.jos...@marklogic.com]
Sent: Tuesday, May 23, 2017 12:53 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Processing Large Number of Docs to Get
Statistics
Hi Eliot,
I¹d consider using taskbot
(http://registry.demo.marklogic.com/package/taskbot), and using that in
combination with either $tb:OPTIONS-SYNC or $tb:OPTIONS-SYNC-UPDATE. It
will make optimal use of the TaskServer of the host on which you initiate
the call. It doesn¹t scale endlessly, but it batches up the work
automatically for you, and will get you a lot further fairly easily..
Cheers,
Geert
On 5/23/17, 5:43 AM, "general-boun...@developer.marklogic.com on behalf of
Eliot Kimber" <general-boun...@developer.marklogic.com on behalf of
ekim...@contrext.com> wrote:
>I haven¹t yet seen anything in the docs that directly address what I¹m
>trying to do and suspect I¹m simply missing some ML basics or just going
>about things the wrong way.
>
>I have a corpus of several hundred thousand docs (but could be millions,
>of course), where each doc is an average of 200K and several thousand
>elements.
>
>I want to analyze the corpus to get details about the number of specific
>subelements within each document, e.g.:
>
>
>for $article in cts:search(/Article, cts:directory-query("/Default/",
>"infinity"))[$start to $end]
> return paras=²{count($article//p}²/>
>
>I¹m running this as a query from Oxygen (so I can capture the results
>locally so I can do other stuff with them).
>
>On the server I¹m using I blow the expanded tree cache if I try to
>request more than about 20,000 docs.
>
>Is there a way to do this kind of processing over an arbitrarily large
>set *and* get the results back from a single query request?
>
>I think the only solution is to write the results to back to the database
>and then fetch that as the last thing but I was hoping there was
>something simpler.
>
>Have I missed an obvious solution?
>
>Thanks,
>
>Eliot
>
>--
>Eliot Kimber
>http://contrext.com
>
>
>
>
>___
>General mailing list
>General@developer.marklogic.com
>Manage your subscription at:
>http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Processing Large Number of Docs to Get Statistics
Hi, Eliot:
One alternative to Geert's good suggestion -- if and only if the number
of element names is small and you can create range indexes on them:
* add an element attribute range index on Article/@id
* add an element range index on p
* execute a cts:value-tuples() call with the constraining element query and
directory query
* iterate over the tuples, incrementing the value of the id in a map
* remove the range index on p
In MarkLogic 9, that approach gets simpler. You can just use TDE
to project rows with columns for the id and element, group on
the id column, and count the rows in the group.
Hoping that's useful (and salutations in passing),
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Geert Josten
[geert.jos...@marklogic.com]
Sent: Tuesday, May 23, 2017 12:53 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Processing Large Number of Docs to Get
Statistics
Hi Eliot,
I¹d consider using taskbot
(http://registry.demo.marklogic.com/package/taskbot), and using that in
combination with either $tb:OPTIONS-SYNC or $tb:OPTIONS-SYNC-UPDATE. It
will make optimal use of the TaskServer of the host on which you initiate
the call. It doesn¹t scale endlessly, but it batches up the work
automatically for you, and will get you a lot further fairly easily..
Cheers,
Geert
On 5/23/17, 5:43 AM, "general-boun...@developer.marklogic.com on behalf of
Eliot Kimber" <general-boun...@developer.marklogic.com on behalf of
ekim...@contrext.com> wrote:
>I haven¹t yet seen anything in the docs that directly address what I¹m
>trying to do and suspect I¹m simply missing some ML basics or just going
>about things the wrong way.
>
>I have a corpus of several hundred thousand docs (but could be millions,
>of course), where each doc is an average of 200K and several thousand
>elements.
>
>I want to analyze the corpus to get details about the number of specific
>subelements within each document, e.g.:
>
>
>for $article in cts:search(/Article, cts:directory-query("/Default/",
>"infinity"))[$start to $end]
> return paras=²{count($article//p}²/>
>
>I¹m running this as a query from Oxygen (so I can capture the results
>locally so I can do other stuff with them).
>
>On the server I¹m using I blow the expanded tree cache if I try to
>request more than about 20,000 docs.
>
>Is there a way to do this kind of processing over an arbitrarily large
>set *and* get the results back from a single query request?
>
>I think the only solution is to write the results to back to the database
>and then fetch that as the last thing but I was hoping there was
>something simpler.
>
>Have I missed an obvious solution?
>
>Thanks,
>
>Eliot
>
>--
>Eliot Kimber
>http://contrext.com
>
>
>
>
>___
>General mailing list
>General@developer.marklogic.com
>Manage your subscription at:
>http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] custom Rest rewriter
Hi, Rajesh and David: While it is possible to modify the rewriter declarations distributed with the REST API, that would be modifying the implementation of the REST API. The rewriter declarations are internal to the REST API, not part of the documented interface. Such changes would fall in the same category as modifying the XQuery and JavaScript modules that implement the REST API. The distributed implementation can change in any release, even a point release. Also, MarkLogic cannot support the REST API if the implementation has been changed. In other words, you should be aware that you are taking a risk. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of David Lee [david@marklogic.com] Sent: Wednesday, April 19, 2017 7:16 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] custom Rest rewriter If I understand your architecture correctly, you have 2 'parts' of an app in one context. Is this a JavaScript browser app ? You want your 'ajax' part to 'piggy back' on the credentials the user entered to login ?The ( stored in the browser as cookies ) The reference to avoiding passing credentials and having to use the same port seems to indicate that. If so, then it will be impossible to securely protect/restrict access to the REST API from all parts of the app, by implication of sharing credentials. So your 2 goals cannot be simultaneously achieved. So focusing on how to simply expose your app in addition to REST -- very easy. Create a new REST application for your combined app. Then edit the generated rewriter.xml (It will be in the Modules DB, the path to it in the App configuration in the element in groups.xml, or the corresponding "Rewriter URI" in the HTTP App config page on 8001. It will look something like this: http://marklogic.com/xdmp/rewriter;> $0 -->>> ADD HERE Insert your own rules before he REST Client API rules. A minimal example might be /myapp/myapi.xqy That would dispatch any URL like/myapp/myapi/myfunction?query=abc to invoke the module /myapp/myapi.xqy You can somewhat constrain 'external' use of the REST API by adding a custom Header or query param to your internal calls then matching those and dispatching to an error page if they don’t exist. A minimal example. Insert just before the Then at the end a That will only allow REST calls which include the "X-INTERNAL" HTTP header. How to do that will depend on what libraries your using. Details on the XML rewriter : https://docs.marklogic.com/guide/app-dev/XMLrewriter From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Rajesh Kumar Sent: Wednesday, April 19, 2017 6:59 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] custom Rest rewriter Thanks David, I can maintain two different ports as app and rest servers. But every time I want to call REST API i have to pass credentials for the request from app server , which I want to avoid by having both in same port. I want to access endpoints ( app API or REST API) whenever header has token info except login page. I tried this using roxy hybrid and rest type, but didn't succeed. Can you help me by sharing any rewriter which I can use to achieve this functionality. Thanks & Regards, Rajesh ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] How to use uri-match in the java api?
Hi, Andreas:
The REST API doesn't support the cts:*-match() functions out of the box.
If you really need to match values in a range index, you can implement a
resource service extension that takes the appropriate parameters and makes the
call.
To perform well at scale for a range index with many values (such as the uri
lexicon), you might think about passing in or constructing a constraining query
so the comparison is only on some values instead of all values in the range
index.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Andreas Hubmer
[andreas.hub...@ebcont.com]
Sent: Monday, April 03, 2017 1:09 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] How to use uri-match in the java api?
Hi,
Using queryManager.values I can invoke cts:uris, but how could I invoke
cts:uri-match with a wildcarded uri through the Java API?
Thanks,
Andreas
2017-03-31 17:22 GMT+02:00 Justin Makeig
<justin.mak...@marklogic.com<mailto:justin.mak...@marklogic.com>>:
Apologies, I read too fast. You were asking about "Java", not "JavaScript".
(Car is to carpet as Java is to JavaScript.)
You'll need to specify a URI values constraint.
http://marklogic.com/appservices/search;>
…
And then queryManager.values("uris", new JacksonHandle()) to use it. (Hat tip:
Sam Mefford.)
Take a look at <http://docs.marklogic.com/guide/java/searches#id_76144> and
<http://docs.marklogic.com/guide/search-dev/appendixa#id_46397>.
Justin
> On Mar 31, 2017, at 8:08 AM, Justin Makeig
> <justin.mak...@marklogic.com<mailto:justin.mak...@marklogic.com>> wrote:
>
> <https://docs.marklogic.com/cts.uriMatch>
>
> In general, the built-in functions map one-to-one between JavaScript and
> XQuery. (There are exceptions.) Because hyphens aren't friendly in JavaScript
> property names, we do a kabab-case to camel-case translation for all of the
> names in JavaScript, i.e. cts:uri-match() in XQuery is cts.uriMatch() in
> JavaScript.
>
> Justin
>
>> On Mar 31, 2017, at 3:31 AM, Andreas Felix
>> <andreas.fe...@ebcont.com<mailto:andreas.fe...@ebcont.com>> wrote:
>>
>> Hi,
>> i need to fetch uris by wildcard, eg. foo*
>> In xquery i use the cts:uri-match function for this.
>> Does anybody know how so solve this with the Java-API?
>>
>> Regards
>> Andreas
>> ___
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Regular Expressions
Hi, Oleksii:
Regarding question 2, aside from a few edge cases,
the MarkLogic libraries have the same core implementation
with JavaScript and XQuery interfaces.
The core behavior of functions in the MarkLogic libraries are
(in almost every case) consistent across environments.
If you are working in JavaScript and the regex implementation
from v8 is a good fit for your requirements, you should take
advantage of JavaScript regex objects and methods.
Hoping that clarifies,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Sewell, David R. (drs2n)
[dsew...@virginia.edu]
Sent: Wednesday, March 22, 2017 1:40 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Regular Expressions
I’m not sure what the answer is to question 2, but for question 1, the answer
is that MarkLogic’s implementation of XPath doesn’t support the \b character
escape because it is not included in the XPath specification for regular
expressions, which itself is based on "XML Schema Part 2: Datatypes Second
Edition”. The only single-character escapes are these:
https://www.w3.org/TR/xmlschema-2/#nt-charClassEsc
Some XSLT and XQuery processors support extended regular expressions as a
proprietary feature (for example, Saxon has a semi-documented extension that
allows full Java regex), but MarkLogic doesn’t (unless there is undocumented
support that I don’t know about).
David
On Mar 22, 2017, at 3:55 PM, Oleksii Segeda
<oseg...@worldbankgroup.org<mailto:oseg...@worldbankgroup.org>> wrote:
Hi everyone,
Quick questions regarding regex in ML:
1. What’s ML alternative to word boundaries \b? Seems that
fn:analyze-string doesn’t support this special character.
2. Does JS version of this function (fn.analyzeString) use JS regex
engine? If so, why it gives me error for fn.analyzeString("foo bar bar",
"\\b(bar)\\b") ?
Regards,
Oleksii Segeda
IT Analyst
Information and Technology Solutions
___
General mailing list
General@developer.marklogic.com<mailto:General@developer.marklogic.com>
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Finding target geocodes using REST API
Hi, Shiv:
Yes, the REST API supports geospatial queries through the Search API vocabulary
on the /v1/search endpoint.
See the reference starting at:
http://docs.marklogic.com/guide/search-dev/structured-query#id_87280
Also:
http://docs.marklogic.com/guide/rest-dev/search#id_15881
Hoping that helps,
Erik Hennum
From: Shiv Shankar [shiv.shivshan...@gmail.com]
Sent: Friday, February 24, 2017 11:05 AM
To: MarkLogic Developer Discussion; Erik Hennum
Subject: Re: Finding target geocodes using REST API
Hi Erik,
Thanks for the quick update. Is there a way I can enable below geospatial
document vocabularies using REST?
var lat=;
var long=;
cts.search(cts.andQuery([
cts.jsonPropertyPairGeospatialQuery("geoCoordinates",
"lat", "lng", cts.circle(20, cts.point(lat,
long))),cts.collectionQuery(["ADDRESS_COLLECTION"])]));
Thanks
Shan.
---
Hi, Shiv:
QBE doesn't support geospatial document vocabularies at present.
You may want to raise this issue through your support contact.
Erik Hennum
On Fri, Feb 24, 2017 at 11:09 AM, Shiv Shankar
<shiv.shivshan...@gmail.com<mailto:shiv.shivshan...@gmail.com>> wrote:
Hi,
I could able to achieve finding the nearest addresses based on the starting geo
codes in a circle with in the given distance using SJava Script API in the
qconsole and I wanted to achive the same using REST. I see very less
documentation and I found these part of Options in a QBE. Any help on this
please?
My address structure
"address":{
"fullAddress":"Unknown address",
"geoCoordinates" :
{
"lat": 41.335747
"lng:-72.06697
}
http://localhost:8081/LATEST/qbe?query={<http://localhost:8081/LATEST/qbe?query=%7B>"$query":{}}
{
"parent-property": "geoCoordinates",
"lat-property": "lat",
"lon-property": "lng",
"geo-option": "5 miles"
}
Thanks
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Finding target geocodes using REST API
Hi, Shiv:
QBE doesn't support geospatial document vocabularies at present.
You may want to raise this issue through your support contact.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Friday, February 24, 2017 8:09 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Finding target geocodes using REST API
Hi,
I could able to achieve finding the nearest addresses based on the starting geo
codes in a circle with in the given distance using SJava Script API in the
qconsole and I wanted to achive the same using REST. I see very less
documentation and I found these part of Options in a QBE. Any help on this
please?
My address structure
"address":{
"fullAddress":"Unknown address",
"geoCoordinates" :
{
"lat": 41.335747
"lng:-72.06697
}
http://localhost:8081/LATEST/qbe?query={"$query":{}}
{
"parent-property": "geoCoordinates",
"lat-property": "lat",
"lon-property": "lng",
"geo-option": "5 miles"
}
Thanks
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] transform
Hi, Oleskii:
It sounds like some server-side code is trying to read a document twice.
Does the same request work if you omit the transform?
What does the transform look like?
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda
[oseg...@worldbankgroup.org]
Sent: Wednesday, February 15, 2017 3:20 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] transform
Hi all,
When I’m trying to use transforms with DocumentManager (Java API), MarkLogic
gives me this error:
XDMP-MULTIPART-DONE: xdmp:document-load("rest::", /original/4ea94612..) --
All parts are already processed
Please advise.
Regards,
Oleksii Segeda
IT Analyst
Information and Technology Solutions
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] USing REST api to retrieve value for an element
Hi, Bhushan:
It sounds like it would meet your requirements to create a range index on NAME
and use the /v1/values endpoint to retrieve the values from the index:
http://docs.marklogic.com/guide/rest-dev/search#id_24780
http://docs.marklogic.com/REST/POST/v1/values/[name]
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Bhushan Suryawanshi
[bsuryawan...@xenomorph.com]
Sent: Tuesday, February 07, 2017 11:40 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] USing REST api to retrieve value for an element
Hello Team,
We are using MarkLogic 8.0 using REST API.
We have around 200,000 documents inside a collection. Each document has the
following structure
1022
< IDENTIFIER2>KP
SFW
…
One Requirement:
1) Fetch value for an element for all the documents inside a collection.
I have tried below solution using Xquery:
{
let $a := collection("Equity")//*
for $name in distinct-values($a/NAME)
return
{ $name }
}
But the above Xquery takes a long time.
Is there any way where we can solve the above requirements using REST API.
Thanks in advance.
Bhushan Suryawanshi
Developer
t: +1-212-401-7894
www.xenomorph.com<http://www.xenomorph.com/?mail> |
blog<http://xenomorph.typepad.com/?mail> | twitter<http://goo.gl/oaBWO> |
linkedin<http://goo.gl/rdi8W>
This email is confidential and is intended only for the addressee. If you are
not the intended recipient, please note that any dissemination, distribution or
copying of this email is strictly prohibited. Any and all estimates, proposals,
quotes and fees contained herein are for discussion purposes only and are not
contractually binding unless executed in accordance with the standard practices
of the Company. Attachments to this message have been virus checked but no
guarantee can be made that any attachment is virus free. Please notify us
immediately of any problem.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Custom search grammar
Hi, Will: The Search API and its dynamic parser was introduced in MarkLogic 4.0 (or maybe 4.1). A lot has changed since then. MarkLogic 8 made it possible for MarkLogic customers to take advantage of the JavaScript ecosystem. Part of the motivation for taking on the work of building v8 into the server was to give customers access to a broad set of tools and resources that no single company could possibly provide in isolation. That's especially true for customers who are starting new projects. In particular, generating parsers for textual grammars is a general problem (not specific to MarkLogic). If tools that solve the problem exist in the ecosystem, MarkLogic isn't adding value by creating equivalents. Instead, we add value by working on the tools and capabilities that are missing. It makes sense that "glue" examples showing how to use parser generators with MarkLogic would help bridge the gap; I'll raise that suggestion internally. I realize the explanation above might not be convincing but hope that's at least useful in understanding the rationale for suggesting JavaScript parser generators. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Will Thompson [wthomp...@oconnors.com] Sent: Wednesday, February 01, 2017 12:34 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Custom search grammar Hi Oleksi, If it seems odd to you that MarkLogic continues to pressure you *not* to use the API they built, you are not alone. Erik, I don't think it's reasonable to suggest to customers only needing to extend or replace a small portion of Search API that they rewrite significant parts of its functionality using 3rd-party tools (some of which haven't been updated for years) instead of using the clearly documented extensibility hooks of the ML-provided API. If MarkLogic wants to push customers with more complex Search API needs in a different direction, that's fine, but it would be a lot more palatable if ML actually did some of the legwork upfront - provide code examples, blogs, documentation, etc. - to demonstrate how that should be done correctly. At a minimum it's confusing to be told not to use the the provided tools and frustrating that the alternatives suggested require a lot more work and uncertainty. Ideally, if ML doesn't want customers using parts of the Search API, they should just build a replacement that they are willing to endorse (maybe even using one of the JS parsers you recommend). -Will > On Feb 1, 2017, at 11:58 AM, Erik Hennum <erik.hen...@marklogic.com> wrote: > > Hi, Oleksii: > > To be clear, we discourage use of custom grammars. > > Besides the JavaScript parser generators that I mentioned previously, you > might also consider the XQuery approach demonstrated in: > >https://github.com/mblakele/xqysp > > These approaches will support more flexible and performant parsers than the > dynamic grammar of the Search API. > > If you have a requirement that can be addressed only by a custom grammar and > not by one of these approaches, please open a support ticket. > > > Erik Hennum > > > From: general-boun...@developer.marklogic.com > [general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda > [oseg...@worldbankgroup.org] > Sent: Wednesday, February 01, 2017 7:28 AM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Custom search grammar > > Hi Erik, > > Did you figure out how to extend the grammar? > > Regards, > Oleksii Segeda > IT Analyst > Information and Technology Solutions > > > -Original Message- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Oleksii Segeda > Sent: Monday, January 30, 2017 3:09 PM > To: MarkLogic Developer Discussion <general@developer.marklogic.com> > Subject: Re: [MarkLogic Dev General] Custom search grammar > > Hi Erik, > > Yes, that's is desired behavior. > > Ideally, I would like to avoid custom constraints, simply because search > grammar looks cleaner in the search box. In addition, some of our users are > already familiar with simple search operators like AND, OR, so BOOST won't > look like an alien to them. > > I guess a postprocessing can be used as you suggested, however I'm interested > in custom search grammar, because I may need to extend it more in the future. > > Thank you, > Oleksii Segeda > IT Analyst > Information and Technology Solutions > > > -Original Message- > From: general-boun...@developer.marklogic.com > [mailto:general-boun...@developer.marklogic.com] On Behalf Of Erik Hennum > Sent: Monda
Re: [MarkLogic Dev General] Custom search grammar
Hi, Oleksii:
To be clear, we discourage use of custom grammars.
Besides the JavaScript parser generators that I mentioned previously, you might
also consider the XQuery approach demonstrated in:
https://github.com/mblakele/xqysp
These approaches will support more flexible and performant parsers than the
dynamic grammar of the Search API.
If you have a requirement that can be addressed only by a custom grammar and
not by one of these approaches, please open a support ticket.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda
[oseg...@worldbankgroup.org]
Sent: Wednesday, February 01, 2017 7:28 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Custom search grammar
Hi Erik,
Did you figure out how to extend the grammar?
Regards,
Oleksii Segeda
IT Analyst
Information and Technology Solutions
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Oleksii Segeda
Sent: Monday, January 30, 2017 3:09 PM
To: MarkLogic Developer Discussion <general@developer.marklogic.com>
Subject: Re: [MarkLogic Dev General] Custom search grammar
Hi Erik,
Yes, that's is desired behavior.
Ideally, I would like to avoid custom constraints, simply because search
grammar looks cleaner in the search box. In addition, some of our users are
already familiar with simple search operators like AND, OR, so BOOST won't look
like an alien to them.
I guess a postprocessing can be used as you suggested, however I'm interested
in custom search grammar, because I may need to extend it more in the future.
Thank you,
Oleksii Segeda
IT Analyst
Information and Technology Solutions
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of Erik Hennum
Sent: Monday, January 30, 2017 2:42 PM
To: MarkLogic Developer Discussion <general@developer.marklogic.com>
Subject: Re: [MarkLogic Dev General] Custom search grammar
Hi, Oleksii:
Thanks for providing more detail.
Just to confirm, is it clear that, in a boost query, the right-hand
term is optional? Documents with only the left-hand term will still
appear in the results though with less relevance than documents
that have both terms.
By contrast, AND-related terms are both required and both
contribute to relevance.
Anyway, to increase weight, one approach would be to define a tag
for a quoted phrase and pass the phrase to a Search API custom
constraint or to cts:parse() with a binding to a query generator function:
http://docs.marklogic.com/guide/search-dev/cts_query#id_13456
The custom code could then tokenize the phrase and combine the
terms with a boost-query or and-query, adding appropriate weight.
Another approach would be to do postprocessing of the query tree
returned by cts:parse() or search:parse() to replace the default
boost-query or and-query with a query that has more weight.
In either approach, you would then search on the query.
I mention cts:parse() because it parses query text more quickly
than search:parse()
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda
[oseg...@worldbankgroup.org]
Sent: Monday, January 30, 2017 10:55 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Custom search grammar
Hi Erik,
I'm trying to boost some parts of search query. For example, if user types
`trade BOOST water`, I want documents with the word "water" to be higher in the
results.
cts:boost-query seems to be a perfect fit, but the default BOOST doesn't let
you specify weights.
My ultimate goal is to convert `trade BOOST water` to something like this:
cts:boost-query(cts:word-query("trade"), cts:word-query("water", (), 10.0) )
Regards,
Oleksii Segeda
IT Analyst
Information and Technology Solutions
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of
general-requ...@developer.marklogic.com
Sent: Monday, January 30, 2017 1:08 PM
To: general@developer.marklogic.com
Subject: General Digest, Vol 151, Issue 42
Send General mailing list submissions to
general@developer.marklogic.com
To subscribe or unsubscribe via the World Wide Web, visit
http://developer.marklogic.com/mailman/listinfo/general
or, via email, send a message with subject or body 'help' to
general-requ...@developer.marklogic.com
You can reach the person managing the list at
general-ow...@developer.marklogic.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of General digest..."
Today's Topics:
1. Custom search gram
Re: [MarkLogic Dev General] Custom search grammar
Hi, Oleksii:
Thanks for providing more detail.
Just to confirm, is it clear that, in a boost query, the right-hand
term is optional? Documents with only the left-hand term will still
appear in the results though with less relevance than documents
that have both terms.
By contrast, AND-related terms are both required and both
contribute to relevance.
Anyway, to increase weight, one approach would be to define a tag
for a quoted phrase and pass the phrase to a Search API custom
constraint or to cts:parse() with a binding to a query generator function:
http://docs.marklogic.com/guide/search-dev/cts_query#id_13456
The custom code could then tokenize the phrase and combine the
terms with a boost-query or and-query, adding appropriate weight.
Another approach would be to do postprocessing of the query tree
returned by cts:parse() or search:parse() to replace the default
boost-query or and-query with a query that has more weight.
In either approach, you would then search on the query.
I mention cts:parse() because it parses query text more quickly
than search:parse()
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda
[oseg...@worldbankgroup.org]
Sent: Monday, January 30, 2017 10:55 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Custom search grammar
Hi Erik,
I'm trying to boost some parts of search query. For example, if user types
`trade BOOST water`, I want documents with the word "water" to be higher in the
results.
cts:boost-query seems to be a perfect fit, but the default BOOST doesn't let
you specify weights.
My ultimate goal is to convert `trade BOOST water` to something like this:
cts:boost-query(cts:word-query("trade"), cts:word-query("water", (), 10.0) )
Regards,
Oleksii Segeda
IT Analyst
Information and Technology Solutions
-Original Message-
From: general-boun...@developer.marklogic.com
[mailto:general-boun...@developer.marklogic.com] On Behalf Of
general-requ...@developer.marklogic.com
Sent: Monday, January 30, 2017 1:08 PM
To: general@developer.marklogic.com
Subject: General Digest, Vol 151, Issue 42
Send General mailing list submissions to
general@developer.marklogic.com
To subscribe or unsubscribe via the World Wide Web, visit
http://developer.marklogic.com/mailman/listinfo/general
or, via email, send a message with subject or body 'help' to
general-requ...@developer.marklogic.com
You can reach the person managing the list at
general-ow...@developer.marklogic.com
When replying, please edit your Subject line so it is more specific
than "Re: Contents of General digest..."
Today's Topics:
1. Custom search grammar (Oleksii Segeda)
2. Re: Custom search grammar (Erik Hennum)
--
Message: 1
Date: Mon, 30 Jan 2017 16:51:26 +
From: Oleksii Segeda <oseg...@worldbankgroup.org>
Subject: [MarkLogic Dev General] Custom search grammar
To: "general@developer.marklogic.com"
<general@developer.marklogic.com>
Message-ID:
<bn1pr0101mb0769b9cdcd5e7697ace8381bcb...@bn1pr0101mb0769.prod.exchangelabs.com>
Content-Type: text/plain; charset="us-ascii"
Hi there,
I'm trying to declare a custom search grammar. I declared a custom function via
search options, which supposed to parse "BOOST" keyword:
http://worldbankgroup.org/search/grammar; at="/lib/grammar-boost.xqy"
tokenize="word">BOOST
I declared this function and just copied existing implementation from
impl:joiner-boost function in /MarkLogic/appservices/search/search-impl.xqy :
declare function grammar:custom-boost($ps as map:map, $left as element()?,
$opts as element()?) as schema-element(cts:query) {
let $symbol := impl:symbol-lookup($ps)
let $_ := tdop:advance($ps)
let $expr1 := tdop:expression($ps, $symbol/@strength)
return
if (empty($left))
then ($left, impl:msg($ps, ))
else
element { xs:QName($symbol/@element) } {
attribute qtextjoin {concat($symbol/string())},
attribute strength {$symbol/@strength},
attribute qtextgroup {
impl:opts($ps)/opt:grammar/opt:starter[@apply eq "grouping"]/(string(),
@delimiter/string()) },
for $opt in
$symbol/@options/tokenize(normalize-space(.)<mailto:$symbol/@options/tokenize(normalize-space(.)>,
"\s") return {$opt},
element cts:matching-query {
attribute qtextref { "schema-element(cts:query)" },
$left },
element cts:boosting-query {
attribute qtextref { "schema-element(cts:query)" },
$exp
Re: [MarkLogic Dev General] Custom search grammar
Hi, Oleksii:
Can you explain what you are trying to accomplish?
There may be better ways of doing the same thing than creating a custom
grammar, which is really a tool of last resort.
For instance, a custom constraint can map a term to a custom query.
For other cases, it's often useful to do postprocessing on the generated query.
If a custom grammar really is unavoidable, in many cases a special-purpose
third-party parsing tool may provide a faster and more flexible alternative to
the limited custom grammar in the Search API.
For instance, the Jison.js and Peg.js parsers work with server-side JavaScript.
(A nearly.js parser is also available, though I've heard no reports about it
yet.)
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Oleksii Segeda
[oseg...@worldbankgroup.org]
Sent: Monday, January 30, 2017 8:51 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] Custom search grammar
Hi there,
I’m trying to declare a custom search grammar. I declared a custom function via
search options, which supposed to parse “BOOST” keyword:
http://worldbankgroup.org/search/grammar; at="/lib/grammar-boost.xqy"
tokenize="word">BOOST
I declared this function and just copied existing implementation from
impl:joiner-boost function in /MarkLogic/appservices/search/search-impl.xqy :
declare function grammar:custom-boost($ps as map:map, $left as element()?,
$opts as element()?) as schema-element(cts:query) {
let $symbol := impl:symbol-lookup($ps)
let $_ := tdop:advance($ps)
let $expr1 := tdop:expression($ps, $symbol/@strength)
return
if (empty($left))
then ($left, impl:msg($ps, ))
else
element { xs:QName($symbol/@element) } {
attribute qtextjoin {concat($symbol/string())},
attribute strength {$symbol/@strength},
attribute qtextgroup {
impl:opts($ps)/opt:grammar/opt:starter[@apply eq "grouping"]/(string(),
@delimiter/string()) },
for $opt in
$symbol/@options/tokenize(normalize-space(.)<mailto:$symbol/@options/tokenize(normalize-space(.)>,
"\s") return {$opt},
element cts:matching-query {
attribute qtextref { "schema-element(cts:query)" },
$left },
element cts:boosting-query {
attribute qtextref { "schema-element(cts:query)" },
$expr1 }
}
};
Unfortunately this doesn’t work, because for some reason impl:symbol-lookup
returns an empty sequence.
Any ideas what went wrong here?
Oleksii Segeda
IT Analyst
Information and Technology Solutions
[http://siteresources.worldbank.org/NEWS/Images/spacer.png]
[http://siteresources.worldbank.org/NEWS/Images/WBG_Information_and_Technology_Solutions.png]
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Question about patching a JSON document
Hi, Desmond:
Thanks for the correction.
The actual limitation is that a PATCH context or select path can't
match an unnamed node (basically, an object or array within an
array).
Apologies for the misinformation, glad you had success.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Desmond Lim
[desmond@senseinfosys.com]
Sent: Wednesday, January 25, 2017 5:38 PM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Question about patching a JSON document
Hi Erik,
Thanks for the advice.
I've used the "array-node('tags')" and it works with insert.
As for the other things, we might need to rethink our strategy.
Thanks.
Desmond
Date: Wed, 25 Jan 2017 17:51:29 +0000
From: Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>>
Subject: Re: [MarkLogic Dev General] Question about patching a JSON
document
To: MarkLogic Developer Discussion
<general@developer.marklogic.com<mailto:general@developer.marklogic.com>>
Message-ID:
<dfdf2fd50bf5aa42adaf93ff2e3ca1850bd7c...@exchg10-be01.marklogic.com<mailto:dfdf2fd50bf5aa42adaf93ff2e3ca1850bd7c...@exchg10-be01.marklogic.com>>
Content-Type: text/plain; charset="iso-8859-1"
Hi, Desmond:
The PATCH REST API request has an unfortunate limitation that prevents
clients from specifying an array as the context for a request.
The appropriate XPath expression would be:
/resource/array-node('tags')
The REST API uses cts.validIndexPath() to sanitize XPaths, which ensures
security without sacrificing performance but prohibits some useful paths
including the one above.
There is an RFE to address the issue, but it is unlikely to be implemented
in ML9.
In the meantime, the alternatives are either to read the document and modify
it on the client or to install server-side code to implement the change.
Erik Hennum
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>]
on behalf of Desmond Lim
[desmond@senseinfosys.com<mailto:desmond@senseinfosys.com>]
Sent: Wednesday, January 25, 2017 12:11 AM
To: general@developer.marklogic.com<mailto:general@developer.marklogic.com>
Subject: [MarkLogic Dev General] Question about patching a JSON document
Hi there,
I have this document in Marklogic:
{
"resource":
{
"name": "hrRFIrq2JzpiXmc",
"url": "k4llhWiubj2SHtl",
"importer": "UXwZbxVCOfNFeST",
"text_context": "qerXqhThyAtdQkNhXZ2FMZwY5JIX4quQTwQwHflsw3hl2RZNYu",
"title": "QBV877tmh5Y18Z2",
"subject": "8NPCWTGUF5k5iYB",
"author": "xnONnboaNlaL7q5",
"author_role": "tu8kTbQn0yWNMX1",
"author_organisation": "x36qaVbgIXuHgCw",
"tags":
[
{
"tag": "AoFCPDLY33A4fww",
"class": "L52uePB2HcklJu2"
}
,
{
"tag": "7ByYULGv36DhTSa",
"class": "NUjndXpM2y8wQ2g"
}
,
{
"tag": "4MRydSIXDBHcqaE",
"class": "v17DV2Xgkjh6AZq"
}
,
{
"tag": "2nFrUOh0lKGwVUJ",
"class": "NmSIjx2nqd7SIp4"
}
],
"comments": "GtdMrgackvjcP2s",
"date_created": 808426552630988,
"date_modified": 44633806482503
}
}
And I'm trying to add more objects to the Tags Array.
I've tried this:
ObjectNode fragmentNode = mapper.createObjectNode();
fragmentNode.put("tag", "blahblah");
fragmentNode.put("class", "the class");
String fragment = mapper.writeValueAsString(fragmentNode);
documentPatchBuilder.insertFragment("/resource/tags",
DocumentPatchBuilder.Position.LAST_CHILD, fragment);
And also this:
ObjectNode fragmentNode = mapper.createObjectNode();
fragmentNode.put("tag", "blahblah");
fragmentNode.put("class", "the class");
String fragment = mapper.writeValueAsString(fragmentNode);
documentPatchBuilder.replaceInsertFragment("/documents/thefile.json",
"/resource/tags", DocumentPatchBuilder.Position.LAST_CHILD, fragment);
But I keep getting this error message : Object nodes cannot have two children
with the same name.
Now nodes that I'm inserting (insertFragment) works, only this "adding to
existing" doesn't.
Anyone can point me in the right direction?
Thanks.
Desmond
-- next part --
An HTML attachment was scrubbed...
URL:
http://developer.marklogic.com/pipermail/general/attachments/20170125/f07a50f7/attachment-0001.html
--
___
General mailing list
General@developer.marklogic.com<mailto:General@developer.marklogic.com>
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
End of General Digest, Vol 151, Issue 39
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Question about patching a JSON document
Hi, Desmond:
The PATCH REST API request has an unfortunate limitation that prevents
clients from specifying an array as the context for a request.
The appropriate XPath expression would be:
/resource/array-node('tags')
The REST API uses cts.validIndexPath() to sanitize XPaths, which ensures
security without sacrificing performance but prohibits some useful paths
including the one above.
There is an RFE to address the issue, but it is unlikely to be implemented
in ML9.
In the meantime, the alternatives are either to read the document and modify
it on the client or to install server-side code to implement the change.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Desmond Lim
[desmond@senseinfosys.com]
Sent: Wednesday, January 25, 2017 12:11 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] Question about patching a JSON document
Hi there,
I have this document in Marklogic:
{
"resource":
{
"name": "hrRFIrq2JzpiXmc",
"url": "k4llhWiubj2SHtl",
"importer": "UXwZbxVCOfNFeST",
"text_context": "qerXqhThyAtdQkNhXZ2FMZwY5JIX4quQTwQwHflsw3hl2RZNYu",
"title": "QBV877tmh5Y18Z2",
"subject": "8NPCWTGUF5k5iYB",
"author": "xnONnboaNlaL7q5",
"author_role": "tu8kTbQn0yWNMX1",
"author_organisation": "x36qaVbgIXuHgCw",
"tags":
[
{
"tag": "AoFCPDLY33A4fww",
"class": "L52uePB2HcklJu2"
}
,
{
"tag": "7ByYULGv36DhTSa",
"class": "NUjndXpM2y8wQ2g"
}
,
{
"tag": "4MRydSIXDBHcqaE",
"class": "v17DV2Xgkjh6AZq"
}
,
{
"tag": "2nFrUOh0lKGwVUJ",
"class": "NmSIjx2nqd7SIp4"
}
],
"comments": "GtdMrgackvjcP2s",
"date_created": 808426552630988,
"date_modified": 44633806482503
}
}
And I'm trying to add more objects to the Tags Array.
I've tried this:
ObjectNode fragmentNode = mapper.createObjectNode();
fragmentNode.put("tag", "blahblah");
fragmentNode.put("class", "the class");
String fragment = mapper.writeValueAsString(fragmentNode);
documentPatchBuilder.insertFragment("/resource/tags",
DocumentPatchBuilder.Position.LAST_CHILD, fragment);
And also this:
ObjectNode fragmentNode = mapper.createObjectNode();
fragmentNode.put("tag", "blahblah");
fragmentNode.put("class", "the class");
String fragment = mapper.writeValueAsString(fragmentNode);
documentPatchBuilder.replaceInsertFragment("/documents/thefile.json",
"/resource/tags", DocumentPatchBuilder.Position.LAST_CHILD, fragment);
But I keep getting this error message : Object nodes cannot have two children
with the same name.
Now nodes that I'm inserting (insertFragment) works, only this "adding to
existing" doesn't.
Anyone can point me in the right direction?
Thanks.
Desmond
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Query: While inserting existing data again
Hi, Bushan: You can make a PATCH request to insert, replace, or delete nodes within an existing document: http://docs.marklogic.com/REST/PATCH/v1/documents Other write requests replace the entire document or its metadata or both. Hoping that clarifies, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Bhushan Suryawanshi [bsuryawan...@xenomorph.com] Sent: Thursday, January 12, 2017 1:51 PM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Query: While inserting existing data again Hi Guys, I am using MarkLogic8 along with REST API. I already have a document a.xml in the database with some content, but when we re-insert a.xml with updated content. I have following questions 1) Will MarkLogic upload the existing document? 2) Will MarkLogic merge the existing document with new content? If yes, is there any configuring parameter to do so. Thanks & Regards, Bhushan Suryawanshi Developer t: +1-212-401-7894 www.xenomorph.com<http://www.xenomorph.com/?mail> | blog<http://xenomorph.typepad.com/?mail> | twitter<http://goo.gl/oaBWO> | linkedin<http://goo.gl/rdi8W> This email is confidential and is intended only for the addressee. If you are not the intended recipient, please note that any dissemination, distribution or copying of this email is strictly prohibited. Any and all estimates, proposals, quotes and fees contained herein are for discussion purposes only and are not contractually binding unless executed in accordance with the standard practices of the Company. Attachments to this message have been virus checked but no guarantee can be made that any attachment is virus free. Please notify us immediately of any problem. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] sort-order is not working as expected
Hi, Shiv:
The "options" key should be in the object contained by the "search" key, not in
the object
contained by the "$query" key.
To put it another way, the "options" key should be a peer to the "$query" key,
not a child.
The basic structure:
{"search":{
"$query": {...},
"options": {...}
}
}
Also, the key should be $filtered instead of $filter
By the way, if you are supplying options, you can enable filtered search within
the options:
http://docs.marklogic.com/guide/search-dev/appendixa#id_77801
If you moved the filtering into the options, given that you aren't supplying an
actual query,
an alternative would be to use the /v1/search endpoint.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Wednesday, January 11, 2017 10:26 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] sort-order is not working as expected
Hi,
I am using sort-order option in "search" query, but the results are not in
order as expected i.e it is same order for Ascending/descending. I created an
range-index too.
Any advise?
Query:
http://localhost:8080/LATEST/qbe?query={"search":{"$query":{"$filter":true,"options":{"sort-order":[{"direction":"ascending","element":"age"}]=TEST-COLL
Thanks
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] invalid uri: error while inserting document in to database
Hi, Bushan: The REST API enforces best practices by checking URI validity when creating documents. If you need to round-trip identifiers from another system, you can encode the invalid URI characters. Where possible, it will be more straightforward to support only valid URIs. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Bhushan Suryawanshi [bsuryawan...@xenomorph.com] Sent: Thursday, January 05, 2017 3:03 PM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] invalid uri: error while inserting document in to database Hi Guys, I am using MarkLogic 8 along with REST api . While trying to insert a Document with following URI, It is generating an error. Example Document Uri: Gold/s p a c e.xml IS there any workaround to get document through the database. Thanks & Regards, Bhushan Suryawanshi Bhushan Suryawanshi Developer t: +1-212-401-7894 www.xenomorph.com<http://www.xenomorph.com/?mail> | blog<http://xenomorph.typepad.com/?mail> | twitter<http://goo.gl/oaBWO> | linkedin<http://goo.gl/rdi8W> This email is confidential and is intended only for the addressee. If you are not the intended recipient, please note that any dissemination, distribution or copying of this email is strictly prohibited. Any and all estimates, proposals, quotes and fees contained herein are for discussion purposes only and are not contractually binding unless executed in accordance with the standard practices of the Company. Attachments to this message have been virus checked but no guarantee can be made that any attachment is virus free. Please notify us immediately of any problem. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] sort-order is not working with REST
Hi, Shiv:
"field" has a special meaning in MarkLogic -- creating a range index on a field
is different from creating a range index on a JSON property (which is done by
treating the JSON property as an element with no namespace).
QBE doesn't support fields on the grounds that QBE works with concrete JSON
properties and XML elements and attributes instead of abstractions.
Anyway, if you did create a range index on the JSON property, you might turn on
validation of options and queries and inspect the log:
http://docs.marklogic.com/guide/rest-dev/service#id_61169
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Wednesday, January 04, 2017 11:48 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] sort-order is not working with REST
Hi Eric,
Yes, range index is there for that field.
--Shan.
On Wed, Jan 4, 2017 at 2:03 PM, Shiv Shankar
<shiv.shivshan...@gmail.com<mailto:shiv.shivshan...@gmail.com>> wrote:
Thanks Eric,
With this change, I could get the data, but no order change though the
sort-order specified ascending/descending.
Lets say firstName is "abc", "xyz", the query returning the response in the
same order irrespective of sort-order. Am I missing anything?
Regards
Shan.
On Wed, Jan 4, 2017 at 12:05 PM, Shiv Shankar
<shiv.shivshan...@gmail.com<mailto:shiv.shivshan...@gmail.com>> wrote:
Hi,
I am trying to use sort-order in $query. It is returning 0 records. All my
documents are JSON. Any inputs?
{"$query":{"options":{"sort-order":[{"direction":"descending","element":{"name":"firstName","score":
null}}]}}}
Tried with
{"$query":{"options":{"sort-order":[{"direction":"descending","json-property":"firstName"}]}}}
Note: I created Index on that field too.
Regards
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] sort-order is not working with REST
Hi, Shiv:
Does a range index exist on the JSON property? That's necessary to sort on a
property.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Wednesday, January 04, 2017 11:03 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] sort-order is not working with REST
Thanks Eric,
With this change, I could get the data, but no order change though the
sort-order specified ascending/descending.
Lets say firstName is "abc", "xyz", the query returning the response in the
same order irrespective of sort-order. Am I missing anything?
Regards
Shan.
On Wed, Jan 4, 2017 at 12:05 PM, Shiv Shankar
<shiv.shivshan...@gmail.com<mailto:shiv.shivshan...@gmail.com>> wrote:
Hi,
I am trying to use sort-order in $query. It is returning 0 records. All my
documents are JSON. Any inputs?
{"$query":{"options":{"sort-order":[{"direction":"descending","element":{"name":"firstName","score":
null}}]}}}
Tried with
{"$query":{"options":{"sort-order":[{"direction":"descending","json-property":"firstName"}]}}}
Note: I created Index on that field too.
Regards
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] sort-order is not working with REST
Hi, Shiv:
The $query and options can be sibling properties in a QBE payload
for MarkLogic 8.
The basic JSON structure when combining a query and options is:
{"search":{
"$query": ... the QBE query ...,
"$format": ... the QBE optional format ...,
"$validate": ... the QBE optional validation ...,
"options": ... the search options ...
}}
You can also persist the options and name them on the query:
http://docs.marklogic.com/guide/search-dev/qbe#id_32338
(As a footnote, it's also possible to omit the search container
property when sending a $query without options.)
In the example of options from the question, the JSON property
and score should be separate objects:
http://docs.marklogic.com/guide/search-dev/appendixa#id_44212
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Wednesday, January 04, 2017 9:05 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] sort-order is not working with REST
Hi,
I am trying to use sort-order in $query. It is returning 0 records. All my
documents are JSON. Any inputs?
{"$query":{"options":{"sort-order":[{"direction":"descending","element":{"name":"firstName","score":
null}}]}}}
Tried with
{"$query":{"options":{"sort-order":[{"direction":"descending","json-property":"firstName"}]}}}
Note: I created Index on that field too.
Regards
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] xdmp:value
Hi, Pragya:
If the goal is a sequence of two items, the stated code should be
possible to simplify to the following single line:
(" ABC XYZ ", $data/fn:data(following-sibling::p))
If the goal is a single concatenated string, the stated code should be
possible to simplify to the following single line:
fn:concat(" ABC XYZ ", $data/fn:string(following-sibling::p))
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Kapoor, Pragya
[pkapo...@innodata.com]
Sent: Wednesday, December 28, 2016 10:38 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] xdmp:value
I was able to do this by
let $docText := fn:data($data/following-sibling::p)
let $x := fn:data([ ABC XYZ ])
return( $x,$docText)
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Kapoor, Pragya
<pkapo...@innodata.com>
Sent: Thursday, December 29, 2016 12:03:40 PM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] xdmp:value
Hi
I need to join the below items:
let $docText := fn:data($data/following-sibling::p)
let $x := fn:data([ ABC XYZ ])
let $str := '$x || $docText'
return xdmp:value($str)
This is giving error:
XDMP-ARGTYPE: (err:XPTY0004) $x || $docText -- arg1 is not of type
xs:anyAtomicType?
Please let me know, how can this be achieved.
Thanks
Pragya
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action taken in reliance on this e-mail is
strictly prohibited and may be unlawful."
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action taken in reliance on this e-mail is
strictly prohibited and may be unlawful."
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] ML error on http-put
Hi, Pavadai:
The data element takes only text, not an element as described in:
http://docs.marklogic.com/xdmp:http-put
Try using xdmp:quote() to serialize the element payload, as in:
{xdmp:quote(
http://marklogic.com/manage;>
MLuser
MLpassword
)}
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of
pavadaidurai.ar...@cognizant.com [pavadaidurai.ar...@cognizant.com]
Sent: Tuesday, December 27, 2016 4:33 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] ML error on http-put
Hello Team,
I am getting an error as shown below while trying to execute the below http-put
REST service. Can someone please confirm what is wrong here?
Code:
(: execute this against the security database :)
xquery version "1.0-ml";
xdmp:http-put("http://localhost:8002/manage/v2/users/duraiDec/properties;,
admin
admin
http://marklogic.com/manage;>
MLuser
MLpassword
)
Error:
1.0-ml] XDMP-NOTSIMPLE:
xdmp:http-put("http://localhost:8002/manage/v2/users/duraiDec/properties;,
admin<passwo...) -- Node does not
have simple content: /*:options/*:data
Thanks,
Durai.
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended recipient(s), please reply to the sender and
destroy all copies of the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email,
and/or any action taken in reliance on the contents of this e-mail is strictly
prohibited and may be unlawful. Where permitted by applicable law, this e-mail
and other e-mail communications sent to and from Cognizant e-mail addresses may
be monitored.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Search for empty falues
Hi, Shiv:
> I need to search for empty values for a given key
You might need to use a cts.query() constructor instead of QBE for this kind
of query.
Given this test document:
xdmp.documentInsert('/test/testdoc.json', {test1:"value", test2:null,
test3:""});
If "empty" means null, you can use:
jsearch.documents().where(cts.jsonPropertyValueQuery('test2',
null)).result();
If "empty" means an empty string, you can use:
jsearch.documents().where(cts.jsonPropertyValueQuery('test3', "")).result();
If "empty" means the absence of a property, you can use:
jsearch.documents().where(cts.notQuery(cts.jsonPropertyScopeQuery('test4',
cts.trueQuery(.result();
Hoping that helpsl,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Tuesday, December 20, 2016 10:13 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Search for empty falues
Hi,
I need to search for empty values for a given key, Tried below, but resulted
nothing. I see values with empty in the collection. Any advice?
jsearch.documents().where(jsearch.byExample({EMP_ID: ""})).result();
Thanks
Shan.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs
Hi, Gary: Passing in the query options to a custom snippetting transform or to a REST API response transform would be another RFE. The interim workaround would be to write a resource service extension and execute the Search API from custom code that passing the query options where needed. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Gary Vidal [gvidalsass...@gmail.com] Sent: Friday, December 09, 2016 10:48 PM To: general@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs Thanks Erik, I think this would be a good RFE to file. On another request, is it possible to get the search:options name passed via the rest request and subsequently retrieving the options during the transform phase? Regards, Gary Vidal ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs
Hi, All: To reinforce Geert's important point, all of the APIs imported in the gist are internal: * MarkLogic Support cannot provide support for any application that uses those APIs. * Those APIs can and will change at any time, including in point releases. I just wanted to make sure anyone who takes that approach knows what they're getting into. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Geert Josten [geert.jos...@marklogic.com] Sent: Friday, December 09, 2016 12:04 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs Hi Gary, This little gist shows how you could do it. I’m reluctant to share though, as it works against internal libraries, so no guarantee it will work after installing updates. Could be useful input for such an RFE, though: http://github-search.demo.marklogic.com/detail/joemfb/structured-query-utils.xqy.json Cheers, Geert From: <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> Reply-To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Date: Friday, December 9, 2016 at 8:34 PM To: MarkLogic Developer Discussion <general@developer.marklogic.com<mailto:general@developer.marklogic.com>> Subject: Re: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs Hi, Gary: Unfortunately, there isn't any stable API like that at present. Such an API would be a reasonable RFE. Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Gary Vidal [gvidalsass...@gmail.com<mailto:gvidalsass...@gmail.com>] Sent: Friday, December 09, 2016 11:25 AM To: general@developer.marklogic.com<mailto:general@developer.marklogic.com> Subject: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs All, I am trying to figure out how to parse the query(in json format) so I can get the equivalent cts.query. Is there an api call that will resolve json-query back to cts? Regards Gary ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs
Hi, Gary: Unfortunately, there isn't any stable API like that at present. Such an API would be a reasonable RFE. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Gary Vidal [gvidalsass...@gmail.com] Sent: Friday, December 09, 2016 11:25 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Parsing json version of search:search query in json using custom transform in sjs All, I am trying to figure out how to parse the query(in json format) so I can get the equivalent cts.query. Is there an api call that will resolve json-query back to cts? Regards Gary ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Using Node.js in MarkLogic
Hi, Rahesh: You can definitely use Node.js with any JavaScript application or presentation framework that runs on Node.js as the middle tier and MarkLogic as the persistence tier. You must run Node.js on a different box than MarkLogic because MarkLogic is designed to exploit the machine completely. That is, don't run a different application server on the same box as MarkLogic. MarkLogic can act as an application server for two tier applications. However, MarkLogic itself cannot host asynchronous frameworks designed to run on Node.js Hoping that clarifies, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Rajesh Kumar [rajava...@gmail.com] Sent: Thursday, October 27, 2016 5:23 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Using Node.js in MarkLogic Hi Team, I'm planning to develop an application using React.js talking to MarkLogic. We also have suggestion to use Node.js as middle layer. Can we deploy the React and Node code within MarkLogic application server? Regards, Rajesh ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Using extract-document-data with GET and a REST endpoint
Hi, Mark: You can persist an options file on the server and refer to it by name: http://docs.marklogic.com/guide/rest-dev/search#id_48838 That said, my understanding is that POST is only specified to prevent caching by HTTP intermediaries. Unless there are gateways, proxies, and so on between the client and server, I wouldn't expect that to be a big issue. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Mark Shanks [markshanks...@hotmail.com] Sent: Monday, October 24, 2016 5:56 PM To: General@developer.marklogic.com Subject: [MarkLogic Dev General] Using extract-document-data with GET and a REST endpoint Hi, I have a REST endpoint up and can query documents so that the elements returned are limited by the following Options.xml file: http://marklogic.com/appservices/search;> test http://marklogic.com/appservices/search;> /audit/auditor /audit/date false I call the REST endpoint using: curl --anyauth --user admin:admin -X POST -i -H "Content-type: application/xml" -H "Accept: multipart/mixed; boundary=BOUNDARY" -d @./Options.xml http://localhost:8111/LATEST/search However, am I right that this is not the best way to do it as POST requests never get cached, whereas if I used a GET cache the requests could be cached?? However, is it possible to run the above constraint using GET rather than POST? I tried setting a predefined search using something like the following: curl --anyauth --user admin:admin -T "Options.xml" -H "Content-type: application/xml" http://localhost:8111/LATEST/config/query/test However, I just get an error. Is it possible to POST the Options.xml and then use GET to actually retrieve the data? Thanks. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Issue with special charecters in MarkLogic
Hi, Vinod: If you read the character stream as UTF-8 in Java, you shouldn't need to do anything special on either the client or server. Are you using the MarkLogic Java Client API? The Java API knows the server encodes as UTF-8 and defaults to reading as UTF-8. The only other question that occurs to me is whether the code that's writing the content might be double encoding. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of thangella.vinodre...@cognizant.com [thangella.vinodre...@cognizant.com] Sent: Wednesday, October 05, 2016 3:23 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Issue with special charecters in MarkLogic Hi All, We are facing an issue with special characters involved in mathematical expressions such as “ write 혢(혹)/혣(혹) in the form “ which has italic characters (혢(혹)/혣(혹)). In MarkLogic these are stored as UTF-8 encoded values as “write ()/() in the form”. In Java application When we are retrieving data from MarkLogic DB ,these special characters are being received as “ write ݘ¢(ݘ¹)/ݘ£(ݘ¹) in the form” . These are being corrupted by the time we received them in Java layer. We tried many options from Java side to receive content as UTF-8 encoded, but did not resolved the issue. Can anyone please suggest a way to resolve this kind of issues from MarkLogic side. Thanks, Vinod. This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Sorting results of lexicon calls
Hi, Andreas: You can use cts:value-tuples() to form tuples. List the range index for the sort first and the lexicon to be sorted second (for instance, date and then uri): http://docs.marklogic.com/cts:value-tuples Be aware that, if the sorting or sorted values occurs more than once in the document, you'll get a tuple for the cross product over the document (that is, the co-occurrence within the document). The logtfidf score uses the document relevance to the query. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Andreas Hubmer [andreas.hub...@ebcont.com] Sent: Wednesday, September 21, 2016 5:04 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Sorting results of lexicon calls Hi, All the lexicon functions (cts:uris, cts:element-values, ...) support the options "ascending" and "descending" to define the sort order. But sometimes I would like to sort the result of a lexicon call by some other range index. Example: I would like to get the URIs of all documents, ordered by some date element. Can this be done with the lexicon functions? I've seen that all the lexicon functions support the option "score-logtfidf". What is the effect of that option and the other scoring options (in lexicon calls)? I've tried it and I didn't get any different results. Thanks, Andreas -- Andreas Hubmer Senior IT Consultant EBCONT enterprise technologies GmbH Millennium Tower Handelskai 94-96 A-1200 Vienna OUR TEAM IS YOUR SUCCESS UID-Nr. ATU68135644 HG St.Pölten - FN 399978 d ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Adding PDF to an existing json document using Patch
Hi, Shiv:
A patch request returns a 204 if none of the operations match on the patched
document.
That's an indication to check the paths in the operations to make sure they
match the structure and vocabulary of the target JSON document.
The patch from your earlier post would change this document:
{..., "parent":{..., "childe":{...}, ...}, ...}
into this document
{..., "parent":{..., "childe":{..., "image":"/example/scdhhs_fm_300.pdf"},
...}, ...}
Without the actual patch operations and actual target document structure, no
one can make more specific suggestions.
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Tuesday, August 23, 2016 3:12 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] General Digest, Vol 146, Issue 46
Hi Erik,
Thanks for quick reply,
With the error message, I see we can PATCH only json/XML.
So, to have the PDF linked to current document, I would like to do the
following.
1. Load binary document with a URI ( ex: /documents/drivier-license.pdf)
2. Add this uri ( /documents/drivier-license.pdf) to existing document as a
patch)
When I did the same above, i.e adding a json as part of content
{"uri":"/documents/drivier-license.pdf"} as a PATCH, it is coming as 204
unchanged.
Please advice.
Regards
Shiv.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Patch update PDF/binary data
Hi, Shiv:
PATCH is changes nodes within an existing tree structure. As such, it only
makes sense for JSON or XML documents, which have tree structure.
For binary documents, just write the replacement document. The request looks
like it's sending the entire document, so PUT is the right method.
Hoping that clarifies,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Shiv Shankar
[shiv.shivshan...@gmail.com]
Sent: Friday, August 19, 2016 1:14 PM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] Patch update PDF/binary data
Hi,
I am getting when I do patch update of binary data to an existing json object.
url -v -X POST --basic --user user:pwd
--data-binary '@mydoc.pdf' -i -H "Content-type: application/binary"
-H "X-HTTP-Method-Override: PATCH"
'http://localhost:8004/LATEST/documents?uri=/test/899'
Error below:::
{"errorResponse":{"statusCode":400, "status":"Bad Request",
"messageCode":"RESTAPI-INVALIDREQ", "message":"RESTAPI-INVALIDREQ:
(err:FOER) Invalid request: reason: patch is not or JSON
patch object for uri: /test/899"}}
Any advice or is there any other way like apart from insert binary and put that
uri as patch to existing document?
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Getting multiple documents using REST
Hi, Shiv: What programming language are you working with? If you use the Node.js API, you can get a list of JSON result documents as a single JavaScript array. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Shiv Shankar [shiv.shivshan...@gmail.com] Sent: Thursday, August 18, 2016 8:53 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Getting multiple documents using REST Hi, I'm using QBE and have the parameters with 'Accept: multipart/mixed'. I'm getting the results as separate values. Is there a way I can get all results ( content) in one document ? Tried with Qbe params, but no success. Regards Shan ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] : Marklogic Integration with DITA OT
Hi, Chandan: RSI has a publishing solution that, I believe, uses the DITA Open Toolkit with MarkLogic. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Chandan Garg [chandan.g...@zensar.com] Sent: Tuesday, August 09, 2016 3:07 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] : Marklogic Integration with DITA OT Dear All, Currently we are using Astoria as CMS with OXYGEN as XML editor & DITA OT (for rendering/publishing). Kindly suggest if we can build a solution with Marklogic & OXYGEN as XML editor & DITA OT. Thanks for your help in advance. PS: It can be treated as publishing use case. Thanks, Chandan ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] #CGP#RE: Need Technical Support
Hi, Udaya: Did you confirm you can access the REST port using curl? That's a good first step. If you're using Scala, you may want to install and call the Java Client API: http://docs.marklogic.com/guide/java For more information about the REST API, see: http://docs.marklogic.com/guide/rest-dev Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of udaya.chill...@wipro.com [udaya.chill...@wipro.com] Sent: Monday, July 18, 2016 1:06 AM To: general@developer.marklogic.com Cc: suresh.bo...@wipro.com Subject: Re: [MarkLogic Dev General] #CGP#RE: Need Technical Support Hello Team, We tried creating the Rest API Instance however we are unable to perform any CRUD activities . Could you please let us know how can we perform CRUD operations by connecting to REST end point using JavaScript/AJAX without using curl. Thanks Udaya From: general-boun...@developer.marklogic.com <general-boun...@developer.marklogic.com> on behalf of Indrajeet Verma <indrajeet.ve...@gmail.com> Sent: Friday, July 15, 2016 12:49 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] #CGP#RE: Need Technical Support ** This mail has been sent from an external source ** Hi Udaya, As per my understanding you should always use latest version of ML if you are installing from scratch. The latest version of ML (MarkLogic-8.0-5.4) is available here https://developer.marklogic.com/products. MarkLogic 8 — MarkLogic Developer Community<https://developer.marklogic.com/products> developer.marklogic.com MarkLogic is the only Enterprise NoSQL Database ... Daniel, I'm not sure I understand your question, since you've specified a Linux version. Did you follow all the installation steps of MarkLogic those are mentioned here https://docs.marklogic.com/guide/installation/procedures#id_28962? (mainly configuration and security set up). If you are able to access MarkLogic using http://localhost:8001 and http://localhost:8000/qconsole then it means your installation completed successfully. Now you may try below curl command to create rest instance: curl --anyauth --user admin:admin -X POST -d@"config/rest-instance.xml" -i -H "Content-type:application/xml" http://localhost:8002/v1/rest-apis Make sure rest-instance.xml has been created under <>/config in your system from where you are running command. rest-instance.xml http://marklogic.com/rest-api;> test-rest Default test-rest-db test-rest-mods 8045 Now you may open http://localhost:8001 and see whether rest instance and db has been created successfully. You should check database and attached forest as well. You may try another rest end point to load content: curl --anyauth --user admin:admin -X PUT -T docs-rest/song1.xml "http://localhost:8045/v1/documents?uri=/plays/play1.xml=xml=plays; For more information regarding setting up rest instance, please refer https://developer.marklogic.com/learn/rest/setup You may attend some training courses that MarkLogic provides: http://mlu.marklogic.com/ondemand/index.xqy?q=Version%3A%22MarkLogic%208%22 Hope that helps! Regards, Indy On Fri, Jul 15, 2016 at 12:24 PM, Khan, Zishan <zishan.k...@capgemini.com<mailto:zishan.k...@capgemini.com>> wrote: ++below From: Khan, Zishan Sent: Friday, July 15, 2016 12:23 PM To: general@developer.marklogic.com<mailto:general@developer.marklogic.com> Subject: RE: Need Technical Support Hi udaya, Did you use the curl command properly ?. check it once again it should work if coded properly. Regards, Zishan From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [mailto:general-boun...@developer.marklogic.com] On Behalf Of udaya.chill...@wipro.com<mailto:udaya.chill...@wipro.com> Sent: Friday, July 15, 2016 12:16 PM To: general@developer.marklogic.com<mailto:general@developer.marklogic.com> Subject: Re: [MarkLogic Dev General] Need Technical Support Hello Team, We are new to MarkLogic DB and we are analysing it to use in our project which is programmed in scala .We have downloaded the MarkLogic 7 trial version but are unable to create a rest API instance. We tried to insert a document but we are unable to come forward. Can you please assist us technically to get started. Thanks Udaya From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> <general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>> on behalf of general-requ...@developer.marklogic.com<mailto:general-requ...@developer.marklogic.com> <general-requ...@developer.marklogic.com<mailto:general-requ...@developer.marklogic.com>> Sent: Friday, July 15, 2016 12
Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?
Hi, Steve: Our cookbook examples are maybe a bit misleading in this respect. Each example creates its own database client merely so it can stand alone. A realistic application would keep a singleton database client until the application completely finishes using the REST server. To put it the other way, there's no reason to close a database client if the application is still using the same REST server as the same user. The connection pool is associated with the database client (and, in fact, is Apache HttpClient in the lower layers), so using a singleton database client (including sharing across multiple threads) should greatly improve the socket efficiency. Hoping that helps (and that things are well with you), Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Steve Anderson [steve.ander...@gmail.com] Sent: Friday, July 01, 2016 5:02 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load? Thanks for the fast response, Erik, hope you're doing well. I am not using a single database client object. The docs say to close() the client after use. Doing that makes the client unusable for further transactions, so I've been creating a new one for each request. Should I have a singleton client and never use close(), or is there some in-between best practice?? I believe I am writing and reading to completion. Steve On Fri, Jul 1, 2016 at 4:40 PM Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Hi, Steve: A few questions: * Are you creating a single database client object and using it for all requests on the REST port? That's important for connection pooling. The database client is thread safe after configuration. For instance, in a Java servlet environment, the single database client would be created during application initialization. * Are you writing and reading to completion? For instance, closing streams after reading the data. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Steve Anderson [steve.ander...@gmail.com<mailto:steve.ander...@gmail.com>] Sent: Friday, July 01, 2016 4:09 PM To: general@developer.marklogic.com<mailto:general@developer.marklogic.com> Subject: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load? I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java 1.8.0_72 and MarkLogic 8.0-5.1. I've noticed that my app under load (multiple requests to different end points) is leaving of sockets between the server and the database in the CLOSE_WAIT state. Under load (thousands of document writes in multiple threads), it climbs and climbs and climbs, with no apparent end in sight. If I force a cleanup via the debugger (System.gc(); System.runFinalization ();) the majority of the sockets close. When I run AllCookbookExamples, and, in another terminal, running "lsof -i -n -P|grep _WAIT" I see the same thing; the number of sockets left open grows until the session ends. It's no different with the current develop branch on github. I suspect it's something I'm missing related to connection pooling, but I don't see any documentation on connection pooling configuration. I know I can modify ulimit on my system, but I'm afraid it's an underlying issue in my code that will keep the sockets open forever and eventually cause network issues. Has anyone else seen anything like this? Steve ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load?
Hi, Steve: A few questions: * Are you creating a single database client object and using it for all requests on the REST port? That's important for connection pooling. The database client is thread safe after configuration. For instance, in a Java servlet environment, the single database client would be created during application initialization. * Are you writing and reading to completion? For instance, closing streams after reading the data. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Steve Anderson [steve.ander...@gmail.com] Sent: Friday, July 01, 2016 4:09 PM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Sockets being left in CLOSE_WAIT under load? I'm using version 3.0.5 of the Java API client on a Mac with Oracle's Java 1.8.0_72 and MarkLogic 8.0-5.1. I've noticed that my app under load (multiple requests to different end points) is leaving of sockets between the server and the database in the CLOSE_WAIT state. Under load (thousands of document writes in multiple threads), it climbs and climbs and climbs, with no apparent end in sight. If I force a cleanup via the debugger (System.gc(); System.runFinalization ();) the majority of the sockets close. When I run AllCookbookExamples, and, in another terminal, running "lsof -i -n -P|grep _WAIT" I see the same thing; the number of sockets left open grows until the session ends. It's no different with the current develop branch on github. I suspect it's something I'm missing related to connection pooling, but I don't see any documentation on connection pooling configuration. I know I can modify ulimit on my system, but I'm afraid it's an underlying issue in my code that will keep the sockets open forever and eventually cause network issues. Has anyone else seen anything like this? Steve ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] order of cts:search result
Hi, Vikas: No, the order of listed values in a range query has no influence on document ordering. In your second example, the predicate is scanning the entire list of documents for each sequence value. Given that the list of documents is in the thousands, you might want to iterate over the documents once to order them -- something like the following should work: let $id-order := let $id-map := map:map() return ( for $id at $i in $seq return map:put($id-map,$id,$i), $id-map ) for $content in cts:search(fn:doc(),cts:element-attribute-range-query(xs:QName(“emp”) ,xs:QName(“id”),”=”,$seq )) let $id := $content//emp/@id/string(.) let $ord := map:get($id-order,$id) order by $ord return $content If you can materialize a value in each document and create a range index over those values, you can use cts:index-order(), which would be faster yet. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of vikas.sin...@cognizant.com [vikas.sin...@cognizant.com] Sent: Tuesday, June 21, 2016 1:09 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] order of cts:search result Hi All, I am passing sequence of value to cts:search function as attribute range-query . Will cts: search return the content in same order of sequence element passed. I want to get the result as sorted by “18hduye” ,” 38h34ye” ,” 28h55e”. Example: let $seq := (“18hduye”,”38h34ye”,”28h55e”) return cts:search(fn:doc(),cts:element-attribute-range-query(xs:QName(“emp”) ,xs:QName(“id”),”=”,$seq )) Will the above function return the Employee in same sequence as above mentioned in seq variable. Or do we have any other way to sort the content as passed $seq variable. As per my understanding order by will work only resultset which we got from cts:search . I found one way as : let $content := cts:search(fn:doc(),cts:element-attribute-range-query(xs:QName(“emp”) ,xs:QName(“id”),”=”,$seq )) for $each in $seq return $content[@id=$each] This way I am making sure the correct order but it will slow our process the get the result set for thousands of object. Regards, Vikas Singh This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Avoiding Facets in search:snippet highlight
Hi, Tabish: This might not be what you're asking but... If you want to turn off the facet output for a range index, set the facet attribute or subproperty to false: http://docs.marklogic.com/guide/search-dev/appendixa#id_42752 If you want to prefer other elements or properties to properties with a range index, used preferred-matches: http://docs.marklogic.com/guide/search-dev/appendixa#id_50098 If you need to implement a custom snippeting function that skips a blacklist of elements or properties, see: http://docs.marklogic.com/guide/search-dev/query-options#id_58295 If none of that helps, please provide a specific example of what the current and desired output. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of ShamsTabish Sheikh [shams4...@hotmail.com] Sent: Tuesday, June 14, 2016 4:49 AM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Avoiding Facets in search:snippet highlight Hello team, I'm are trying to display the search keyword matches to the user in our app. I have used search:snippet to highlight the matches, in some cases facet values are appearing in the results which we don't want, please suggest a way to avoid facet values in search:highlight. Thanks & Regards, Tabish. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Regarding compilation of schema for validation
Hi, Gnana: Can you explain why it's necessary to validate against XSD using a single-step XSLT as opposed to a two-step approach that validates against the XSD using an XQuery validate expression and then validates against the Schematron rules that supplement the XSD using XSLT? While it's possible, it's not a given that validating an XSD with templates in a single-step XSLT will be faster than the two-step approach. A built-in validate expression might validate an XSD faster than XSLT can process the equivalent templates, even excluding the XSLT invocation startup costs. The only way to be sure would be to code up some templates for a representative schema and quantify the benefits. As Florent pointed out, you would need to generate the XSLT in advance instead of on request to out perform the two-step approach. If you decide to go the conversion route, you could try to use the sc:* library to parse the XSD. Supporting all of the nuances of XSD (for instance, type restriction and substitution groups) would require a ton of work. If you do determine that a single XSLT step is optimal, given the complexity allowed by XSD, you might be better off simply writing all of the rules in Schematron instead of trying to converting an XSD to XSLT. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Florent Georges [li...@fgeorges.org] Sent: Wednesday, May 11, 2016 5:46 AM To: MarkLogic Developer Discussion; gnanaprakash.bodire...@cognizant.com Subject: Re: [MarkLogic Dev General] Regarding compilation of schema for validation Hi, The standard for Schematron (ISO 19757-3) does not define how to embed it within W3C XML Schema. This is something *you* (your project, your company) use. That makes perfect sense, but you have to define how it is done, and how to treat it. Especially for corner cases like imports, etc. I would suggest you try and formalize how you embed Schematron rules into XML Schemas at "Your Project ©". What's allowed, what's not. If the schemas are known at deployment, I would also strongly advise to compile them to Schematron (extract the rules out of the XML Schemas), and probably even to compile them to XSLT stylesheet at build time. So you can apply them directly, as there is no need to compile them every time you use them if they are static. Regards, -- Florent Georges http://fgeorges.org/ http://h2oconsulting.be/ On 11 May 2016 at 11:43, ? wrote: > Thanks Florent Georges > > It helps a lot as we can extract the schematron and compile and use it. > > I am looking for any out of the box solution which can do it without > extracting the schematron from schema. I understand schema and schematron are > 2 different concepts but as we are now having schematron embedded in Schema > and trying to understand how best we can validate this. > > Also I am looking at what if schema (with schematron) is importing other > schemas (with schematron). Schema validation will work without issues as they > are referring to other schemas but schematron I am not sure how it works. > > Regards, > Gnana(GP) > > -Original Message- > From: fgeor...@gmail.com [mailto:fgeor...@gmail.com] On Behalf Of Florent > Georges > Sent: 11 May 2016 10:15 > To: MarkLogic Developer Discussion; Bodireddy, Gnanaprakash (Cognizant) > Subject: Re: [MarkLogic Dev General] Regarding compilation of schema for > validation > > On 10 May 2016 at 21:46, ? wrote: > > Hi, > >> If a schema is having a schematron with in it. Is there any way we can >> validate both schema and schematron rules against an XML document? > > Have you seen my response yesterday? Have you tried it? It should fulfill > exactly that need. > > Regards, > > -- > Florent Georges > http://fgeorges.org/ > http://h2oconsulting.be/ > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. If you are not the intended recipient(s), please reply to the > sender and destroy all copies of the original message. Any unauthorized > review, use, disclosure, dissemination, forwarding, printing or copying of > this email, and/or any action taken in reliance on the contents of this > e-mail is strictly prohibited and may be unlawful. Where permitted by > applicable law, this e-mail and other e-mail communications sent to and from > Cognizant e-mail addresses may be monitored. > ___ > General mailing list > General@developer.marklogic.com > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General
Re: [MarkLogic Dev General] Regarding compilation of schema for validation.
Hi, Shabana: > We need to compile our XSD file into XSLT so that we can use a compiled > version for validation Is there a reason you can't use the MarkLogic builtin support for validating a document against an XSD? If you insert an XML Schema into the schema database for your content database and restart the server, you should be able to validate documents against the schema using either an XQuery validate expression: https://www.w3.org/TR/xquery-30/#id-validate or the MarkLogic validate function: http://docs.marklogic.com/xdmp:validate The builtin validation will almost certainly be faster than an XSLT invocation. The other question is whether your Schematron compiler supports compiling an XSD. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of shabana khan [shabana.0...@gmail.com] Sent: Monday, May 09, 2016 7:18 AM To: General@developer.marklogic.com Subject: [MarkLogic Dev General] Regarding compilation of schema for validation. Hi All, We need to compile our XSD file into XSLT so that we can use a compiled version for validation When we use sch:compile-schema($schematron) and give schematron as input it successfully generates the XSLT. But we have as input an XSD which comprises both schematron part and schema part like we do have pattern elements and schema elements as well. We are getting errors : sc:compile-schema($schematron) -- Invalid coercion Can anyone give suggestion regarding this. ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Not able to use indexes in cts:search
Hi, Jaydip:
To expand on Rob's good suggestion ...
The index always matches a document fragment.
A searchable expression like //SUGGESTION doesn't change that.
The unfiltered search against the index matches document fragments that have
both a SUGGESTION element and a value of "wireless keyboard" in the path range
index.
For the matched documents, the searchable expression also extracts the
SUGGESTION nodes.
Because the search is unfiltered, my understanding is that the extracted nodes
would not be checked for false positives. The predicate would just take the
subset of the extracted nodes for the matched documents.
Back to Rob's point -- the standard guidance is to model each document as a
row, not a table. That should solve the problem because any SUGGESTION
document that matches will be a true positive.
By the way, it's a better practice to use fn.subsequence(cts.search(...), 1, 2)
instead of a [1 to 2] predicate. fn.subsequence() iterates lazily. A
predicate is applied to all results (except where optimized away).
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Rob Szkutak
[rob.szku...@marklogic.com]
Sent: Friday, May 06, 2016 7:12 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Not able to use indexes in cts:search
Hello,
It seems to me the simplest and best approach here would be to have one
suggestion per document rather than multiple suggestions in each document.
Best,
Rob
Rob Szkutak
Senior Consultant
MarkLogic Corporation
rob.szku...@marklogic.com
www.marklogic.com<http://www.marklogic.com>
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Jaydip Rananavare
[jaydiprananav...@gmail.com]
Sent: Friday, May 06, 2016 8:30 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Not able to use indexes in cts:search
Hi All,
We have following document
computer
laptop
super computer
keyboard
virtual keyboard
wireless keyboard
when user searches for any ALT-LABELS or HIDDEN-LABELS we want to show
PREF-LABEL as suggestion for that we have used following query
cts:search(//SUGGESTION, cts:path-range-query(
"/SUGGESTIONS/SUGGESTION/HIDDEN-LABEL", "=" , ("wireless keyboard"),
("collation=http://marklogic.com/collation/en/S1;)), ("unfiltered"))[1 to 2]
When we pass "filtered" option to cts:search, it works properly with a little
performance hit, but when we pass "unfiltered" option it does not give correct
result rather it fetches only first result,
We have also added path range index for above path.(also tried with
element-value-query)
Any help would be appreciated :)
--
Thanks and regards,
Jaydip Rananavare
M +91 98 60 07 39 90
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Search API total not equal to number of search results
Hi, Danny: Yes, 7.0-5 and 8.0-1 and following should have the accurate total on the last page of search results where the last page has fewer results than the page length. Note that if the last page happens to have the same number of results as the page length, the page can't be recognized as the last page. In this case, the start offset of the first empty page will be the accurate total. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Danny Sinang [d.sin...@gmail.com] Sent: Wednesday, April 20, 2016 7:24 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Search API total not equal to number of search results Hi Erik, Would you know if this bug has been fixed already ? Regards, Danny On Tue, Jan 6, 2015 at 11:15 AM, Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Hi, Danny: Thanks for the additional information. The total is set to the estimate based on the indexes. That's pragmatic for pages other than the last page. In that case, we can calculate an accurate total based on the start page, page length, and result count. I'll file a bug. Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Danny Sinang [d.sin...@gmail.com<mailto:d.sin...@gmail.com>] Sent: Tuesday, January 06, 2015 6:24 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Search API total not equal to number of search results One thing I discovered is that if I change "search-option" to "unfiltered", @total and the number of search results are both 88. Regards, Danny On Tue, Jan 6, 2015 at 9:14 AM, Danny Sinang <d.sin...@gmail.com<mailto:d.sin...@gmail.com>> wrote: Hi Erik, Here's the query and the options : xquery version "1.0-ml"; import module namespace search = "http://marklogic.com/appservices/search; at "/MarkLogic/appservices/search/search.xqy"; declare option xdmp:mapping "false"; declare variable $options := http://marklogic.com/appservices/search;> filtered false wildcarded case-insensitive punctuation-sensitive diacritic-sensitive ( - OR AND http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy" element="cts:element-query" tokenize="word">CHILD LT LE GT GE NE " : http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy"/> http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy"/> http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy"/> http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy"/> http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy"/> http://www.nejm.org/custom-field-query; at="/lib/custom-fields.xqy"/> true false false http://marklogic.com/cts;> question ; let $search := '(qAuthoringState:"Approved" OR qAuthoringState:"Warning") AND (qAudiencePoolGroup:(qApName:"Internal Medicine Audience Pool" AND qApRelevance GE "1" AND qApRelevance LE "1")) AND (entry:(key:1 AND value:(qBankDimensions:(name:"Sub-Specialty" AND dimensionFacts:(name:"Cardiology") AND sort:requestparam' return search:search($search, $options, 1, 100) On Mon, Jan 5, 2015 at 9:45 PM, Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Hi, Danny: Can you share your query and options? Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf
Re: [MarkLogic Dev General] Xquery : Empty Content
Hi, Pragya:
If whitespace only appears in container elements and not in leaf elements, you
can use node tests:
if (empty($in-xml//*[empty(*) and exists(text())])) then ...
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of VISH RAJPUT
[svishnu.sin...@gmail.com]
Sent: Monday, April 18, 2016 4:02 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Xquery : Empty Content
Hi Pragya,
I think we can test normalize-space here like below:-
let $in-xml :=
(
)
return (if ($in-xml[normalize-space() = '']) then ("Empty In-XML") else ("Not
Empty"))
Regards,
Vishnu
On Mon, Apr 18, 2016 at 3:37 PM, Kapoor, Pragya
<pkapo...@innodata.com<mailto:pkapo...@innodata.com>> wrote:
Hi,
I have an xml :
let $in-xml :=
Now, how can I check if the content of all the elements and there child
elements in $in-xml are empty(no content)?
if(content is empty) then "do this" else "do this"
Thanks
Pragya
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action taken in reliance on this e-mail is
strictly prohibited and may be unlawful."
___
General mailing list
General@developer.marklogic.com<mailto:General@developer.marklogic.com>
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
--
Vishnu Singh | http://marklogicgd.blogspot.in/
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] /node-client-api cannot get it working
Hi, Alex: Many significant bug fixes have gone into MarkLogic Server since 8.0-3 One of those fixes might have affected requests for the roles endpoint or the roles that the setup.js module is trying to create. You could always take a look inside the etc/test-setup-users.js module and try the equivalent POST requests to the roles endpoint using curl to see if it's a REST issue rather than a Node.js issue. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Alex Muir [alex.g.m...@gmail.com] Sent: Wednesday, April 06, 2016 8:30 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] /node-client-api cannot get it working On Wed, Apr 6, 2016 at 3:22 PM, Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Are you using MarkLogic 8.0-5? it's version 8.0-3 running in a docker container. I mentioned before that: I can however connect and load documents using the examples http://docs.marklogic.com/8.0/guide/node-dev/intro# node loaddocument.js Loaded the following documents: /gs/aardvark.json /gs/bluebird.json /gs/cobra.json so it suggests my credentials and ability to connect with node js at the very least is working. Regards Alex www.jahtoe.com<http://www.jahtoe.com> ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] /node-client-api cannot get it working
Hi, Alex:
The sequence of operations that I suggested worked without hiccup for me.
Especially given the 400 error, I'm wondering if the issue is on the MarkLogic
side.
Are you using MarkLogic 8.0-5?
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Alex Muir
[alex.g.m...@gmail.com]
Sent: Wednesday, April 06, 2016 7:59 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] /node-client-api cannot get it working
Hi Erik,
I've tried that previously and I've tried again deleting the node-client-api
and getting a fresh copy from git.
I get the same errors.
Regards
Alex
www.jahtoe.com<http://www.jahtoe.com>
On Tue, Apr 5, 2016 at 3:46 PM, Erik Hennum
<erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote:
Hi, Alex:
Have you tried the following?
git clone https://github.com/marklogic/node-client-api.git
cd node-client-api/
npm install
That should install all of the dependencies listed in the package.json
The --save argument will update the package.json, which shouldn't
be necessary.
In particular, I would expect the following command to make the
cloned MarkLogic Node.js API dependent on itself.
npm install -f marklogic --save
Also, try installing the gulp and mocha command-line utilities
globally, as in:
npm install gulp -g
npm install mocha -g
Hoping that helps,
Erik Hennum
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>]
on behalf of Alex Muir [alex.g.m...@gmail.com<mailto:alex.g.m...@gmail.com>]
Sent: Tuesday, April 05, 2016 6:00 AM
To: general@developer.marklogic.com<mailto:general@developer.marklogic.com>
Subject: [MarkLogic Dev General] /node-client-api cannot get it working
Hi,
I'm running linux mint 17 with node --version v4.4.2 with npm --version 3.8.5
I've done the following to get the marklogic client api working
git clone https://github.com/marklogic/node-client-api.git
cd node-client-api/
npm install -f marklogic --save
npm install --save
however when I execute node etc/test-setup.js
admin user (default=admin): admin
admin password (default=admin):
checking for REST users
/mnt/mntlech/externalgit/node-client-api/node_modules/bluebird/js/main/async.js:43
fn = function () { throw arg; };
^
Error: POST /manage/v2/roles: cannot process response with 400 status
Any ideas what the problem may be going wrong there?
I can however connect and load documents using the examples
http://docs.marklogic.com/8.0/guide/node-dev/intro#
node loaddocument.js
Loaded the following documents:
/gs/aardvark.json
/gs/bluebird.json
/gs/cobra.json
so it suggests my credentials and ability to connect with node js at the very
least is working.
also if I run 'node gulp doc' in node-client-api/ i get errors as well
node gulp doc
module.js:327
throw err;
^
Error: Cannot find module '/mnt/mntlech/externalgit/node-client-api/gulp'
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:139:18)
at node.js:968:3
Regards
Alex
www.jahtoe.com<http://www.jahtoe.com>
___
General mailing list
General@developer.marklogic.com<mailto:General@developer.marklogic.com>
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Need help on document patch operation with replace-insert
Hi, Kevin:
The fundamental issue here is that, in the insert case, the goal is to add a
key but, in the replace case, the goal is to change the value. The two cases
require different content.
Unfortunately, there's no good solution at present using a patch request.
The easiest alternative is to read the document, modify it on the client, and
write the document back.
The most efficient alternative to install some server-side JavaScript or XQuery
that reads and modifies the document using the xdmp.node*() functions and then
invoke the server-side module. See:
http://docs.marklogic.com/guide/rest-dev/extensions#id_59188
Working with your support contact to create a bug would also be appropriate.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Kevin Leturc
[klet...@nuxeo.com]
Sent: Wednesday, April 06, 2016 1:14 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Need help on document patch operation with
replace-insert
Hi Geert,
I try to patch document with this java call :
PatchHandle patch = ...
docManager.patch("/----.json", patch);
The document to update is :
{
"ecm__id": "----",
"ecm__racl": [ "Administrator", "Members" ],
"dc__modified": "1970-01-01T00:00:00.000"
}
The first time, I want to insert { "ecm__minorVersion": 0 } with this call :
{
"pathlang": "xpath",
"patch": [{
"replace-insert": {
"select": "/node('ecm__minorVersion')",
"context": "/node('ecm__id')",
"position": "after",
"content": {
"ecm__minorVersion": 0
}
}
}]
}
The document is updated to :
{
"ecm__id": "----",
"ecm__minorVersion": 0,
"ecm__racl": ["Administrator", "Members"],
"dc__modified": "1970-01-01T00:00:00.000"
}
Then I want to update ecm__minorVersion to 1, with this call :
{
"pathlang": "xpath",
"patch": [{
"replace-insert": {
"select": "/node('ecm__minorVersion')",
"context": "/node('ecm__id')",
"position": "after",
"content": {
"ecm__minorVersion": 1
}
}
}]
}
The document is updated to :
{
"ecm__id": "----",
"ecm__minorVersion": {
"ecm__minorVersion": 1
},
"ecm__racl": ["Administrator", "Members"],
"dc__modified": "1970-01-01T00:00:00.000"
}
but I expect this :
{
"ecm__id": "----",
"ecm__minorVersion": 1,
"ecm__racl": ["Administrator", "Members"],
"dc__modified": "1970-01-01T00:00:00.000"
}
I have tried several patch requests, see below a non exhaustive list :
- put a 'parent' node at root -> same behaviour
- don't use node('...') in select path -> same behaviour
- don't use node('...') in context path -> same behaviour
Thanks for your help
Regards
Kevin Leturc Software developer
Twitter @kevinleturc
nuxeo.com<https://www.nuxeo.com>
[www.nuxeo.com - Content Management Platform]<http://www.nuxeo.com>
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] /node-client-api cannot get it working
Hi, Alex:
Have you tried the following?
git clone https://github.com/marklogic/node-client-api.git
cd node-client-api/
npm install
That should install all of the dependencies listed in the package.json
The --save argument will update the package.json, which shouldn't
be necessary.
In particular, I would expect the following command to make the
cloned MarkLogic Node.js API dependent on itself.
npm install -f marklogic --save
Also, try installing the gulp and mocha command-line utilities
globally, as in:
npm install gulp -g
npm install mocha -g
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Alex Muir
[alex.g.m...@gmail.com]
Sent: Tuesday, April 05, 2016 6:00 AM
To: general@developer.marklogic.com
Subject: [MarkLogic Dev General] /node-client-api cannot get it working
Hi,
I'm running linux mint 17 with node --version v4.4.2 with npm --version 3.8.5
I've done the following to get the marklogic client api working
git clone https://github.com/marklogic/node-client-api.git
cd node-client-api/
npm install -f marklogic --save
npm install --save
however when I execute node etc/test-setup.js
admin user (default=admin): admin
admin password (default=admin):
checking for REST users
/mnt/mntlech/externalgit/node-client-api/node_modules/bluebird/js/main/async.js:43
fn = function () { throw arg; };
^
Error: POST /manage/v2/roles: cannot process response with 400 status
Any ideas what the problem may be going wrong there?
I can however connect and load documents using the examples
http://docs.marklogic.com/8.0/guide/node-dev/intro#
node loaddocument.js
Loaded the following documents:
/gs/aardvark.json
/gs/bluebird.json
/gs/cobra.json
so it suggests my credentials and ability to connect with node js at the very
least is working.
also if I run 'node gulp doc' in node-client-api/ i get errors as well
node gulp doc
module.js:327
throw err;
^
Error: Cannot find module '/mnt/mntlech/externalgit/node-client-api/gulp'
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:139:18)
at node.js:968:3
Regards
Alex
www.jahtoe.com<http://www.jahtoe.com>
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Need help in updating xml
Hi, Krishna: The persisted document has a serious problem. Everything up to the item element is persisted as XML. The contents of the item element are also XML but are persisted as text. That's why you have to unquote to get at the nested XML. You should take a look at the code that's persisting the document and fix that code so the whole tree is persisted as XML. Then, xdmp:node-replace() will work without issue on the nested elements. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of krishnanghri reddy [krishnang...@gmail.com] Sent: Thursday, March 31, 2016 9:15 AM To: general@developer.marklogic.com Subject: Re: [MarkLogic Dev General] Need help in updating xml Hi, I have an xml in the below format. I am trying to update Version element inside tag and facing an issue. "cannot update constructed nodes" I tried "in-mem-update.xqy" ,it updated the node on the fly and is not updating in the database. How to update the node in the database ? Here is the query I used: let $_ := mem:node-replace(xdmp:unquote(doc($uri")/*:pod_catalog_pod/*:item/text(),(),"repair-full")/*:pod/*:metadata/*:Version,http://podmetadata;>0) return $_ XML in the database. http://services-pod/1/;> online http://www.w3.org/2001/XMLSchema; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;> http://www.w3.org/2001/XMLSchema; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; Key="0385788e-70a7-4109-bbf9-7b5be6d7f690" xmlns="http://entities;> 1000516 search http://podmetadata;>podmetadata http://podmetadata;>Search Box http://podmetadata;>1 http://podmetadata;>2015-11-06T10:45:45.5223577Z http://podmetadata;>2016-01-12T01:05:33.5575597Z http://podmetadata;>description http://podmetadata;>testuser http://podmetadata;>testuser http://podmetadata;> 123576 http://podmetadata;>published Thanks Krishna ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Return of xdmp.filesystemFile() not a proper string object?
Hi, Florent:
To inspect the type of a returned object, you can call toString on the
prototype, as in:
Object.prototype.toString.call(xdmp.filesystemFile('/path/to/some.js'));
which yields:
[object xs.string]
So, an xs.string and not a JavaScript String or string literal, which you can
convert with valueOf():
Object.prototype.toString.call(xdmp.filesystemFile('/path/to/some.js').valueOf());
which yields:
[object String]
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Florent Georges
[li...@fgeorges.org]
Sent: Saturday, March 26, 2016 3:51 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Return of xdmp.filesystemFile() not a proper
string object?
Hi,
The documentation for xdmp.filesystemFile() (that is the JavaScript
version of it, at https://docs.marklogic.com/xdmp.filesystemFile) says
it returns a `String` object. But it seems to be of a different type.
I use the NPM package `yamljs`, so you can reproduce it, but I think
the example shows clearly the difference without requiring knowledge
of the package. Given you have a file tmp.yaml (e.g. with the content
"---\n foo: bar") then the following code (with an absolute path for
the YAML file):
var yaml = require('./node_modules/yamljs/lib/Yaml.js');
var src = xdmp.filesystemFile('.../tmp.yaml');
yaml.parse(src);
gives the following error:
JS-JAVASCRIPT: if (value.indexOf("\r") !== -1) { -- Error running
JavaScript request: TypeError: undefined is not a function
I assume that `value`, somewhere in the package code, comes from the
parameter to yaml.parse() and is supposed to be a String object,
containing a function `indexOf`. Indeed, if I explicitly create a
String object on the return value of `xdmp.filesystemFile()`, then it
succeeds:
var yaml = require('./node_modules/yamljs/lib/Yaml.js');
var src = xdmp.filesystemFile('.../tmp.yaml');
yaml.parse(new String(src));
The only conclusion I can reach is that the return value of
`xdmp.filesystemFile()` is not a proper String object. Did I miss
anything?
Regards,
--
Florent Georges
http://fgeorges.org/
http://h2oconsulting.be/
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Issue with field range and field value indexes
Hi, Jaydip:
Working with MarkLogic will be much easier and more flexible if you model each
book as a separate document. (If you're familiar with relational databases,
think of each document as a row, not as a table.)
Can you explain what you are trying to accomplish with the field? If you
create separate element value constraints for the title and genre, does that
work?
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Jaydip Rananavare
[jaydiprananav...@gmail.com]
Sent: Thursday, February 11, 2016 6:19 AM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] Issue with field range and field value
indexes
Hi Rob,
Thanks for help,
I tried debug options its gives me overview of what kind of queries are getting
created,
But still I am wondering why above query works fine when i pass
'book:(test-field:(Computer))' but does not work when I search for
'book:(test-field:(Fantasy))'
I tried same query with multiple files and my observation is it works fine only
if given value is present in first element.
In above case value "Computer" is present in first book element and "Fantasy"
is present in second book element.
On Thu, Feb 11, 2016 at 6:53 PM, Rob Szkutak
<rob.szku...@marklogic.com<mailto:rob.szku...@marklogic.com>> wrote:
Try adding true to your options and taking a look at the
cts:search() expression that's being generated. That should give you some clues.
Best,
Rob
Rob Szkutak
Senior Consultant
MarkLogic Corporation
rob.szku...@marklogic.com<mailto:rob.szku...@marklogic.com>
www.marklogic.com<http://www.marklogic.com>
From:
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>
[general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>]
on behalf of Jaydip Rananavare
[jaydiprananav...@gmail.com<mailto:jaydiprananav...@gmail.com>]
Sent: Thursday, February 11, 2016 5:57 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] Issue with field range and field value indexes
Hi All,
We are facing an issue with field value searches in below query,
example document is
Gambardella, Matthew
XML Developer's Guide
Computer
44.95
2000-10-01
An in-depth look at creating applications
with XML.
Ralls, Kim
Midnight Rain
Fantasy
5.95
2000-12-16
A former architect battles corporate zombies,
an evil sorceress, and her own childhood to become queen
of the world.
query is
xquery version "1.0-ml";
declare namespace html = "http://www.w3.org/1999/xhtml;;
import module namespace search = "http://marklogic.com/appservices/search; at
"/MarkLogic/appservices/search/search.xqy";
let $options := http://marklogic.com/appservices/search;>
unfiltered
true
true
false
http://marklogic.com/collation/en/S1;>
http://marklogic.com/collation/en/S1;>
/catalog/book/author
let $term := 'book:(test-field:(Midnight Rain))'
let $query := cts:query(search:parse($term, $options))
return search:resolve({$query}/node(), $options)
-
where test-field is path field with /catalog/book/title and
/catalog/book/genre paths and field value searches and positions are enabled
when we combined field-value search with element query as above query it does
not give results when we search for value that belongs to second book tag i.e
"Midnight Rain"
but when we search for Computer it gives us one result
same query works for both terms when we do word searches on field
Any help appreciated !
--
Thanks and regards,
Jaydip Rananavare
M +91 98 60 07 39 90
___
General mailing list
General@developer.marklogic.com<mailto:General@developer.marklogic.com>
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
--
Thanks and regards,
Jaydip Rananavare
M +91 98 60 07 39 90
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] using node streams to write many documents into database
Hi, Jakob: I'm glad to hear that mlcp may meet the practical requirement. > I would still be interested in getting the streaming functionality from Node > into MarkLogic because it would seem like a logical fit. +1 on the potential and the value of the goal. Even if an optimal Node.js implementation with workers, batch assignment, and backpressure would be non-trivial, something simpler would still be useful. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Jakob Fix [jakob@gmail.com] Sent: Tuesday, February 09, 2016 4:21 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] using node streams to write many documents into database Erik, thanks for the clarification. Actually I was coming from the Node angle this time around and see how I could use their streams capabilities. Yes, I think I'll be able to load the CSV file using mlcp. I'll give that a go. I would still be interested in getting the streaming functionality from Node into MarkLogic because it would seem like a logical fit. Thanks again. cheers, Jakob. On Tue, Feb 9, 2016 at 1:41 AM, Erik Hennum <erik.hen...@marklogic.com<mailto:erik.hen...@marklogic.com>> wrote: Hi, Jakob: We don't currently provide an equivalent to mlcp for Node.js A content pump for Node.js might have characteristics similar to the following: * forming documents from input records parsed from one or more input streams * adding documents to batches based on forest assignment * sending batches to the appropriate dnodes with multiple concurrent requests each for multiple worker processes * adding workers or providing backpressure to input streams as needed to maintain optimal throughput While that's possible and would be an interesting challenge, the streaming libraries available on npm and the Node.js client API certainly don't do all of that heavy lifting by themselves. Could mlcp be used for ingestion in your environment? Erik Hennum From: general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com>] on behalf of Jakob Fix [jakob@gmail.com<mailto:jakob@gmail.com>] Sent: Monday, February 08, 2016 3:29 PM To: General Mark Logic Developer Discussion Subject: [MarkLogic Dev General] using node streams to write many documents into database Hi, I've found the documentation that explains how to use a WritableStream to get /one/ document into MarkLogic, but I couldn't find any example where it shows how one could stream /many thousands/ of documents. The idea is to load a CSV file with > 1M lines as a ReadableStream and csv-parse and on each "readable" event to push the corresponding JSON object as a document into MarkLogic. The signature for the db.documents.createWriteStream [1] seems to require a document URI to be present at the time of the stream creation, which I cannot supply at the stage of stream creation. The example given in the documentation on how to load many documents doesn't really scale to "big data proportions" ... [2]. Thanks for any help. cheers, Jakob. [1] https://github.com/marklogic/node-client-api/blob/master/lib/documents.js#L468 [2] http://docs.marklogic.com/guide/node-dev/documents#id_18341 ___ General mailing list General@developer.marklogic.com<mailto:General@developer.marklogic.com> Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] If xml element exists
Hi, Pragya:
I'd like to make a different point:
* If you know exactly which document contains the information and need to
inspect that document, use fn:doc() to read the document.
* If you're not sure which document contains the information, use search
(whether cts:search() or the JSearch or Search APIs) to match the document that
contains the information.
To put it the other way, avoid reading a large set of candidate documents with
fn:doc and then checking each candidate document with XPaths to determine
whether the candidate contains the information.
(The equivalent to that antipattern in a relational database would be using a
stored procedure to read and filter a large number of candidate rows instead of
using an SQL WHERE clause to read only the appropriate rows.)
Either way, after you have retrieved the correct document, the suggestions from
the previous note apply to inspecting the document with XPaths.
Hoping that clarifies,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Kapoor, Pragya
[pkapo...@innodata.com]
Sent: Wednesday, January 27, 2016 9:52 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] If xml element exists
Erik,
If I understand correctly, you mean to say, if I know the document from where I
need to fetch the elements, still the cts:search will be faster than fn:doc()
For example: fn:doc("/misc/SavedTextSearches.xml")/TextSearches/User[Name =
$User] ) (: slower :)
cts:search(/TextSearches/User,
cts:element-value-query(xs:QName("Name"), $User) (: faster:)
Thanks
Pragya
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Erik Hennum
<erik.hen...@marklogic.com>
Sent: Wednesday, January 27, 2016 7:13 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] If xml element exists
Hi, Pragya:
If you need to retrieve the document anyway, you can test for the existence or
absence of nodes with:
http://docs.marklogic.com/fn:exists
http://docs.marklogic.com/fn:empty
as in
let $root:= fn:doc("/misc/SavedTextSearches.xml")/TextSearches
let $hasUser := ($root/User/Name/string(.) = $User)
let $isHidden := fn:exists($root/Hidden)
return ...
To find documents of interest based on criteria, however, you should use
cts:search() instead of fn:doc(),
fn:doc() retrieves documents into memory before inspecting the document, where
cts:search() matches against the indexes and thus (depending on the criteria)
retrieves many fewer documents much more quickly.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Kapoor, Pragya
[pkapo...@innodata.com]
Sent: Tuesday, January 26, 2016 10:32 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] If xml element exists
Few more examples:
if ($DocRecord[fn:not(.[Hidden])])
then
()
else
$DocRecord
if ($NumOfGroups = 1)
then
$DocRecords[fn:not(.[Hidden])]
else ()
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Kapoor, Pragya
<pkapo...@innodata.com>
Sent: Wednesday, January 27, 2016 11:38 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] If xml element exists
Hi,
I need to check the below conditions at multiple places, if there are other
more performant way to achieve these:
if ( fn:doc("/misc/SavedTextSearches.xml")/TextSearches/User[Name = $User] )
then
"YES"
else
"NO"
if ( fn:doc("/misc/SavedTextSearches.xml")/TextSearches )
then
( )
else 'p'
Thanks
Pragya
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action taken in reliance on this e-mail is
strictly prohibited and may be unlawful."
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action ta
Re: [MarkLogic Dev General] If xml element exists
Hi, Pragya:
If you need to retrieve the document anyway, you can test for the existence or
absence of nodes with:
http://docs.marklogic.com/fn:exists
http://docs.marklogic.com/fn:empty
as in
let $root:= fn:doc("/misc/SavedTextSearches.xml")/TextSearches
let $hasUser := ($root/User/Name/string(.) = $User)
let $isHidden := fn:exists($root/Hidden)
return ...
To find documents of interest based on criteria, however, you should use
cts:search() instead of fn:doc(),
fn:doc() retrieves documents into memory before inspecting the document, where
cts:search() matches against the indexes and thus (depending on the criteria)
retrieves many fewer documents much more quickly.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of Kapoor, Pragya
[pkapo...@innodata.com]
Sent: Tuesday, January 26, 2016 10:32 PM
To: MarkLogic Developer Discussion
Subject: Re: [MarkLogic Dev General] If xml element exists
Few more examples:
if ($DocRecord[fn:not(.[Hidden])])
then
()
else
$DocRecord
if ($NumOfGroups = 1)
then
$DocRecords[fn:not(.[Hidden])]
else ()
From: general-boun...@developer.marklogic.com
<general-boun...@developer.marklogic.com> on behalf of Kapoor, Pragya
<pkapo...@innodata.com>
Sent: Wednesday, January 27, 2016 11:38 AM
To: MarkLogic Developer Discussion
Subject: [MarkLogic Dev General] If xml element exists
Hi,
I need to check the below conditions at multiple places, if there are other
more performant way to achieve these:
if ( fn:doc("/misc/SavedTextSearches.xml")/TextSearches/User[Name = $User] )
then
"YES"
else
"NO"
if ( fn:doc("/misc/SavedTextSearches.xml")/TextSearches )
then
( )
else 'p'
Thanks
Pragya
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action taken in reliance on this e-mail is
strictly prohibited and may be unlawful."
"This e-mail and any attachments transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential , proprietary or
privileged information. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding, printing or
copying of this e-mail or any action taken in reliance on this e-mail is
strictly prohibited and may be unlawful."
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Search Constraints in XQuery
Hi, Ravindra: Have you tried search:parse()? That returned the expected query in my testing. Also, you can add true to your query options to see the query being executed by search:search(). If it works for search:parse() but not search:search(), that's a bug. What version are you on? Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Ravindra B Mamidipaka [ravi_...@hotmail.com] Sent: Friday, January 22, 2016 9:19 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Search Constraints in XQuery Hello Erik, Thank you for the suggestion eric, but that does not seems to work.. I get the following exception [1.0-ml] XDMP-AS: (err:XPTY0004) $container-elem as element() -- Invalid coercion: () as element() The workaround for now I did is created a field for the nested properties and have a range index.. This seems to be working but I want a generic enough solution as , I have 90 nested fields with different hierarchies and I do not want to create fields for each hierarchy.. Thanks, Ravi Hello Charles, Thank you for your suggestions but when I try that way.. It does not work.. Thanks Ravi > From: erik.hen...@marklogic.com > To: general@developer.marklogic.com > Date: Thu, 21 Jan 2016 19:28:23 + > Subject: Re: [MarkLogic Dev General] Search Constraints in XQuery > > Hi, Ravindra: > > It should work if you prefix the container before the contained > constraint, as in: > > let $search := "concept.registrar:username:WHITEMK" > > By the way, when you're trying to understand how a query text is being > parsed, > it's sometimes easier to work with search:parse() at first. > > > Hoping that helps, > > > Erik Hennum > > > From: general-boun...@developer.marklogic.com > [general-boun...@developer.marklogic.com] on behalf of Ravindra B > Mamidipaka [ravi_...@hotmail.com] > Sent: Wednesday, January 20, 2016 5:55 AM > To: MarkLogic Developer Discussion > Subject: Re: [MarkLogic Dev General] Search Constraints in XQuery > > Hello Charles, > Thank you for the suggestions.. Is it possible to provide a container > -query example for this case.. I am trying to wrap my head around the > concept and getting nowhere :).. Following is what I tried, and it does > not work > > let $search := "(username:WHITEMK) AND (concept.registrar: )" > > let $options := > http://marklogic.com/appservices/search;> > > > registrar > > > > > username > case-insensitive > wildcarded > > > false > false > false > false > false > unfiltered > unfaceted > format-json > score-simple > > > let $start := 1 > let $page-length :=1 > > return search:search($search, $options, $start, $page-length) > > Thanks, > Ravi > > > > > From: charles.gr...@marklogic.com > To: general@developer.marklogic.com > Date: Wed, 20 Jan 2016 04:04:06 + > Subject: Re: [MarkLogic Dev General] Search Constraints in XQuery > > Hello Ravi, > > Your guess is correct -- json-property can not be used for nested elements. > > You can add additional query terms to limit the search to 'registrar', > like a container-query > (that would be a runtime additional query to AND with the bare json-property) > > or if you can use a path range index (wouldn't be for matches and > wildcards though), > a path range index with path definition registrar/username would match > the location you have. > > Not sure how helpful this response is though.. > > Charles > > > > From: general-boun...@developer.marklogic.com > [general-boun...@developer.marklogic.com] on behalf of Ravindra B > Mamidipaka [ravi_...@hotmail.com] > Sent: Tuesday, January 19, 2016 12:54 PM > To: general@developer.marklogic.com > Subject: [MarkLogic Dev General] Search Constraints in XQuery > > > In the search:search in XQuery.. for search constraints , how do I > specify the json-property to be nested ? I am trying this and it does > not work > > xmlns="http://marklogic.com/appservic...<http://marklogic.com/appservices/search>"> > > > registrar/username > case-insensitive > <term-option& gt;wildc arded > > > > > > > Thanks, > > Ravi > > > > ___ General mailing list > General@developer.marklogic.com Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general > > ___ General mailing
Re: [MarkLogic Dev General] Search Constraints in XQuery
Hi, Ravindra: It should work if you prefix the container before the contained constraint, as in: let $search := "concept.registrar:username:WHITEMK" By the way, when you're trying to understand how a query text is being parsed, it's sometimes easier to work with search:parse() at first. Hoping that helps, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Ravindra B Mamidipaka [ravi_...@hotmail.com] Sent: Wednesday, January 20, 2016 5:55 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Search Constraints in XQuery Hello Charles, Thank you for the suggestions.. Is it possible to provide a container -query example for this case.. I am trying to wrap my head around the concept and getting nowhere :).. Following is what I tried, and it does not work let $search := "(username:WHITEMK) AND (concept.registrar: )" let $options := http://marklogic.com/appservices/search;> registrar username case-insensitive wildcarded false false false false false unfiltered unfaceted format-json score-simple let $start := 1 let $page-length :=1 return search:search($search, $options, $start, $page-length) Thanks, Ravi From: charles.gr...@marklogic.com To: general@developer.marklogic.com Date: Wed, 20 Jan 2016 04:04:06 + Subject: Re: [MarkLogic Dev General] Search Constraints in XQuery Hello Ravi, Your guess is correct -- json-property can not be used for nested elements. You can add additional query terms to limit the search to 'registrar', like a container-query (that would be a runtime additional query to AND with the bare json-property) or if you can use a path range index (wouldn't be for matches and wildcards though), a path range index with path definition registrar/username would match the location you have. Not sure how helpful this response is though.. Charles From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Ravindra B Mamidipaka [ravi_...@hotmail.com] Sent: Tuesday, January 19, 2016 12:54 PM To: general@developer.marklogic.com Subject: [MarkLogic Dev General] Search Constraints in XQuery In the search:search in XQuery.. for search constraints , how do I specify the json-property to be nested ? I am trying this and it does not work http://marklogic.com/appservic...<http://marklogic.com/appservices/search>"> registrar/username case-insensitive <term-option& gt;wildc arded Thanks, Ravi ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] XDMP-RWINVAL
Hi, Mahitha: The hostid prefix and separator was dropped, so the id is just the txid: http://docs.marklogic.com/guide/relnotes/chap4#id_43461 Given that the request had to go to the host with the transaction, the host id was redundant. You could consider using the Java API, which encapsulates such details for you. Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of Mahitha T U [mahita...@gmail.com] Sent: Wednesday, November 11, 2015 3:01 PM To: General Mark Logic Developer Discussion Subject: [MarkLogic Dev General] XDMP-RWINVAL Hi, In my project, from java code, i am invoking Marklogic Rest API to insert a document and read it, both within a transaction PUT /v1/documents?uri=abc.xml=_ GET/v1/documents?uri=abc.xml=_ This was working fine in Marklogic 7. But after upgrading to Marklogic 8 , during the insertion I am getting the following error and document is not getting inserted. Status 500: XDMP-RWINVAL: $0 _ set-transaction The credentials used has rest-reader, rest-writer privileges Please help. Thanks Mahitha ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] .sjs vs .js
Hi, David: > We are getting heavily into using Javscript in our projects in all tiers. That's great to hear. Feedback on what works and what can be improved is, of course, is very welcome. > We would like to be able to import/require javascript ending in .js 8.0-4 should (with the usual forward-looking caveats) support exactly that behavior by default. By contrast with library modules, *.js main modules will still default to treatment as resources for transfer to the client for execution in the browser (instead of server-side executables). Hoping that's useful, Erik Hennum From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of David Ennis [david.en...@hinttech.com] Sent: Thursday, September 17, 2015 6:07 PM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] .sjs vs .js Dear friends, We are getting heavily into using Javscript in our projects in all tiers. For practical purposes(sharing existing libraries), We would like to be able to import/require javascript ending in .js and not only .sjs In our build processes, we could refactor the libraries to use sjs, but that is not ideal. One of our developers found that my adding the mime-type to ML, he could successfully use .js files in ML. The Question: Does anyone know of any negative effect this might have - or possibly an opinion of how this would go against any best-practices for the JS part of ML? I see some concerns about (maybe ) a security issue (other JS files would then by default be accessible) - but that is all I can come up with Kind Regards, David Ennis David Ennis Content Engineer [HintTech] <http://www.hinttech.com/> Mastering the value of content creative | technology | content Delftechpark 37i 2628 XJ Delft The Netherlands T: +31 88 268 25 00 M: +31 63 091 72 80 [http://www.hinttech.com]<http://www.hinttech.com> [http://www.hinttech.com/signature/Twitter_HintTech.png] <https://twitter.com/HintTech> [http://www.hinttech.com/signature/Facebook_HintTech.png] <http://www.facebook.com/HintTech> [http://www.hinttech.com/signature/Linkedin_HintTech.png] <http://www.linkedin.com/company/HintTech> ___ General mailing list General@developer.marklogic.com Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Issue with order of request parameters
Hi, Rahul:
HTTP mandates that URI parameters are order-independent.
I'm wondering if it's an issue with the client. What client are you using? If
you try a different client (for instance, curl or wget or postman), does it
make a difference?
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of
rahul.kata...@cognizant.com [rahul.kata...@cognizant.com]
Sent: Thursday, September 03, 2015 1:47 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Issue with order of request parameters
The order of the REST request parameters is working fine if q is the first
request parameter.
Wrong Format:
/v1/search?collection=Cars=test-transform<http://172.17.103.44:9000/v1/search?collection=Lessonplan=search-lessonplan-transform-json=facets=json=Auth%3aorgId%3dPERF0006%3bplatform%3dTCK>=test
Correct Format:
/v1/search?q=test=test-transform=Cars<http://172.17.103.44:9000/v1/search?options=search-lessonplan-transform-json=Lessonplan=json=Auth%3aorgId%3dPERF0006%3bplatform%3dTCK>
Is q supposed to be the first parameter of the REST request?
Thanks,
Rahul K
From: Kataram, Rahul (Cognizant)
Sent: Thursday, September 03, 2015 1:12 PM
To: general@developer.marklogic.com
Subject: Issue with order of request parameters
Hi,
There is an issue with the order of the request parameters passed in the URL of
ML 6 REST Server.
We have identified that the request parameter "collection" should be passed
only after the "options" parameter.
If not passed, the collection is being passed as empty and we get 'XDMP-URI:
cts:collection-query("") -- Invalid URI format: ""' error
Wrong Format:
/v1/search?collection=Cars=test-transform<http://172.17.103.44:9000/v1/search?collection=Lessonplan=search-lessonplan-transform-json=facets=json=Auth%3aorgId%3dPERF0006%3bplatform%3dTCK>
Correct Format:
/v1/search?options=test-transform=Cars<http://172.17.103.44:9000/v1/search?options=search-lessonplan-transform-json=Lessonplan=json=Auth%3aorgId%3dPERF0006%3bplatform%3dTCK>
Can anyone confirm, is this supposed to be the order in which they are supposed
to be passed?
Thanks,
Rahul K
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended recipient(s), please reply to the sender and
destroy all copies of the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email,
and/or any action taken in reliance on the contents of this e-mail is strictly
prohibited and may be unlawful. Where permitted by applicable law, this e-mail
and other e-mail communications sent to and from Cognizant e-mail addresses may
be monitored.
___
General mailing list
General@developer.marklogic.com
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Server Message: SVC-EXTIME: xdmp:lock-for-update(/docs/0f04c4fb-6058-4758-92eb-f9cfa2ea18bc) -- Time limit exceeded.
Hi, Abhinav:
It looks like you are hitting the transaction time limit before committing the
multi-statement transaction, so the transaction no longer exists for those
writes.
It's not a good idea to build up huge transactions. In particular, the code
that's trying to pool the multi-statement transactions might misunderstand the
purpose of multi-statement transactions in MarkLogic.
Really, multi-statement transactions are for changes that must commit or fail
together to leave the database in a consistent state. The canonical examples
are a balance transfer or the write and delete operations for a document move.
If the write for one document failed to write (maybe it had a bad format) and
the rest wrote correctly, would that be preferable to having all writes fail?
If so, you shouldn't use multi-statement transactions. Single document writes
are still transactional without the explicit multi-statement transaction.
If you're using MarkLogic 8, you might also consider writing documents in bulk
(typically of around 100 for optimal sized documents) for better performance.
Bulk writes are also transactional.
Hoping that helps,
Erik Hennum
From: general-boun...@developer.marklogic.com
[general-boun...@developer.marklogic.com] on behalf of
abhinav.mish...@cognizant.com [abhinav.mish...@cognizant.com]
Sent: Tuesday, August 18, 2015 5:05 AM
To: general@developer.marklogic.com
Subject: Re: [MarkLogic Dev General] Server Message: SVC-EXTIME:
xdmp:lock-for-update(/docs/0f04c4fb-6058-4758-92eb-f9cfa2ea18bc) -- Time
limit exceeded.
Hi Team,
Just want add few more info. We are sometimes getting below errors as well.
Java Layer:
com.marklogic.client.FailedRequestException: Local message: transaction
rollback failed: Bad Request. Server Message: XDMP-NOTXN:
xdmp:xa-complete1(xs:unsignedLong(5231619190085809367),
xs:unsignedLong(13101616944000907690), fn:false()) -- No transaction with
identifier 13101616944000907690
at
com.marklogic.client.impl.JerseyServices.completeTransaction(JerseyServices.java:1488)
at
com.marklogic.client.impl.JerseyServices.rollbackTransaction(JerseyServices.java:1420)
at
com.marklogic.client.impl.TransactionImpl.rollback(TransactionImpl.java:70)
MarkLogic Log:
2015-08-18 15:39:24.551 Info: REST-ContentStore: Status 500: XDMP-NOTXN:
xdmp:invoke(/MarkLogic/rest-api/lib/transaction-runner-update.xqy,
(as:QName(, method), PUT, as:QName(, headers), ...), options
xmlns=xdmp:evaltransaction-id13101616944000907690/transaction-id/options)
-- No transaction with identifier 13101616944000907690
2015-08-18 15:39:25.347 Info: REST-ContentStore: Status 500: XDMP-NOTXN:
xdmp:xa-complete1(xs:unsignedLong(5231619190085809367),
xs:unsignedLong(13101616944000907690), fn:false()) -- No transaction with
identifier 13101616944000907690
Regards,
Abhinav
From: Mishra, Abhinav Kumar (Cognizant)
Sent: Tuesday, August 18, 2015 12:24 PM
To: general@developer.marklogic.com
Subject: Server Message: SVC-EXTIME:
xdmp:lock-for-update(/docs/0f04c4fb-6058-4758-92eb-f9cfa2ea18bc) -- Time
limit exceeded.
Hi Team,
We are running into an issue, where in we are getting exception from MarkLogic
as given below:
“write failed: Internal Server Error. Server Message: SVC-EXTIME:
xdmp:lock-for-update(/docs/0f04c4fb-6058-4758-92eb-f9cfa2ea18bc) -- Time
limit exceeded.”
But In MarkLogic logs I don’t see any log for the above statement. I have
enabled log to debug mode on MarkLogic as well. We are not sure what is the
exact reason of this failure.
I would really appreciate if you can provide some clues.
Now let me explain what we are doing..
We are saving xml documents to MarkLogic from a java based application. We have
used MarkLogic’ s OOTB REST Java Client API. It will internally use
”/documents” service via PUT request.
I have highlighted the RESTServices.putDocument(..) method below which will
actually put the documents to MarkLogic.
We are opening the transaction, saving document to MarkLogic and then closing
the transaction.
Code snippet:
//rest-user has all the privileges required to load the documents in
MarkLogic.
DatabaseClient client = DatabaseClientFactory.newClient(“somehost”,
8004,rest-user, rest-user-password, Authentication.DIGEST);
Transaction transaction = client.openTransaction();
try {
//Here we are sending following parameters from outside, “[String
path, InputStream is, Transaction transaction, String collection]”
//Where, path=/docs/0f04c4fb-6058-4758-92eb-f9cfa2ea18bc”
// InputStream will have the xml document as a stream
// transaction is the implementation of com.marklogic.clinet.Transaction
// collection = “doc-versions”
TransactionPoolableFactory transactionFactory = new
TransactionPoolableFactory();
GenericKeyedObjectPoolTransaction, DatabaseClient transactionPool =
new GenericKeyedObjectPoolTransaction, DatabaseClient
