Re: [MarkLogic Dev General] Transitioning from SQL

2015-06-12 Thread Damon Feldman
ormalized dates, etc. [...] This is nice because you can change formats or data inside the section and your Java application can still pull out just the portion without your header changes affecting the content. It decouples the DB changes and Java dev changes. Da

Re: [MarkLogic Dev General] Boosting with simple scoring

2015-06-12 Thread Damon Feldman
What makes the logtfidf score inappropriate for metadata? It occurs to me that if you put the metadata in every doc one time (though not in the metadata section if it is not really “there” for the doc) you’ll fool the system into thinking that all metadata has the same tf and idf, so more like

Re: [MarkLogic Dev General] Transitioning from SQL

2015-06-15 Thread Damon Feldman
s, what are the tradeoffs ? Regards, Hank On Fri, Jun 12, 2015 at 10:57 AM, Damon Feldman wrote: > Hank, > > I think your idea to store something similar to the existing entities in your > Java layer is a good one, at least as a starting point. NoSQL Document > databases ten

Re: [MarkLogic Dev General] Migrating from ML 4.x to ML 8

2015-06-29 Thread Damon Feldman
Danny's upgrade path should definitely work (and should be tested before done in production). Depending on database size and how repeatable your deploy process is with respect to configuring MarkLogic, as well as the outage window available, you might also simply export your data to disk via m

Re: [MarkLogic Dev General] encryption of username and password in gardle properties and set different user for harmonization

2018-03-29 Thread Damon Feldman
Vikas, Some information on encrypting passwords is located here: https://github.com/marklogic-community/ml-gradle/wiki/Encrypting-passwords . Let us know if that meets your need for item 1. Damon From: general-bounces-dev-marklogic on behalf of "vikas.sin...@cognizant.com" Reply-To: MarkLog

Re: [MarkLogic Dev General] Near Query with min-occurs

2018-04-04 Thread Damon Feldman
Praveen, Can you tell us the use case? It looks like you might want a phrase query on a set of terms like property tax, propensity tax, etc, and perhaps some notion of relevance or scoring where 4 mentions is more important than just one or two. There are other tools that may help if you tell

Re: [MarkLogic Dev General] Change the collection TAG for set of documents

2018-04-11 Thread Damon Feldman
Hi, The collections are part of the document, so updating the collections updates the document. The document is the unit of caching, indexing and rewrite, and it is optimized in various ways, including being rewritten to a single, continuous record on disk – therefore it will be rewritten to di

Re: [MarkLogic Dev General] how to save multiple documents to file system.

2018-05-10 Thread Damon Feldman
Hi, I believe you are saving both docs to the file-system with two separate calls. I suggest when staring (and even generally) to include the XQuery header: declare option xdmp:mapping "false";. Which will not treat a sequence as an instruction to call a function multiple times. That’s w

Re: [MarkLogic Dev General] Collections and documents

2012-09-10 Thread Damon Feldman
Ankur, Modifying collections on a document is just like modifying the XML (in fact a collection is a lot like an invisible XML element in the document) - it causes a rewrite of the document itself to disk. So the real question is how to efficiently update a large number of existing documents. T

Re: [MarkLogic Dev General] ML 6 upgrade, reindexing, and CPU load

2012-09-18 Thread Damon Feldman
David, You may want to set the "reindexer throttle" to a lower number for that database, if you have not already tried that. Yours Damon -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of David Sewell Sent: Tue

Re: [MarkLogic Dev General] XDMP-RANGEINDEX while invoking a module

2012-09-19 Thread Damon Feldman
Raghu, Are you accidentally adding the userId inside the modification date? The index on modificationdate will index the "atomized" value fn:data(1232002-11-11) => 1232002-11-11. So that might explain why you corrupt your date value to be invalid only when you add a child. Just a guess. If yo

Re: [MarkLogic Dev General] Multiple inserts, maintaining order

2012-09-19 Thread Damon Feldman
Dominic, You could instead build the sequence, merge it together and then replace the whole meta node. Un-tested code: Let $new-fields := for return { {$old-meta/@*} (: keep all old attributes :) {$old-meta/node()[local-name(.) ne "field"] (: keep all nodes except fields :)

Re: [MarkLogic Dev General] Issue with search api constraints

2012-10-01 Thread Damon Feldman
Sana, I think MarkLogic will be great for modeling virtual worlds, like multiplayer games. The scene or part of the world that people are interacting with, and each person's personal profile or situation would very naturally be modeled as documents. An interaction would access all the visible

Re: [MarkLogic Dev General] Scala and XCC

2012-10-18 Thread Damon Feldman
erson[name="Bob"]//ssn.text() and just use the resulting string in Scala. If you do return XML, I suspect Scala's XML parsing (scala.xml.pull.Source ?) will parse it to the Scala mode most efficiently. Damon -- Damon Feldman, Sr. P

Re: [MarkLogic Dev General] Scala and XCC

2012-10-18 Thread Damon Feldman
. San Diego CA Sally Smith 1234 Elm St. Los Angeles CA On Oct 18, 2012, at 7:40 PM, Damon Feldman mailto:damon.feld...@marklogic.com>> wrote: David

Re: [MarkLogic Dev General] Too Many Stands

2012-10-30 Thread Damon Feldman
ning that are causing the ExpandedTreeCacheFull error? Often a query causes that because it is accessing too much of the database and can be fixed by changing the query approach. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun..

Re: [MarkLogic Dev General] Xquery - Maps

2012-11-21 Thread Damon Feldman
quot; (see MarkLogic docs for server field functions). Can you tell us what you need to accomplish so we can help you pick the right tool for the job? It may also help us tell you which way to use maps. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic ___

Re: [MarkLogic Dev General] Marklogic Transaction Commit - 1K document

2012-12-01 Thread Damon Feldman
/test/",$x) return xdmp:document-insert($uri,$xml) ') And you should see better performance because each eval runs a separate transaction. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@develope

Re: [MarkLogic Dev General] Export from Marklogic

2012-12-03 Thread Damon Feldman
Sini, I'm not sure exactly what you mean, but you can do this with simple templating: let $doc1 := let $doc2 := let $combined := {$doc1/node(), $doc2/node()} return xdmp:save("c:/tmp/combined-doc.xml", $combined) Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic

Re: [MarkLogic Dev General] Export from Marklogic

2012-12-04 Thread Damon Feldman
:= $data/text() let $fileName := concat($ID,".xml") return xdmp:save("/tmp/Demo/example.xml", doc($fileName)) Thanks On Mon, Dec 3, 2012 at 2:58 PM, Damon Feldman mailto:damon.feld...@marklogic.com>> wrote: Sini, I'm not sure exactly what you mean, but you can do

Re: [MarkLogic Dev General] using maps - when to clear

2012-12-07 Thread Damon Feldman
:map(); [...] any code that imports the module can share that map (update or read) but other concurrent or subsequent requests will have their own copy. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@dev

Re: [MarkLogic Dev General] Filesystem caching

2012-12-12 Thread Damon Feldman
temp directory and then back to the directory with your code. Please let us know if a faster workaround works. Yours, Damon -- Damon Feldman Senior Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Iain

Re: [MarkLogic Dev General] cpf install not scriptable via admin api?

2012-12-13 Thread Damon Feldman
I think a couple people have scripted CPF install, including Roxy, though I have not tried it myself. See: https://lists.marklogic.com/message/c67fdpx5cdgchvy4. Damon From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com]

Re: [MarkLogic Dev General] cpf install not scriptable via admin api?

2012-12-13 Thread Damon Feldman
klogic.com [general-boun...@developer.marklogic.com] on behalf of Damon Feldman [damon.feld...@marklogic.com] Sent: Thursday, December 13, 2012 9:38 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] cpf install not scriptable via admin api? I think a couple people have sc

Re: [MarkLogic Dev General] content pump

2013-01-03 Thread Damon Feldman
Manoj, Can you describe the configuration and transformation, and post the transformation code? Yours, Damon From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of manoj viswanadha Sent: Thursday, January 03, 2013 12:27 AM To: MarkLogic Dev

Re: [MarkLogic Dev General] How a Task Server works in cluster

2013-01-07 Thread Damon Feldman
pages? Are those processes properly coded, and can you profile them in queryConsole? Might there be lock contention issues or DEBUG-level retry messages showing up in the logs? Yours, Damon -- Damon Feldman Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com

Re: [MarkLogic Dev General] "left join" using search:search or cts:search?

2013-01-08 Thread Damon Feldman
Jakob, I want to highlight that Charles' code uses range index values on both "sides" of the join. He pulls values using element-values(), so that comes from a range index. Then he uses element-range-query() to do the shotgun OR. This is important because the values will line up properly (parti

Re: [MarkLogic Dev General] directory facet?

2013-01-16 Thread Damon Feldman
ot;, (), $q) ! functx:substring-before-last-fslash(.)) Note the "!" operator is a shorthand for a for loop in ML 6. Note the function is based on functx, but has the "/" delimiter hard-coded. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic -Original Messa

Re: [MarkLogic Dev General] is cts:search of root attributes possible?

2013-01-25 Thread Damon Feldman
Tim, There was a change in some version (I forget which) where some queries (particularly involving attributes) now apply to the element they are immediately nested inside of. I wonder if your e-a-v-q is only looking for matches properly inside /record, instead of including the /record element

Re: [MarkLogic Dev General] Cannot install MarkLogic 6 on Centos 6.0

2013-02-01 Thread Damon Feldman
Ashwini, There are some required libraries (notably, both the 32 and 64 bit versions of glibc) that are required. Please refer to the install instructions at: http://docs.marklogic.com/guide/installation. Yours, Damon From: general-boun...@developer.marklogic.com [mailto:general-boun...@deve

Re: [MarkLogic Dev General] regarding error: XDMP-EXPNTREECACHEFULL

2013-02-04 Thread Damon Feldman
nd use any relational tool, or Excel, to convert the query result to .csv. Does this help in your case? Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of ranjan sarma Se

Re: [MarkLogic Dev General] finding an id that does not exist

2013-02-08 Thread Damon Feldman
Paul, I believe you can range-index the uniqueId, element or attribute, then call cts:element-values() with the option to return data as a map. You can put your other sequence into a map also and "subtract" maps via the "-" operator to get a fast set difference. Yours, Da

Re: [MarkLogic Dev General] Triggers Handling : Directory scope constraint and collection

2013-02-08 Thread Damon Feldman
hout using triggers in either case. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Stephane Toussaint Sent: Friday, February 08, 2013 9:12 AM To:

Re: [MarkLogic Dev General] finding an id that does not exist

2013-02-08 Thread Damon Feldman
to 1 return map:put($m2, xs:string(xdmp:random(100)), true()) return ( count(map:keys($m1 - $m2)), xdmp:elapsed-time() ) Yours, Damon From: Paul M [mailto:pjm...@yahoo.com] Sent: Friday, February 08, 2013 10:54 AM To: Damon Feldman; MarkLogic Developer Discussion Subject: Re

Re: [MarkLogic Dev General] finding an id that does not exist

2013-02-08 Thread Damon Feldman
ry) Damon From: Paul M [mailto:pjm...@yahoo.com] Sent: Friday, February 08, 2013 11:38 AM To: Damon Feldman; MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] finding an id that does not exist xquery version "1.0-ml"; declare namespace html = "http://www.w3.org/1999/

Re: [MarkLogic Dev General] UDFs, Filtered, Unfiltered

2013-02-11 Thread Damon Feldman
Bek, If you want to access and re-write arbitrary content in MarkLogic rather than work with indexes you can use Hadoop's MapReduce capabilities directly with MarkLogic as shown in http://docs.marklogic.com/guide/mapreduce. Can you tell us more about the MapReduce task you'd like to run - perh

Re: [MarkLogic Dev General] storing a html 'as it is' in xml

2013-02-12 Thread Damon Feldman
sion (6.0-2) the former serialized with an explicit close tag but the latter serializes as a self-closing tag. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Raghu Sent: Tues

Re: [MarkLogic Dev General] Somebody please help me!

2013-02-12 Thread Damon Feldman
. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Raghu Sent: Tuesday, February 12, 2013 7:10 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General

Re: [MarkLogic Dev General] Somebody please help me!

2013-02-13 Thread Damon Feldman
can: Hamlet Shakespeare For efficiency, ease of coding and human readability. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Vedavalli Ra

Re: [MarkLogic Dev General] Triggers Handling : Directory scope constraint and collection

2013-02-18 Thread Damon Feldman
ate "set-definition" document to track "collections" (but in that case beware of lock contention on the set definition document). Or tolerate the extra triggers and ensure they do no harm when fired additional times. Yours, Damon -- Damon Feldman Sr. Principal Consultant,

Re: [MarkLogic Dev General] search:page-length

2013-02-18 Thread Damon Feldman
You should set that option to the largest number of documents your overall system or user can comfortably handle - at some point you will exceed various timeouts, exhaust memory, overwhelm your users, or cause system failures at all layers of your architecture (including but not limited to MarkL

Re: [MarkLogic Dev General] (no subject)

2013-02-18 Thread Damon Feldman
you post a working query we can give more specific advice. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Varun Varunesh Sent: Monday, February 18, 2013 12:01 PM To

Re: [MarkLogic Dev General] Asyncronous Status Updates

2013-02-24 Thread Damon Feldman
rhead. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Tim Sent: Saturday, February 23, 2013 3:16 PM To: 'MarkLogic Developer Discussion' Subject: [MarkLo

Re: [MarkLogic Dev General] Asyncronous Status Updates

2013-02-24 Thread Damon Feldman
used by many/all transactions, even with different elements for different statuses, there may be lock contention. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Tim Se

Re: [MarkLogic Dev General] Single transaction and rollback on failure

2013-02-27 Thread Damon Feldman
Jonna, To commit to two different databases you need an XA transaction controller, such as a JTA component in the JEE world, that runs outside MarkLogic. Can you tell us the business need behind the technical requirement of using two separate but transactionally-coherent databases? There may be

Re: [MarkLogic Dev General] problem interpreting an XPath expression?

2013-02-27 Thread Damon Feldman
ss is an attribute node, but contains needs a string. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Jakob Fix Sent: Wednesday, February 27, 2013 4:07 PM To: Ge

Re: [MarkLogic Dev General] Tree Cache Ful

2013-03-07 Thread Damon Feldman
? MarkLogic assigns a set portion of the system RAM to that cache when you install the software, so the RAM on the machine at install time will determine cache settings. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.

Re: [MarkLogic Dev General] Facing performance issue with element-attribute-value-query/element-attribute-range-query/path-range-query on attributes

2013-03-14 Thread Damon Feldman
query in queryConsole, and/or turn on query-trace via the function of that name. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of gnanaprakash.bodire...@cognizant.com

Re: [MarkLogic Dev General] Elapsed time in xdmp:query-meter() , Profile time is showing more difference (Why it is not same for same query)- reg.,

2013-03-20 Thread Damon Feldman
re showing you that your commit time is very, very high. Only 11 updates in 2 minutes indicates something is wrong. Do you have huge documents or system problems of some kind? Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mail

Re: [MarkLogic Dev General] search:estimate

2013-03-21 Thread Damon Feldman
() is part of the low-level search api, along with cts:search(), that uses cts:query constructs. search:search() and search:resolve() use the XML serialization of those cts:query structures, so for xdmp:estimate(), you need to convert via cts:query() Yours, Damon -- Damon Feldman Sr. Pr

Re: [MarkLogic Dev General] Time limit exceeded

2013-03-28 Thread Damon Feldman
Sini, You may have changed the max time limit, rather than default time limit. I believe the default time limit is what terminates a normal request, and max time limit disallows an xdmp:eval() from increasing the time limit past a certain point. Fortunately, if you only need the URIs you can u

Re: [MarkLogic Dev General] Deadlock detected locking

2013-04-15 Thread Damon Feldman
Gurbeer, The Debug messages in the logs are normal, and should not affect processing. The deadlock message means the server had two writes deadlock, so it backed one off and re-tried it automatically. It says "Update 1" so it did not get repeatedly re-started, which you would see if you had

Re: [MarkLogic Dev General] Try Catch Conflicting updates

2013-04-19 Thread Damon Feldman
-- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Pradeep Maddireddy Sent: Friday, April 19, 2013 10:15 AM To: General Mark Logic Developer Discussion Subject: [MarkLogic Dev General] Try

Re: [MarkLogic Dev General] Try Catch Conflicting updates

2013-04-22 Thread Damon Feldman
ted property." } catch($err) { "Error updating property." } (: - :) Thanks Pradeep Maddireddy On Fri, Apr 19, 2013 at 8:09 PM, Damon Feldman mailto:damon.feld...@marklogic.com>> wrote: Pradeep, The commit of all updates from a transaction occurs after the XQuery is done

Re: [MarkLogic Dev General] Try Catch Conflicting updates

2013-04-22 Thread Damon Feldman
t had no other option. Thanks Pradeep Maddireddy On Mon, Apr 22, 2013 at 6:06 PM, Damon Feldman mailto:damon.feld...@marklogic.com>> wrote: Pradeep, I see that behavior too (thanks for the excellent, self-contained example code). I can only assume that recent versions of MarkLogic added s

Re: [MarkLogic Dev General] Maximum size for a map?

2013-05-06 Thread Damon Feldman
up but can simplify your code. If you describe the nature of the lookup we can brainstorm other ideas. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Beha

Re: [MarkLogic Dev General] Attaching more than one forest to a DB in ML

2013-05-15 Thread Damon Feldman
different from many systems where the developer is burdened by sharding code and managing the locations of documents explicitly, so you may find it much easier. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:ge

Re: [MarkLogic Dev General] string-join output not as expected

2013-05-16 Thread Damon Feldman
XQuery has two syntactic modes - one for inside direct XML element expression and one everywhere else, and the {} toggles into the normal mode. In any case, remember that these expressions are using XML strings as syntax but are building tree data structures. Yours, Damon -- Damon Feldma

Re: [MarkLogic Dev General] cts:cluster

2013-05-21 Thread Damon Feldman
Varun, cts:cluster will group similar documents based on important terms in the documents, including words, element/word pairs and similar. If you build a separate document with only the people, you may be able to group them using cts:cluster, but cluster is intended for moderate sized sets or

Re: [MarkLogic Dev General] ML Project Issues

2013-05-21 Thread Damon Feldman
Hi Bob, nice to see you on the lists. Are the errors logged to ErrorLog.txt taking up 2MB, or is that being logged somewhere else? There may be very large records involved that are included in the log if that's the case. I have not encountered that problem before, so it will help if you can cla

Re: [MarkLogic Dev General] Unable to get the value of Element Š.

2013-05-28 Thread Damon Feldman
Kashif, You need to get XML into the system to use XQuery - XQuery does not work on strings that happen to have XML in them, but on XML documents. MarkLogic adds "unquote," "document-insert" etc. because there is now method in the XQuery spec to add XML. You must use implementation-specific m

Re: [MarkLogic Dev General] Termlist database error

2013-05-28 Thread Damon Feldman
which suggests you are operating outside the ideal parameters of the system. If you post the forest sizes we can confirm that. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On

Re: [MarkLogic Dev General] Termlist database error

2013-05-28 Thread Damon Feldman
general-boun...@developer.marklogic.com<mailto:general-boun...@developer.marklogic.com> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Damon Feldman Sent: Tuesday, May 28, 2013 2:34 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Termlist database error Gar

Re: [MarkLogic Dev General] Computing ML uptime percentage (Danny Sinang)

2013-05-30 Thread Damon Feldman
Danny, No - this will require working memory for perhaps 2x the log file size (1x for the overall file and 1x for the tokenized sequence of lines), but the ETC only holds persisted documents from the database. ETC needs to hold all documents for all concurrent transactions that are ongoing on e

Re: [MarkLogic Dev General] Adding a new node to cluster

2013-06-12 Thread Damon Feldman
node, or re-ingest the content. Yours, Damon -- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Gourav Kakkar Sent: Wednesday, June 12, 2013 7:01 AM To: general@developer.marklogic.com

Re: [MarkLogic Dev General] Queryfor searchingdocuments witha null node

2013-06-17 Thread Damon Feldman
ade change ever, since that was when MarkLogic adopted the then-fairly-new XQuery 1.0 spec.) You should not need to write code for the search function, as I recall. If you are done, then congratulations - but if not feel free to post your issues and approach to this list. Yours, Damon -- Damon

Re: [MarkLogic Dev General] Queryfor searchingdocuments witha null node

2013-06-17 Thread Damon Feldman
<mailto:general-boun...@developer.marklogic.com> [mailto:general-boun...@developer.marklogic.com] On Behalf Of Damon Feldman Sent: Monday, June 17, 2013 7:13 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Queryfor searchingdocuments witha null node Mariano, What are the

Re: [MarkLogic Dev General] Merge Policy / Large Delete

2013-06-25 Thread Damon Feldman
-- Damon Feldman Sr. Principal Consultant, MarkLogic From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Alex Milowski Sent: Tuesday, June 25, 2013 2:16 PM To: General Mark Logic Developer Discussion Subject: [MarkLogic Dev General] Merge Policy

Re: [MarkLogic Dev General] Best approach for intersect/AND operation

2013-06-26 Thread Damon Feldman
Gnanaprakash, If you are looking for identical elements in two sequences, you can put them in maps for fast set intersection. Using sequences has O(n^2) performance because it's basically a nested loop checking for pair-wise equality. Maps will use hashing, so have linear performance. let $x :

Re: [MarkLogic Dev General] Best approach for intersect/AND

2013-06-27 Thread Damon Feldman
Gnana, The map is a hash map, so I assume the order of the elements is affected by how they get allocated to hash buckets. You could try this instead: let $x := (1,2,3,4,5, 9) ! xs:string(.) let $y := (2,3,6,7,9, 10, 11, 22, 101) ! xs:string(.) let $mx := map:map() let $add-all := for

Re: [MarkLogic Dev General] Performance degradation sequence intersection

2013-07-17 Thread Damon Feldman
Abhisekh, There are some examples using map:map() structures that may help in a thread from last month: http://markmail.org/thread/c4hq6n3mwbuxnl2i. Yours, Damon From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Abhishek53 S Sent: Wedne

Re: [MarkLogic Dev General] Node value replace

2013-08-10 Thread Damon Feldman
I don't quite understand the desired behavior. Can you build a self-contained example and show the desired output so someone can paste in qconsole and run it? Typically, you insert a document, use a semicolon to start a new transaction, then show the query. Yours, Damon

Re: [MarkLogic Dev General] Issue with Profilling - reg.,

2013-09-20 Thread Damon Feldman
Santhosh, When you say hanged up, is it that the function never returns (and is visible in the server status tabs on port 8001) or it returns but no output is displayed, or that it times out? I can see a few possibilities - one is that there is a bug in the profiler possibly related to deep re

Re: [MarkLogic Dev General] full search or not

2013-09-23 Thread Damon Feldman
a database can return huge amounts of data, however, so you may need to use another technique to limit the results. Yours, Damon -- Damon Feldman MarkLogic Corporation www.marklogic.com -Original Message- From: general-boun...@developer.marklogic.com [mailto:general-boun

Re: [MarkLogic Dev General] Count the occurrences of highlighted text

2013-09-28 Thread Damon Feldman
If simplicity is your goal, you might avoid xdmp:set() logic via: let $doc := Google is a big data company with lots of data. let $terms := ('Google','data','company') let $query := cts:or-query($terms) (: in a query a string is coerced to a cts:word-query() by default :) return count(cts:hig

Re: [MarkLogic Dev General] Query against index

2013-10-03 Thread Damon Feldman
Shripad, Could you describe a more concrete use case of what you want to accomplish? I’m not familiar with using token parsing for ranking, so the big picture will help here. Yours, Damon -- Damon Feldman MarkLogic Corporation www.marklogic.com<http://www.marklogic.com/> From: genera

Re: [MarkLogic Dev General] Security on MarkLogic Local Host Fail Over

2014-03-17 Thread Damon Feldman
Heather, Great point - it is critical to also have replicas of the forests for security and schemas. I would even suggest multiple replicas for those forests, as they are small and not only critical to your database being up, but also required for your administrative access to work properly whe

Re: [MarkLogic Dev General] path range query taking more time when multiple values are supplied

2014-05-05 Thread Damon Feldman
Ganaprakash, That first form is a 4-way OR so may be slightly more complex to build, though I don't see it on my machine. This line - the actual building of the constraint structure - is likely to have a minor performance impact compared to the actual query execution. Is there a reason you wa

Re: [MarkLogic Dev General] Browsing directories

2014-05-20 Thread Damon Feldman
Florent, I think you could easily parse the slash-terminated substrings out of cts:uris(). This should be quite fast, unless your database has a huge number of documents. I don't think directory creation would be worth the overhead - you can get lock contention as well if you are not careful ho

Re: [MarkLogic Dev General] Strict validation error encountered in ML7 but not in ML6

2014-07-16 Thread Damon Feldman
I can only think of wanting to clear the Expanded Tree Cache when you are doing performance testing and want to compare cold to warm caches. MarkLogic automatically invalidates documents in the cache when they change, so there should be no impact on your running system other than slowing down p

Re: [MarkLogic Dev General] mlcp export problem/question

2014-07-25 Thread Damon Feldman
I'd be interested to know what MarkLogic is doing around the time of the lost connection. The app server status page will show active queries and how long they have been running (click "show more" and mouse-hover over the query description). I believe mlcp requests will show the request size (

Re: [MarkLogic Dev General] Minutely scheduled job is executing more than once within a minute - reg.,

2014-09-16 Thread Damon Feldman
an option depending on the update logic. Best, Damon -- Damon Feldman, MarkLogic Consulting From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] on behalf of santhosh.rajasekar...@cognizant.com [santhosh.rajasekar

Re: [MarkLogic Dev General] cts:highlight for large document

2011-02-17 Thread Damon Feldman
Judie, As of version 4.1 of MarkLogic Server, cts:highlight allows you to use xdmp:set with $cts:action to stop the processing after some time. You might count the matches via xdmp:set, then stop after you have a few. Alternatively, you can transform the document to be smaller or simpler, which

Re: [MarkLogic Dev General] efficient storage/retrieval scheme

2011-03-02 Thread Damon Feldman
Mike, Items 2 and 3 are both great approaches, IMO. Item 1 seems less clean and I suggest keeping discrete system values in attributes instead. Lexicons, including a collection lexicon, uri lexicon, or range indexs, are always the fastest, but do consume more memory. That may be a premature op

Re: [MarkLogic Dev General] namespace

2011-03-03 Thread Damon Feldman
Strictly speaking, documents don't have namespaces, but all the elements in them do. If you want all documents with root elements in a particular namespace, you can query for /ns:*, but that won't resolve out of the indexes, since the element's "qualified names" (QName) is indexed, but the names

Re: [MarkLogic Dev General] Attribute-range-index for xml:lang

2011-03-07 Thread Damon Feldman
For the benefit of the archive, note that you can query for any attribute using cts:element-attribute-value-query() without creating an index. Adding a range index has benefits such as cts:element-attribute-values() support, but all XML structure is fully indexed by default, without schemas or

Re: [MarkLogic Dev General] Xquery unit testing

2011-03-08 Thread Damon Feldman
Consider xquery-unit: http://code.google.com/p/xquery-unit/updates/list which has methods to build an ad-hoc xquery function call by adding import statements etc. You can then test the result using XPath or any string based method. Damon From: general-boun...@developer.marklogic.com [mailto:ge

Re: [MarkLogic Dev General] Does MarkLogic have tail recursion?

2011-03-09 Thread Damon Feldman
No, there is no tail-recusion optimization right now. But you can pass the has-value flag into the function rather than use xdmp:set(): xquery version "1.0-ml"; declare function local:is-good-value($n) { $n lt 500 and $n mod 11 eq 0 }; declare function local:f-rec($n, $done) { let $val := if

Re: [MarkLogic Dev General] question about cts:query

2011-03-14 Thread Damon Feldman
Helen, Use: cts:element-query( xs:QName("ns:mytag"), cts:and-query(()) ) The empty sequence passed to an and-query evaluates like "true." Damon From: general-boun...@developer.marklogic.com [general-boun...@developer.marklogic.com] On Behalf Of Helen Chen [hc.

Re: [MarkLogic Dev General] Handling request parameter in both xcc and http connection

2011-03-18 Thread Damon Feldman
Ambika, Here's a trick, though it's cleaner to use something like XQuery unit that can actually test using HTTP requests. "declared" variables are lazily evaluated, and won't throw an error if they are missing, until they are actually accessed. So you can prevent the access of an external

Re: [MarkLogic Dev General] How to invoke shell script from marklogic

2011-03-21 Thread Damon Feldman
Manoj, I suggest using wget to send an HTTP request. The URI of the HTTP request should specify a main module that uses xdmp:get-request-field() to extract text parameters from the HTTP request. I believe the cqsh utility also allows xquery to be evaluated from a shell script. Yours, Damon _

Re: [MarkLogic Dev General] connecting to a unix server box from Marklogic

2011-03-21 Thread Damon Feldman
Prasanth, Not directly. You can make HTTP calls via xdmp:http-get() and related functions, so you would stand up some other server (such as some cgi server or java http server) that will return the "ls" command returning the results via HTTP. Damon From: gener

Re: [MarkLogic Dev General] Performance improvement

2011-04-01 Thread Damon Feldman
Judie, I assume $resultsLimit is 10? Can you tell us what the word queries in $searchTermsQ are (particularly any wildcards or options such as case-sensitive options), and query-meters() output from: let $results := cts:search(collection("x"), cts:or-query(($searchTermsQ)))[1 to 10] return xd

Re: [MarkLogic Dev General] Formatting issue

2011-04-07 Thread Damon Feldman
Prasanth, I think your problem is actually with the way your browser is displaying the response that CQ sends back. If the browser interprets your code as html or XML it will render it using special rules. CQ in Firefox typically does what you see - shows the text as HTML, where, by spec, white

Re: [MarkLogic Dev General] Getting closest matches

2011-04-07 Thread Damon Feldman
Jeni, That's a pretty interesting requirement. I played with it and find you can at least approximate that sort of thing by checking for all adjacent words. See the query below. I have both "fast phrase" and "fast element phrase" search indexes on and extra words do lower the score. My intuiti

Re: [MarkLogic Dev General] Node replace without changing its content

2011-04-10 Thread Damon Feldman
The simplest is to use xdmp:node-replace($name-node, {$name-node/text()}) Damon From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of ambika arumugam Sent: Saturday, April 09, 2011 2:53 AM To: General MarkLogic Developer Discussion Subject:

Re: [MarkLogic Dev General] delete files from marklogic DB which starts with same word

2011-04-12 Thread Damon Feldman
Ram, Often, if you have a set prefix on your URI that groups related documents together, you should use directories. In that case, the shared prefix must end with a slash: /articles/article1.xml /articles/article2.xml /books/book1.xml etc. then you can use directory functions such as xdmp:dire

Re: [MarkLogic Dev General] Loading xml files in mark logic server

2011-04-19 Thread Damon Feldman
Rajesh, Each module invoke such as yours below runs as a single transaction with all the data in memory. For thousands of XML documents, you should break the work up into smaller chunks. The InformationStudio flows available in version 4.2 will do this automatically, and also provide a nice GU

Re: [MarkLogic Dev General] GET + POST

2011-04-29 Thread Damon Feldman
Greg, I recommend you first inspect the POST to see if the receiver is not interpreting it properly. If you don't have wireshark or similar handy to see it on the wire, you can send the post to MarkLogic, with the following request echo service. Yours, Damon -- (: gather up the HTTP info and

Re: [MarkLogic Dev General] op:except?

2011-05-09 Thread Damon Feldman
Jakob, This query will figure out which documents are in/out of your desired result using the indexes: let $q := cts:and-not-query(cts:collection-query("c1"), cts:collection-query("c2")) you can then list the URIs only if you have a URI lexicon return cts:uris("", (), $q) or retrieve all the

  1   2   >