I don't see anything wrong with creating directories manually, there is even an api for it:
http://docs.marklogic.com/xdmp:directory-create But it seems like it might be a pretty big burden on the application to do that, that is the only reason I was suggesting making that automatic and seeing how cheap or expensive that is for your app (and if you are creating the dirs anyway, how different would that actually be than having MarkLogic create them for you). One other thing to note: newer versions of MarkLogic have a lot of performance improvements around large updates and deletes, so if you are on an older version of MarkLogic, upgrades can be good. -Danny ________________________________ From: [email protected] [[email protected]] on behalf of Michael Sokolov [[email protected]] Sent: Thursday, May 22, 2014 7:18 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] best practices for manual directory creation Thanks for the suggestion, Danny; it seems sensible. At this stage I don't want to modify the rest of the system, which is pretty mature and relies on the system-maintained last-modified property. In fact we already maintain a separate modified timestamp in the documents with different semantics (eg you can copy a document without updating its timestamp), but this can't be used for tracking changes to binary documents. So I think we are stuck with the built-in maintain-last-modified. I did briefly try having directory creation=automatic + maintain directory last modified=false and maintain-last-modified=true, but I thought it looked as if things were slowing down again during a large document import. I didn't measure carefully or continue this experiment for long though because I think I have a solution to the manual directory creation that is effective. All our document updates go through a single java API, so I can track updated uris there and manage directory insertion in batches as a separate process. And it seems that the trick of setting the <directory/> property tickles the modified-time. I suppose if that became unsupported it could cause problems, but I think I can live with that risk. Coming back around to my initial question though -- it seems like the consensus here is that best practice is *not* to create directories manually? -Mike On 5/22/2014 12:05 PM, Danny Sokolsky wrote: I think if you want to maintain these yourself, you should not use the system maintained properties; instead, make up some of your own that do the equivalent things. That being said, have you tried leaving directory creation at automatic, but turning off maintain last modified and maintain directory last modified? Depending upon how deep your directory hierarchy is, this might not cause too much overhead. I would recommend trying that, and then just add a dateTime property (or element in the document if you prefer, allowing you to not have to create a property fragment) to track whatever you want about the last modified (based on your app requirements). I think that might work well, especially if your hierarchy does not not have millions of directories. See how it works and let us know. -Danny From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Keith L. Breinholt Sent: Thursday, May 22, 2014 8:23 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] best practices for manual directory creation <prop:last-modified> is not a property that you can manually set. I believe that is a security issue. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Mike Sokolov Sent: Thursday, May 22, 2014 8:26 AM To: MarkLogic General ML Subject: Re: [MarkLogic Dev General] best practices for manual directory creation I'm getting good results updating the directory timestamps using: xdmp:document-set-properties ($dir-uri, <prop:directory/>) and this seems to limit the number of prop:directory properties to 2 too -Mike On 05/22/2014 10:03 AM, Mike Sokolov wrote: I'm working with a system that requires directories and directory-modified timestamps (for a webDAV-like browsing feature), but have found that automatic directory creation introduces unacceptable lock contention during bulk updates, so I am looking into managing the directory creation and timestamp updates manually. I have one question, and one strange observation - maybe a bug. I'm working with 7.0-2.3. First the question: how should I update the prop:last-modified property? Updating it explicitly raises an error: XDMP-ARG: xdmp:document-set-property("/books/", <prop:last-modified xmlns:prop="http://marklogic.com/xdmp/property";<https://urldefense.proofpoint.com/v1/url?u=http://marklogic.com/xdmp/property&k=wlPCrglRP6kzT4RbABWMaw%3D%3D%0A&r=2FOxwjXkcRFP9Zb5gsGqutGbMyYaH6V5O1y2qyDOE%2Bw%3D%0A&m=tMQwNzleMcPFHrHVywsz7LShGCB7BV0fr4nwOoRO9yE%3D%0A&s=b129b0afdc261f273e05e156d512fac53273fae49be30e4e6423deb66508ad09>>2014-05-22T15:53:46.724003+02:00</prop:last-modified>) -- Invalid argument even though I have "maintain directory last modified" set to false (and directory creation = manual). I do have maintain last modified set to true, so I expect that is happening automatically on directory creation - OK, but in that instance how would I update the directory modified time when inserting or deleting documents in the directory? I tried adding a dummy property using xdmp:set-property, and that does seem to update the timestamp, but I don't really want to do that if I don't have to, of course. Perhaps I could delete and then recreate the directory properties document, but that doesn't seem great either. Any other ideas? Now the weird observation. It seems that every time I modify the directory properties document, it gets another <prop:directory /> property node! Currently I have: <prop:properties xmlns:prop="http://marklogic.com/xdmp/property";<https://urldefense.proofpoint.com/v1/url?u=http://marklogic.com/xdmp/property&k=wlPCrglRP6kzT4RbABWMaw%3D%3D%0A&r=2FOxwjXkcRFP9Zb5gsGqutGbMyYaH6V5O1y2qyDOE%2Bw%3D%0A&m=tMQwNzleMcPFHrHVywsz7LShGCB7BV0fr4nwOoRO9yE%3D%0A&s=b129b0afdc261f273e05e156d512fac53273fae49be30e4e6423deb66508ad09>> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:directory/> <prop:last-modified>2014-05-22T15:47:37+02:00</prop:last-modified> </prop:properties> I thought that properties documents maintained a map with unique keys? -Mike _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general<https://urldefense.proofpoint.com/v1/url?u=http://developer.marklogic.com/mailman/listinfo/general&k=wlPCrglRP6kzT4RbABWMaw%3D%3D%0A&r=2FOxwjXkcRFP9Zb5gsGqutGbMyYaH6V5O1y2qyDOE%2Bw%3D%0A&m=tMQwNzleMcPFHrHVywsz7LShGCB7BV0fr4nwOoRO9yE%3D%0A&s=c9cb9dbd161260f93e52fe3901e1bb716460a6fcc74f86cb436db69aa2cd554c> NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. _______________________________________________ General mailing list [email protected]<mailto:[email protected]> http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
