the Data Movement SDK is also available via the Early Access program or via github: https://github.com/marklogic/java-client-api/tree/4.0.0-EA4
Sam Mefford Senior Engineer MarkLogic Corporation [email protected] Cell: +1 801 706 9731 www.marklogic.com<http://www.marklogic.com> This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. ________________________________ From: [email protected] [[email protected]] on behalf of Dave Cassel [[email protected]] Sent: Thursday, January 12, 2017 9:09 AM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] How to get the number of records ingested by a MLCP hit An additional thought for the benefit of the list archive: in MarkLogic 9, you should be able to get this functionality through the Data Movement SDK (currently still in development). -- Dave Cassel<http://davidcassel.net>, @dmcassel<https://twitter.com/dmcassel> Technical Community Manager MarkLogic Corporation<http://www.marklogic.com/> http://developer.marklogic.com/ From: <[email protected]<mailto:[email protected]>> on behalf of David Lee <[email protected]<mailto:[email protected]>> Reply-To: MarkLogic Developer Discussion <[email protected]<mailto:[email protected]>> Date: Thursday, January 12, 2017 at 10:41 AM To: MarkLogic Developer Discussion <[email protected]<mailto:[email protected]>> Subject: Re: [MarkLogic Dev General] How to get the number of records ingested by a MLCP hit Depending on your use case and scenarios, mlcp is an application layer built on XCC and HDFS libraries, as well as application logic. If you like the feature set of mlcp and just want to call it from java then In would recommend as did Sam to call it as a sub process (Runtime.exec()). I do *not* recommend calling it in the same JVM as your java code -- there is no significant performance or feature gains and there is risk of 'contamination' of the global memory space between your app and MLCP. If you follow the 'best practice' for running sub-processes in Java, MLCP will work well in that mode. In particular, pay attention to making sure that the input and output streams of the process don’t stall or deadlock - this is well documented in the Oracle Java API's for sub process. ( not particular to MLCP). You will need to read the stdout of mlcp concurrently with writing any input provided to it or redirect the subproject's input, output, error streams to a file. This will give you the full feature set of mlcp. If you are looking for a java library intended to be embedded into a Java app, The suggested Java based library for embedded use is the Java SDK which is based on the REST framework. It is different than MLCP but at about the same abstraction level, and is a library intended for embedded use. If you are looking for the same underlying protocol that mlcp uses (but minus the application level features and HDFS support), MLCP uses the XCC library as the transport layer for direct access to ML. XCC is a library intended for imbedded use. It’s a low-level library -- so it doesn’t have the high level features that MLCP has, nor is it as easy to use or debug but it is what MLCP uses for direct ML communication, is publicly downloadable and documented. I would have recommended it as the it only if the Java API does not do what you need. In that case We would appreciate any input into what issues you have as it is designed to be much more usable at the 'application layer' in a Java app then XCC is, supporting directly high level features like complex search, data mapping etc. and in general is a good abstraction layer for middle tier Java Applications, analogous to say Hibernate -- whereas XCC is more analogous to JDBC. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Sam Mefford Sent: Wednesday, January 11, 2017 2:33 PM To: MarkLogic Developer Discussion <[email protected]<mailto:[email protected]>> Subject: Re: [MarkLogic Dev General] How to get the number of records ingested by a MLCP hit I'm sure you could find a way. But using mlcp in a Java application is not a supported usage. mlcp is designed to run at the command-line. Sam Mefford Senior Engineer MarkLogic Corporation [email protected]<mailto:[email protected]> Cell: +1 801 706 9731 www.marklogic.com<http://www.marklogic.com> This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of this e-mail communication by others is strictly prohibited. If you are not the intended recipient, please notify us immediately by returning this message to the sender and delete all copies. Thank you for your cooperation. ________________________________ From:[email protected]<mailto:[email protected]> [[email protected]<mailto:[email protected]>] on behalf of Yogesh Kumar [[email protected]<mailto:[email protected]>] Sent: Wednesday, January 11, 2017 4:10 AM To: [email protected]<mailto:[email protected]> Subject: Re: [MarkLogic Dev General] How to get the number of records ingested by a MLCP hit Gentle reminder!! On 09-Jan-2017 3:44 PM, "Yogesh Kumar" <[email protected]<mailto:[email protected]>> wrote: Gentle reminder!! On 07-Jan-2017 10:13 PM, "Yogesh Kumar" <[email protected]<mailto:[email protected]>> wrote: Hi Team, I am using MLCP in my java code to ingest the data into Mark Logic. How can I get the following details in my java application OUTPUT_RECORDS OUTPUT_RECORDS_COMMITTED OUTPUT_RECORDS_FAILED Thanks, Yogesh
_______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
