Thanks Aries, setting -thread_count to 1 got me to 100% in twenty minutes. BTW, I had to also increase the Java heap space; I now have this:
jfix:test-export jakob$ ls -l total 13955264 -rw-r--r-- 1 jakob staff 55948611 Jul 23 23:34 20140723233226+0200-000000-XML.zip -rw-r--r-- 1 jakob staff 1333218654 Jul 23 23:34 20140723233226+0200-000001-BINARY.zip -rw-r--r-- 1 jakob staff 52766896 Jul 23 23:38 20140723233637+0200-000002-XML.zip -rw-r--r-- 1 jakob staff 1340619106 Jul 23 23:38 20140723233637+0200-000003-BINARY.zip -rw-r--r-- 1 jakob staff 53143727 Jul 23 23:43 20140723234053+0200-000004-XML.zip -rw-r--r-- 1 jakob staff 1429323125 Jul 23 23:43 20140723234053+0200-000005-BINARY.zip -rw-r--r-- 1 jakob staff 53163017 Jul 23 23:48 20140723234531+0200-000006-XML.zip -rw-r--r-- 1 jakob staff 1361493351 Jul 23 23:48 20140723234531+0200-000007-BINARY.zip -rw-r--r-- 1 jakob staff 44951465 Jul 23 23:53 20140723235027+0200-000008-XML.zip -rw-r--r-- 1 jakob staff 1420450001 Jul 23 23:53 20140723235027+0200-000009-BINARY.zip which seems nice. however, just before finishing the process, mlcp spat out a lot of gibberish, some of which you can find here (potentially interesting stuff does not appear before line 765): https://gist.github.com/jfix/5139fd9238965553ccff so my hopefully last question on this topic is: how can I be sure (and ideally check) that these generated archives are actually valid (a part from attempting to load them back into MarkLogic)? would be nice if there were some kind of checksum or such. cheers, Jakob. On Wed, Jul 23, 2014 at 11:09 PM, Aries Li <[email protected]> wrote: > Hi Jakob, > > I assume you are running mlcp and ML server on the same machine. Ideally, > client and server should be on different machines. Is any resource maxed > out during the export? > > If that’s the case or suspicion, you can try -thread_count 1. Then mlcp > will be less aggressive when doing the export. > > > > Aries > > > > *From:* [email protected] [mailto: > [email protected]] *On Behalf Of *Jakob Fix > *Sent:* Wednesday, July 23, 2014 1:56 PM > *To:* MarkLogic Developer Discussion > *Subject:* Re: [MarkLogic Dev General] mlcp export problem/question > > > > Hi Aries, > > as suggested I increased the timeout to 30. the process still doesn't > finish, I get until 19% (see > https://gist.github.com/jfix/6bb7fe80cab7e3ef78e6), but before and after > there are similar and different errors. in addition to the "server > connection lost?" suspicion, MarkLogic now offers a new error on line 124 > (and later) "java.lang.RuntimeException: Could not buffer value as string" > > of course, I could probably further increase the keepalive timeout, but > that seems like a band aid. so, unfortunately, I'm not much further ahead > and it's slightly frustrating because on paper mlcp is supposed to replace > xqsync, but it's hard to get it to do basic stuff (because I suppose > exporting a database is quite a standard task). > > > > I welcome any help to get this done. > > cheers, > Jakob. > > On Wed, Jul 23, 2014 at 1:18 AM, Aries Li <[email protected]> wrote: > > Maybe the server is too busy to response in time. I would try raise the > keepalive timeout from 5 to 30 on the xdbc server. Please let us know if > that helps. > > > > Aries > > *From:* [email protected] [mailto: > [email protected]] *On Behalf Of *Jakob Fix > *Sent:* Tuesday, July 22, 2014 4:02 PM > *To:* General Mark Logic Developer Discussion > *Subject:* [MarkLogic Dev General] mlcp export problem/question > > > > Hi, > > I'm trying to export a database using mlcp. I'm not having much success. > > Here is a gist of the error output after 1009 seconds running (that's what > mlcp says): > > https://gist.github.com/jfix/2ef60350f8af9a4c2f33 > > mlcp wonders whether "Server connection lost?" but it's running as far as > I can make out. Are there any special precautions to take when exporting > (i.e. disabling potentially running tasks on the Taskserver, ...)? > > > the database is about 3 GB big (according to the Size indication in the > status page: 3,045 MB). > > I'm running this on a Macbook Pro with 8Gb of RAM, latest MarkLogic > (7.0-3) and latest mlcp (Hadoop2-1.2-2). > > cheers, > Jakob. > > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > > > > _______________________________________________ > General mailing list > [email protected] > http://developer.marklogic.com/mailman/listinfo/general > >
_______________________________________________ General mailing list [email protected] http://developer.marklogic.com/mailman/listinfo/general
