This leads me to suspect that this has to do with the authentication challenge.
I suspect you'd have been as successful simply recreating the stream and
trying again, or performing a trivial request before beginning the loop.
The 4.1-8 client also supports pre-emptive basic authentication, which behaves
more like XCC did before digest auth was added. It might be worth trying that
to see if it fixes the problem, though I don't recommend using basic auth for
production use if you have a choice.
I'm going to log a bug based on my guess as to what's happening.
Sam Neth
Lead Engineer
MarkLogic Corporation
On Nov 17, 2010, at 7:15 PM, Michael Sokolov wrote:
I've also seen that non-rewindable exception occur in a totally unexpected
circumstance. It was a while ago, but this sounds very familiar. In my case I
*know* nobody had processed the stream yet. Basically I was looping over a
bunch of documents and streaming them into XCC. As a workaround, I wrapped an
exception handler, and when I caught this particular error, I loaded the whole
document into a buffer and wrote it non-streaming. The interesting thing was
that once I had done this, subsequent document inserts were able to complete in
streaming mode. It was only the first one that caused the exception. You
can't really tell this from the code here, but I did observe that when setting
breakpoints in the debugger, I only got the exception once per loop. Here's a
code snippet that reproduced the problem for me:
RequestOptions nocache = fromSession.getEffectiveRequestOptions();
nocache.setCacheResult (false);
for (String uri : uris) {
... uninteresting stuff omitted ...
try {
r = fromSession.newAdhocQuery ("doc(" + quri + ")", cache);
results = fromSession.submitRequest (r);
ResultItem doc = results.next();
istream = doc.asInputStream();
// streaming
com.marklogic.xcc.Content newdoc = ContentFactory.newUnBuff\
eredContent(uri, istream, opts);
try {
toSession.insertContent (newdoc);
} catch (ContentInsertException e) {
// It always seems to complain about the first document
being non-rewindable?
results.close();
istream.close();
r = fromSession.newAdhocQuery ("doc(" + quri + ")",
nocache);
results = fromSession.submitRequest (r);
doc = results.next();
istream = doc.asInputStream();
ByteArrayOutputStream buf = readStreamToBuffer(istream);
newdoc = ContentFactory.newContent (uri,
buf.toByteArray(), opts);
toSession.insertContent (newdoc);
}
}
On 11/17/2010 1:19 PM, Sam Neth wrote:
This exception indicates that a streaming insert has failed. Because streams
can't be rewound, the insert operation can't be retried automatically by XCC
when a retryable exception occurs. Unfortunately the underlying exception that
causes the failure appears to be mishandled and consequently not displayed, but
you may be able to find some indication in the server log. I'm not sure if
you're running into a bug in RecordLoader, XCC, or both, but I wouldn't expect
to see this mode of failure for a properly implemented tool built on XCC.
I'm going to log a bug on the exception reporting issue, which if endemic to
XCC generally should be fixed.
Sam Neth
Lead Engineer
MarkLogic Corporation
On Nov 17, 2010, at 9:25 AM, David Scott Gurney wrote:
In attempting to use the autoloader the following exception occurs:
com.marklogic.xcc.exceptions.ContentInsertException: Cannot auto-restart
insert, non-rewindable content already processed: /content/drop/one_event/1.xml
[Session: user=admin, cb={default} [ContentSource: user=admin, cb={none}
[provider: address=vm/192.168.116.135:8438, pool=0/64]]]
[Client: XCC/4.1-8]
at
com.marklogic.xcc.impl.handlers.ContentInsertController.assertRestartable(ContentInsertController.java:287)
at
com.marklogic.xcc.impl.handlers.ContentInsertController.serverDialog(ContentInsertController.java:84)
at
com.marklogic.xcc.impl.handlers.AbstractRequestController.runRequest(AbstractRequestController.java:78)
at com.marklogic.xcc.impl.SessionImpl.insertContent(SessionImpl.java:198)
at com.marklogic.xcc.impl.SessionImpl.insertContent(SessionImpl.java:226)
at com.marklogic.recordloader.xcc.XccContent.insert(XccContent.java:48)
at com.marklogic.recordloader.AbstractLoader.insert(AbstractLoader.java:221)
at com.marklogic.recordloader.Loader.processStartElement(Loader.java:279)
at com.marklogic.recordloader.Loader.processRecords(Loader.java:145)
at com.marklogic.recordloader.Loader.process(Loader.java:91)
at com.marklogic.recordloader.AbstractLoader.call(AbstractLoader.java:68)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
The error seems to indicate that there are duplicate records and that this one
has already been processed. However, the database is clear and there is only 1
record, so I'm not understanding why it could have "already processed". Can
someone explain what this error means?
I'm using the latest jars from the autoloader url
(http://developer.marklogic.com/code/AutoLoader)
I'm running MarkLogic 4.1-7.1 on Ubuntu 10.04
Here is the script file that is being used:
-------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
java -cp
lib/autoloader-1.0.1.jar:lib/xcc.jar:lib/xpp3-1.1.4.jar:lib/RecordLoaderMod.jar
com/marklogic.autoloader.AutoLoader /Users/gurneyds/recordLoader/drop
/Users/gurneyds/recordLoader/recordLoader-drop autoLoaderConfig.properties
-------------------------------------------------------------------------------------------------------------------------------------
And here is the contents of the record loader properties file:
-------------------------------------------------------------------------------------------------------------------------------------
THREADS=1
CONNECTION_STRING=xcc://admin:ad...@vm:8438/
INPUT_PATH=/Users/gurneyds/recordLoader/recordLoader-drop
ID_NAME=#AUTO
URI_PREFIX=/content/drop/
URI_SUFFIX=.xml
-------------------------------------------------------------------------------------------------------------------------------------
The xml file is dropped into the "drop" folder. After a few moments the
autoloader code is run and the exception is thrown.
Using record loader by itself works just fine. Here is the script file to run
just the record loader:
-------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash
java -cp lib/recordloader.jar:lib/xcc.jar:lib/xpp3-1.1.4.jar
com.marklogic.ps.RecordLoader recordLoaderConfig.properties
-------------------------------------------------------------------------------------------------------------------------------------
And here is the contents of the record loader properties file:
-------------------------------------------------------------------------------------------------------------------------------------
THREADS=1
CONNECTION_STRING=xcc://admin:ad...@vm:8438/
INPUT_PATH=/Users/gurneyds/recordLoader/drop
ID_NAME=#AUTO
URI_PREFIX=/content/drop/
URI_SUFFIX=.xml
-------------------------------------------------------------------------------------------------------------------------------------
Any ideas?
Thanks
David
NOTICE: This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized review,
use, disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all copies of
the original message.
_______________________________________________
General mailing list
[email protected]<mailto:[email protected]>
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]<mailto:[email protected]>
http://developer.marklogic.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://developer.marklogic.com/mailman/listinfo/general
