Thanks Mike as always for the detailed answer.
Alan
On Nov 20, 2008, at 8:02 PM, Michael Blakeley wrote:
Alan,
As best I understand it, XDMP-DEADLOCK means that the database
observed two or more queries trying to lock the same fragment in
concurrent transactions. At the Debug level, these messages are not
an error. But the resulting retries can slow down the contending
updates. Depending on the cause of the contention, it might indicate
an error in your ingestion logic.
The Developer Guide chapter "Understanding Transactions" discusses
these concepts. When considering RecordLoader ingestion, there are
several possible causes. The two most likely are:
If "maintain-directory-last-modified" is enabled for the database,
then concurrent updates to /a/b/1 and /a/b/2 will each try to lock /
a/ and /a/b/, causing lock contention. The simplest way to avoid
this is to disable maintain-directory-last-modified.
If two queries are both trying to lock /a/b/foo.xml, that will cause
lock contention. This may indicate a bug in your ingestion logic:
for example, an algorithm used to generate unique ids might generate
duplicate ids under some circumstances.
Debugging the latter can be tricky, especially since multiple
threads are involved. RecordLoader's ERROR_EXISTING option might
help, since it will throw an error if the document to be inserted
already exists in the database. However, this option cannot be
enforced if your RecordLoader configuration uses
XccModuleContentFactory or uses its own plug-in class. In those
cases, you must implement the ERROR_EXISTING check yourself (http://developer.marklogic.com/svn/recordloader/trunk/README.html
).
Another debugging technique is to set LOG_LEVEL=FINE, resulting in a
log message for every new document. You could then grep the log file
for output like this:
$ grep -A1 'AbstractLoader insert' simplelogger-0-0.log
Nov 20, 2008 2:22:24 PM com.marklogic.recordloader.AbstractLoader
insert
FINE: inserting 1.xml
--
Nov 20, 2008 2:22:24 PM com.marklogic.recordloader.AbstractLoader
insert
FINE: inserting 1.xml
Examining the output, I can see that I have a duplicate URI "1.xml".
That might happen if I am using the IGNORE_FILE_BASENAME option, and
two of my documents are called "1.xml", or might have other causes.
The exact resolution depends on the input content and the
RecordLoader configuration.
Note that this technique could be misleading if your RecordLoader
configuration uses XccModuleContentFactory, or if you are providing
your own LoaderInterface or ContentInterface plug-in class. In
either case RecordLoader will log its idea of the final document
URI, but that might not match the URI actually inserted by your
XQuery module or Java class.
-- Mike
Alan Darnell wrote:
Sometimes when we are bulk loading records, usually with record
loader, we experience every slow rates of "ingest" and see a flurry
of messages about DEADLOCKs in the ErrorLog file. Why does this
happen for some databases and some loads but not others. Has
anyone else experienced this?
2008-11-20 17:11:03.065 Debug: Retrying
XDBCConnectionTask::handleConnection 30 because XDMP-DEADLOCK:
Deadlock detected locking ebooks2 #2100171885223584270
2008-11-20 17:11:03.435 Debug: Retrying
XDBCConnectionTask::handleConnection 6 because XDMP-DEADLOCK:
Deadlock detected locking ebooks1 #2100171885223584270
2008-11-20 17:11:07.444 Debug: Retrying
XDBCConnectionTask::handleConnection 31 because XDMP-DEADLOCK:
Deadlock detected locking ebooks1 #2100171885223584270
2008-11-20 17:11:07.660 Debug: Retrying
XDBCConnectionTask::handleConnection 12 because XDMP-DEADLOCK:
Deadlock detected locking ebook3 #2100171885223584270
Thanks in advance,
Alan
Alan Darnell
University of Toronto
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
