Alan,
As best I understand it, XDMP-DEADLOCK means that the database observed
two or more queries trying to lock the same fragment in concurrent
transactions. At the Debug level, these messages are not an error. But
the resulting retries can slow down the contending updates. Depending on
the cause of the contention, it might indicate an error in your
ingestion logic.
The Developer Guide chapter "Understanding Transactions" discusses these
concepts. When considering RecordLoader ingestion, there are several
possible causes. The two most likely are:
If "maintain-directory-last-modified" is enabled for the database, then
concurrent updates to /a/b/1 and /a/b/2 will each try to lock /a/ and
/a/b/, causing lock contention. The simplest way to avoid this is to
disable maintain-directory-last-modified.
If two queries are both trying to lock /a/b/foo.xml, that will cause
lock contention. This may indicate a bug in your ingestion logic: for
example, an algorithm used to generate unique ids might generate
duplicate ids under some circumstances.
Debugging the latter can be tricky, especially since multiple threads
are involved. RecordLoader's ERROR_EXISTING option might help, since it
will throw an error if the document to be inserted already exists in the
database. However, this option cannot be enforced if your RecordLoader
configuration uses XccModuleContentFactory or uses its own plug-in
class. In those cases, you must implement the ERROR_EXISTING check
yourself
(http://developer.marklogic.com/svn/recordloader/trunk/README.html).
Another debugging technique is to set LOG_LEVEL=FINE, resulting in a log
message for every new document. You could then grep the log file for
output like this:
$ grep -A1 'AbstractLoader insert' simplelogger-0-0.log
Nov 20, 2008 2:22:24 PM com.marklogic.recordloader.AbstractLoader insert
FINE: inserting 1.xml
--
Nov 20, 2008 2:22:24 PM com.marklogic.recordloader.AbstractLoader insert
FINE: inserting 1.xml
Examining the output, I can see that I have a duplicate URI "1.xml".
That might happen if I am using the IGNORE_FILE_BASENAME option, and two
of my documents are called "1.xml", or might have other causes. The
exact resolution depends on the input content and the RecordLoader
configuration.
Note that this technique could be misleading if your RecordLoader
configuration uses XccModuleContentFactory, or if you are providing your
own LoaderInterface or ContentInterface plug-in class. In either case
RecordLoader will log its idea of the final document URI, but that might
not match the URI actually inserted by your XQuery module or Java class.
-- Mike
Alan Darnell wrote:
Sometimes when we are bulk loading records, usually with record
loader, we experience every slow rates of "ingest" and see a flurry of
messages about DEADLOCKs in the ErrorLog file. Why does this happen
for some databases and some loads but not others. Has anyone else
experienced this?
2008-11-20 17:11:03.065 Debug: Retrying
XDBCConnectionTask::handleConnection 30 because XDMP-DEADLOCK:
Deadlock detected locking ebooks2 #2100171885223584270
2008-11-20 17:11:03.435 Debug: Retrying
XDBCConnectionTask::handleConnection 6 because XDMP-DEADLOCK: Deadlock
detected locking ebooks1 #2100171885223584270
2008-11-20 17:11:07.444 Debug: Retrying
XDBCConnectionTask::handleConnection 31 because XDMP-DEADLOCK:
Deadlock detected locking ebooks1 #2100171885223584270
2008-11-20 17:11:07.660 Debug: Retrying
XDBCConnectionTask::handleConnection 12 because XDMP-DEADLOCK:
Deadlock detected locking ebook3 #2100171885223584270
Thanks in advance,
Alan
Alan Darnell
University of Toronto
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
