[
https://issues.apache.org/jira/browse/MARMOTTA-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997397#comment-14997397
]
Mark Triggs commented on MARMOTTA-620:
--------------------------------------
Pull request sent: https://github.com/apache/marmotta/pull/15
Thanks!
Mark
> Lock contention with multiple writers hitting the LDP service
> -------------------------------------------------------------
>
> Key: MARMOTTA-620
> URL: https://issues.apache.org/jira/browse/MARMOTTA-620
> Project: Marmotta
> Issue Type: Improvement
> Components: KiWi Triple Store
> Affects Versions: 3.3.0
> Reporter: Mark Triggs
> Attachments: marmotta-coarse-locking-all-responses.png,
> marmotta-fine-locking-all-responses.png
>
>
> This issue references the following mailing list post:
> http://mail-archives.apache.org/mod_mbox/marmotta-users/201511.mbox/%3C87pozjwzp4.fsf%40dishevelled.net%3E
> -----
> I've been doing some performance tuning work with Tom Johnson and Mark
> Breedlove of the DPLA (who have both posted recently), and have noticed a
> possible improvement for performance when multiple writers are PUTing records
> concurrently.
> While running a benchmark where multiple workers PUT records via Marmotta's
> LDP interface, I noticed that all but one of the threads were waiting on a
> lock inside the JVM. The code in question was this `synchronized` block in
> KiWiValueFactory.createStatement:
> https://github.com/dpla/marmotta/blob/master/libraries/kiwi/kiwi-triplestore/
> src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java#L567
> This code takes a lock on the `registry` object (shared by all connections),
> and holds it while calling connection.getTripleId(). That query is generally
> pretty fast, but it's called frequently enough that the cost of the locking
> can still be quite high.
> I ran a test where 5 concurrent writers each PUT random records via
> Marmotta's LDP interface. I ran 6 test rounds, with each round adding an
> extra millisecond delay to the getTripleId() call (simulating between 0 and 5
> milliseconds of network/database latency). As that query gets slower, the
> effect of the Java locking becomes more dramatic, as you can see from the
> response times here:
> http://dishevelled.net/marmotta-coarse-locking-all-responses.png
> With no added delay (and Postgres running on the same machine as Marmotta),
> most PUT requests complete within about 500ms. With 5 milliseconds of delay,
> the same PUT requests take between 2.5 and 3 seconds.
> I reworked the code to reduce the time that lock was held, and reran my test.
> You can see the response times are much better, even with 5ms of latency:
> http://dishevelled.net/marmotta-fine-locking-all-responses.png
> I've got a patch for this that I'm happy to send along. I drop the lock prior
> to running the getTripleId() query (allowing multiple connections to run the
> query at once), then only take it again if I need to write a new triple ID to
> the registry:
> https://github.com/marktriggs/marmotta/blob/mst-perf/libraries/kiwi/kiwi-triplestore/src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java#L578
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
