[
https://issues.apache.org/jira/browse/MARMOTTA-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997395#comment-14997395
]
ASF GitHub Bot commented on MARMOTTA-620:
-----------------------------------------
GitHub user marktriggs opened a pull request:
https://github.com/apache/marmotta/pull/15
MARMOTTA-620 Reduce locking to improve concurrent write performance
Holding a shared lock while calling `connection.getTripleId()` makes
throughput quite sensitive to small variations in database latency.
Drop the lock prior to running this query, and only take it again when
necessary.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marktriggs/marmotta-1 MARMOTTA-620
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/marmotta/pull/15.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15
----
commit 7212e63e8613ed4e276117fab0bcd5271281408f
Author: Mark Triggs <[email protected]>
Date: 2015-11-08T03:42:53Z
MARMOTTA-620 Reduce locking to improve concurrent write performance
Holding a shared lock while calling `connection.getTripleId()` makes
throughput quite sensitive to small variations in database latency.
Drop the lock prior to running this query, and only take it again when
necessary.
----
> Lock contention with multiple writers hitting the LDP service
> -------------------------------------------------------------
>
> Key: MARMOTTA-620
> URL: https://issues.apache.org/jira/browse/MARMOTTA-620
> Project: Marmotta
> Issue Type: Improvement
> Components: KiWi Triple Store
> Affects Versions: 3.3.0
> Reporter: Mark Triggs
> Attachments: marmotta-coarse-locking-all-responses.png,
> marmotta-fine-locking-all-responses.png
>
>
> This issue references the following mailing list post:
> http://mail-archives.apache.org/mod_mbox/marmotta-users/201511.mbox/%3C87pozjwzp4.fsf%40dishevelled.net%3E
> -----
> I've been doing some performance tuning work with Tom Johnson and Mark
> Breedlove of the DPLA (who have both posted recently), and have noticed a
> possible improvement for performance when multiple writers are PUTing records
> concurrently.
> While running a benchmark where multiple workers PUT records via Marmotta's
> LDP interface, I noticed that all but one of the threads were waiting on a
> lock inside the JVM. The code in question was this `synchronized` block in
> KiWiValueFactory.createStatement:
> https://github.com/dpla/marmotta/blob/master/libraries/kiwi/kiwi-triplestore/
> src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java#L567
> This code takes a lock on the `registry` object (shared by all connections),
> and holds it while calling connection.getTripleId(). That query is generally
> pretty fast, but it's called frequently enough that the cost of the locking
> can still be quite high.
> I ran a test where 5 concurrent writers each PUT random records via
> Marmotta's LDP interface. I ran 6 test rounds, with each round adding an
> extra millisecond delay to the getTripleId() call (simulating between 0 and 5
> milliseconds of network/database latency). As that query gets slower, the
> effect of the Java locking becomes more dramatic, as you can see from the
> response times here:
> http://dishevelled.net/marmotta-coarse-locking-all-responses.png
> With no added delay (and Postgres running on the same machine as Marmotta),
> most PUT requests complete within about 500ms. With 5 milliseconds of delay,
> the same PUT requests take between 2.5 and 3 seconds.
> I reworked the code to reduce the time that lock was held, and reran my test.
> You can see the response times are much better, even with 5ms of latency:
> http://dishevelled.net/marmotta-fine-locking-all-responses.png
> I've got a patch for this that I'm happy to send along. I drop the lock prior
> to running the getTripleId() query (allowing multiple connections to run the
> query at once), then only take it again if I need to write a new triple ID to
> the registry:
> https://github.com/marktriggs/marmotta/blob/mst-perf/libraries/kiwi/kiwi-triplestore/src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java#L578
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
