[
https://issues.apache.org/jira/browse/MARMOTTA-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997578#comment-14997578
]
ASF GitHub Bot commented on MARMOTTA-620:
-----------------------------------------
Github user wastl commented on a diff in the pull request:
https://github.com/apache/marmotta/pull/15#discussion_r44345877
--- Diff:
libraries/kiwi/kiwi-triplestore/src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java
---
@@ -575,13 +577,29 @@ public Statement createStatement(Resource subject,
URI predicate, Value object,
registry.registerKey(cacheKey,
connection.getTransactionId(), result.getId());
} else {
// not found in registry, try loading from database
-
result.setId(connection.getTripleId(ksubject,kpredicate,kobject,kcontext));
+ needsDBLookup = true;
}
+ }
+
+ if(needsDBLookup) {
+
result.setId(connection.getTripleId(ksubject,kpredicate,kobject,kcontext));
+ }
- // triple has no id from registry or database, so we
create one and flag it for reasoning
- if(result.getId() < 0) {
- result.setId(connection.getNextSequence());
- result.setNewTriple(true);
+ // triple has no id from registry or database, so we create
one and flag it for reasoning
+ if(result.getId() < 0) {
+ synchronized (registry) {
+ // It's possible a concurrent thread might have
created this
+ // triple while we were blocked. Check the registry
again.
+ long tripleId = registry.lookupKey(cacheKey);
+
+ if(tripleId >= 0) {
+ // A concurrent thread got in first. Take the one
it created.
+ result.setId(tripleId);
--- End diff --
I guess in this case the call to registerKey below is not needed (it came
from the registry).
> Lock contention with multiple writers hitting the LDP service
> -------------------------------------------------------------
>
> Key: MARMOTTA-620
> URL: https://issues.apache.org/jira/browse/MARMOTTA-620
> Project: Marmotta
> Issue Type: Improvement
> Components: KiWi Triple Store
> Affects Versions: 3.3.0
> Reporter: Mark Triggs
> Attachments: marmotta-coarse-locking-all-responses.png,
> marmotta-fine-locking-all-responses.png
>
>
> This issue references the following mailing list post:
> http://mail-archives.apache.org/mod_mbox/marmotta-users/201511.mbox/%3C87pozjwzp4.fsf%40dishevelled.net%3E
> -----
> I've been doing some performance tuning work with Tom Johnson and Mark
> Breedlove of the DPLA (who have both posted recently), and have noticed a
> possible improvement for performance when multiple writers are PUTing records
> concurrently.
> While running a benchmark where multiple workers PUT records via Marmotta's
> LDP interface, I noticed that all but one of the threads were waiting on a
> lock inside the JVM. The code in question was this `synchronized` block in
> KiWiValueFactory.createStatement:
> https://github.com/dpla/marmotta/blob/master/libraries/kiwi/kiwi-triplestore/
> src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java#L567
> This code takes a lock on the `registry` object (shared by all connections),
> and holds it while calling connection.getTripleId(). That query is generally
> pretty fast, but it's called frequently enough that the cost of the locking
> can still be quite high.
> I ran a test where 5 concurrent writers each PUT random records via
> Marmotta's LDP interface. I ran 6 test rounds, with each round adding an
> extra millisecond delay to the getTripleId() call (simulating between 0 and 5
> milliseconds of network/database latency). As that query gets slower, the
> effect of the Java locking becomes more dramatic, as you can see from the
> response times here:
> http://dishevelled.net/marmotta-coarse-locking-all-responses.png
> With no added delay (and Postgres running on the same machine as Marmotta),
> most PUT requests complete within about 500ms. With 5 milliseconds of delay,
> the same PUT requests take between 2.5 and 3 seconds.
> I reworked the code to reduce the time that lock was held, and reran my test.
> You can see the response times are much better, even with 5ms of latency:
> http://dishevelled.net/marmotta-fine-locking-all-responses.png
> I've got a patch for this that I'm happy to send along. I drop the lock prior
> to running the getTripleId() query (allowing multiple connections to run the
> query at once), then only take it again if I need to write a new triple ID to
> the registry:
> https://github.com/marktriggs/marmotta/blob/mst-perf/libraries/kiwi/kiwi-triplestore/src/main/java/org/apache/marmotta/kiwi/sail/KiWiValueFactory.java#L578
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
