[
https://issues.apache.org/jira/browse/JENA-703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14215641#comment-14215641
]
Bala Kolla commented on JENA-703:
---------------------------------
I think have found a way to do this by checking the size of
commitedAwaitingFlush list and stop accepting the transaction (in begin method)
until it can flush. This seems to be working and was able to stop the CPU drain
and was able to support up to 500 users. Without these changes I would be
hitting the CPU spike at 350 user load and system never recovered until I
stopped the load.
One drawback with this approach is if there is a really long running query (ex:
filter system property like > or < modified date which matches many triples)
then we may unnecessarily drag on all other queries. May be we can start
checking this at lower threshold (ex:10 committed but not yet flushed) and try
to stop queries for a limited amount of time (ex: 2 seconds) and if that
pending queries doesn't free up then let it go until the next threshold (ex:
20) and if it hits the second threshold we will wait until all the pending
transactions are done.
Anyhow in my workload I don't have really complex queries and hence is working
fine. Would have to test adding a long running query and see how it handles it.
Please let me know your thoughts about this approach.
> TDB - Ensure readers don't block writeback forever.
> ---------------------------------------------------
>
> Key: JENA-703
> URL: https://issues.apache.org/jira/browse/JENA-703
> Project: Apache Jena
> Issue Type: Improvement
> Components: TDB
> Affects Versions: Jena 2.11.1
> Reporter: Andy Seaborne
>
> The commit queue can only be flushed when the main storage is quiet. A
> stream of readers can lead to the main storage never being quiet.
> One possibility is to place a maximum (quite high) on the number of readers
> allowed after the decision to flush the journal to main storage. If exceeded,
> then lock readers for a while, flush and resume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
