[jira] [Commented] (JENA-703) TDB - Ensure readers don't block writeback forever.
[ https://issues.apache.org/jira/browse/JENA-703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437354#comment-16437354 ] Andy Seaborne commented on JENA-703: This situation does not occur in in TDB2 because write transactions commit as they exit and the write updates the main database. While this is a more expensive scheme, it doesn't have the same issue as this ticket. [~kbkreddy] - is there code from your experiment in 2014? > TDB - Ensure readers don't block writeback forever. > --- > > Key: JENA-703 > URL: https://issues.apache.org/jira/browse/JENA-703 > Project: Apache Jena > Issue Type: Improvement > Components: TDB >Affects Versions: Jena 2.11.1 >Reporter: Andy Seaborne >Priority: Major > > The commit queue can only be flushed when the main storage is quiet. A > stream of readers can lead to the main storage never being quiet. > One possibility is to place a maximum (quite high) on the number of readers > allowed after the decision to flush the journal to main storage. If exceeded, > then lock readers for a while, flush and resume. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (JENA-703) TDB - Ensure readers don't block writeback forever.
[ https://issues.apache.org/jira/browse/JENA-703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221009#comment-14221009 ] Bala Kolla commented on JENA-703: - This experiment is yielding very good results and it seems like the committed but not yet flushed transactions could have significant impact on CPU. We will take a look at our code to batch up the writes as much we could but we can't eliminate the possibility of high volume of reads at the higher user load and essentially blocking the writes forever. These pending transactions would add additional load on CPU and hence make the queries to even take longer and hence increase the likely hood of even more pending queries. This explains the behavior we mentioned in Jena 801 and once the system reaches this stage of 100% CPU it never recovers until we stop the use load. TDB - Ensure readers don't block writeback forever. --- Key: JENA-703 URL: https://issues.apache.org/jira/browse/JENA-703 Project: Apache Jena Issue Type: Improvement Components: TDB Affects Versions: Jena 2.11.1 Reporter: Andy Seaborne The commit queue can only be flushed when the main storage is quiet. A stream of readers can lead to the main storage never being quiet. One possibility is to place a maximum (quite high) on the number of readers allowed after the decision to flush the journal to main storage. If exceeded, then lock readers for a while, flush and resume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (JENA-703) TDB - Ensure readers don't block writeback forever.
[ https://issues.apache.org/jira/browse/JENA-703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215641#comment-14215641 ] Bala Kolla commented on JENA-703: - I think have found a way to do this by checking the size of commitedAwaitingFlush list and stop accepting the transaction (in begin method) until it can flush. This seems to be working and was able to stop the CPU drain and was able to support up to 500 users. Without these changes I would be hitting the CPU spike at 350 user load and system never recovered until I stopped the load. One drawback with this approach is if there is a really long running query (ex: filter system property like or modified date which matches many triples) then we may unnecessarily drag on all other queries. May be we can start checking this at lower threshold (ex:10 committed but not yet flushed) and try to stop queries for a limited amount of time (ex: 2 seconds) and if that pending queries doesn't free up then let it go until the next threshold (ex: 20) and if it hits the second threshold we will wait until all the pending transactions are done. Anyhow in my workload I don't have really complex queries and hence is working fine. Would have to test adding a long running query and see how it handles it. Please let me know your thoughts about this approach. TDB - Ensure readers don't block writeback forever. --- Key: JENA-703 URL: https://issues.apache.org/jira/browse/JENA-703 Project: Apache Jena Issue Type: Improvement Components: TDB Affects Versions: Jena 2.11.1 Reporter: Andy Seaborne The commit queue can only be flushed when the main storage is quiet. A stream of readers can lead to the main storage never being quiet. One possibility is to place a maximum (quite high) on the number of readers allowed after the decision to flush the journal to main storage. If exceeded, then lock readers for a while, flush and resume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
