[
https://issues.apache.org/jira/browse/JENA-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14204018#comment-14204018
]
Bala Kolla commented on JENA-801:
---------------------------------
[~andy.seaborne], sure I can contribute the code with the GuavaCache. It
definitely helped but pushed the bottleneck to BlockMgrJournal and that lead to
the discussion of using mapped i/o and hence the comment about the heap size
issue.
Here are the action items that I can think of..
Deliver the changes to use GuavaCache (to both the NodeTableCache and
BlockMgrCache) - As this is the first time that I am contributing the code
changes, I would like to know the process. I will plan to deliver the changes
next week.
Once I deliver the above changes we can further investigate the issue and see
if we can find any additional changes to improve the scalability. As you
mentioned earlier comments, this could be a case of work load saturating the
system resources and in that case our team had to take look at reducing the
work load.
Stretch goal..
Come up the with triple store that represents our data and come with a test
framework to reproduce this issue on this test data store. This could take some
time as I need to talk to my team if I can share our triple store (test data).
> When the server is under load, many queries are piling up and seems to be in
> some kind of dead lock.
> ----------------------------------------------------------------------------------------------------
>
> Key: JENA-801
> URL: https://issues.apache.org/jira/browse/JENA-801
> Project: Apache Jena
> Issue Type: Bug
> Components: TDB
> Affects Versions: TDB 0.9.4, Jena 2.11.2
> Reporter: Bala Kolla
> Attachments:
> ThreadLocksInBlockMgrJournalAfterGuavaCacheInNodeTable.htm,
> WAITDataReportShowingTheLockContention.zip,
> WAITDataReportShowingTheLockContentionWithoutQueryFilter.zip
>
>
> We were testing our server with repositories of varied sizes and in almost
> all the cases when the server peaks its capacity (of maximum number of users
> it can support), It seems like the queries are piling up because of the lock
> contention in NodeTableCache.
> Here are some details about the repository..
> size of indices on disk - 150GB
> type of hard disk used - SSD and HDD with high RAM (seeing the same result in
> both the cases)
> OS - Linux
> Details on the user load;
> We are trying to simulate a very active user load where all the users are
> executing many usecases that would result in many queries and updates on TDB.
> I would like to know what are the possible solutions to work around and avoid
> this situation. I am thinking of the following, please let me know if there
> is any other way to work around this bottleneck.
> Control the updates to the triple store so that we only do it when there are
> not many queries pending. We would have to experiment how this impact the
> usecases..
> Is there any other way to make this lock contention go away? Can we have
> multiple instances of this cache? For example many (90%) of our queries are
> executed with a query scope (per project). So, can we have a separate
> NodeTable cache for each query scope (project in our case) and one for
> global?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
