[
https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541262
]
Jørgen Løland commented on DERBY-2911:
--------------------------------------
Hi Knut,
Thanks for providing these nice graphs; they make it much easier to understand
the effects of the new buffer manager. The results are dramatic in many of the
illustrated cases!
Intuitively, I would say that your plan of using a more aggressive background
cleaner sounds good since user threads are not using the pages they have
cleaned (although I don't understand why not using that page is better). The
graphs also clearly indicate that this approach is better than letting the user
thread do the job.
One question: what happens if the clock rotates too fast and reaches a
synchronized (due to I/O) page? Will it have to wait for the synchronization to
be lifted, or can the clock jump to the next page somehow? Since the background
cleaner is used to write most of the pages, it (the background cleaner) could
potentially be far behind the clock if I understand this correctly. If the
clock is unnecessarily blocked by I/O it's game over ;)
On a side note - why are performance tests in Derby performed with back-to-back
transactions rather than using a more realistic (i.e., Poisson distributed)
load? Poisson is normally used to simulate the transaction arrival rate from
many independent clients [1]. In your case, I think the background cleaner
could perform even better with this (more realistic) workload since it would be
scheduled in the short periods with slightly fewer active transactions.
[1] http://en.wikipedia.org/wiki/Poisson_process
> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>
> Key: DERBY-2911
> URL: https://issues.apache.org/jira/browse/DERBY-2911
> Project: Derby
> Issue Type: Improvement
> Components: Performance, Services
> Affects Versions: 10.4.0.0
> Reporter: Knut Anders Hatlen
> Assignee: Knut Anders Hatlen
> Priority: Minor
> Attachments: cleaner.diff, cleaner.tar, d2911-1.diff, d2911-1.stat,
> d2911-2.diff, d2911-3.diff, d2911-4.diff, d2911-5.diff, d2911-6.diff,
> d2911-6.stat, d2911-7.diff, d2911-7a.diff, d2911-entry-javadoc.diff,
> d2911-unused.diff, d2911-unused.stat, d2911perf.java, perftest6.pdf
>
>
> There are indications that the buffer manager is a bottleneck for some types
> of multi-user load. For instance, Anders Morken wrote this in a comment on
> DERBY-1704: "With a separate table and index for each thread (to remove latch
> contention and lock waits from the equation) we (...) found that
> org.apache.derby.impl.services.cache.Clock.find()/release() caused about 5
> times more contention than the synchronization in LockSet.lockObject() and
> LockSet.unlock(). That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer
> manager which exploits the concurrency utilities added in Java SE 5.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
