[
https://issues.apache.org/jira/browse/BOOKKEEPER-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496927#comment-13496927
]
Sijie Guo commented on BOOKKEEPER-461:
--------------------------------------
thanks Yixue and Flavio for reviewing.
{quote}
change "Weird! Should not have two threads adding message to cache key {} at
the same time." to neutral "It is unexpected that more than one threads are
adding message to cache key {} at the same time."
{quote}
I will change the text.
{quote}
1. With enqueueWithoutFailureByTopic, does it make sense to keep
enqueueWithoutFailure?
2. I'm not entirely sure why you want some of the operations to execute in
separate threads.
{quote}
enqueueWithoutFailureByTopic is try to submit to different threads, each
threads process portion of topics.
enqueueWithoutFailure is try to use original cache thread to do some other
non-critical operations like updating readahead cache index and cleanup
readahead cache.
I don't want the non-critical operations affect those critical operations like
adding messages to readahead cache (affect publish operations) and delivering
next message (affect deliver operations). so it still keeps
enqueueWithoutFailure.
The other reason to keep enqueueWithoutFailure is that, the two indices (time
index and seq index) is quite difficult to modify them concurrently in high
performance. so it keeps cache thread to handle index staffs.
{quote}
In the following excerpt form your patch, you're inverting the order of
execution and I couldn't convince myself that it is not a problem.
{quote}
since all the indices operations would be enqueueWithoutFailure to process by
cache thread only, so the delay of index updates is OK.
{code}
Please update this: "so dont need to maintain" -> "so don't need to maintain"
{code}
will update the text.
{code}
In the graph you posted last, what are the sharp drops? Is that you restarting
the hub?
{code}
the drops is because we finished one loop of benchmark and starts a new one.
> Delivery throughput degrades when there are lots of publishers w/ high
> traffic.
> -------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-461
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-461
> Project: Bookkeeper
> Issue Type: Bug
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-461.diff, BOOKKEEPER-461.diff,
> pub_sub_multithreads.png, pub_sub_singlethread.png
>
>
> When running benchmarking over the hub server, found that delivery throughput
> degrades when there are lots of publishers publishing messages. And the
> delivery throughput will goes up when there is no publishes.
> This issue is introduced due to ReadAheadCache only runs a single thread. So
> when the netty workers are busy handling publish requests, they are pushing
> lots of messages into ReadAheadCache's queue to put them in to read ahead
> cache. So the readahead cache is busy on updating keys.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
