bharanic-dev commented on a change in pull request #13130:
URL: https://github.com/apache/pulsar/pull/13130#discussion_r762336742
##########
File path:
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/impl/AbstractMetadataStore.java
##########
@@ -317,7 +346,33 @@ public void invalidateAll() {
*/
protected void execute(Runnable task, CompletableFuture<?> future) {
try {
- executor.execute(task);
+ // Wrap the original task, so we can record the thread on which it
is running
+ TaskWrapper taskWrapper = new TaskWrapper(task);
+ executorWatchDog.execute(() -> {
Review comment:
What you propose also works, except that it is not easy to figure out
which code path(task) is the one that is actually blocked. Dumping the stack
trace of all threads to logs could result in large volume of logs that one has
to wade through to get to the task that is blocked. We could dump the stack
trace of all threads to a file, but need alternate mechanisms to extract the
file from the server.
One advantage with monitoring each task is that you get the dump of the
actual task that is potentially blocked. The 5 seconds is an arbitrary
interval, we can change it to 10s or more (whatever we feel is reasonable).
Also, note that at this time, we are only logging the stack trace. We could
kill the broker if the task exceeds more than say 30s, but I did not do it
because I thought that one might want to do that optionally (I can add a knob
if we agree that that is what we want to do).
Can you please explain what is the problem with tracking each task? The
callback executor is already a single-threaded, which can only execute one task
at a time. We can make the monitoring executor multi-threaded but that doesn't
help as long as the callback thread is single-threaded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
