ThomasTaketurns commented on issue #21987:
URL: https://github.com/apache/pulsar/issues/21987#issuecomment-1916768936
Hello @horizonzy ,
Thanks for your quick answer.
Here is what I just did :
1. I updated the log level of recovery via an env variable in my helm chart
yaml autorecovery-statefulset.yaml
env:
- name: BOOKIE_EXTRA_OPTS
value: "-Dpulsar.log.level=debug -Dpulsar.log.root.level=debug"
2. I redeployed the statefulset and can see the DEBUG logs. At this step,
the ReplicationWorker is working as I would expect.
3. Then I verified that I do not have any underreplicated ledger.
./bin/bookkeeper shell listunderreplicated
2024-01-30T11:44:22,039+0000 [main] INFO
org.apache.bookkeeper.tools.cli.commands.autorecovery.ListUnderReplicatedCommand
- Under replicated ledger count: 0
4. When I kill one bookie, I can see the auditor identifying newly under
replicated ledgers.
[auditorLogs.txt](https://github.com/apache/pulsar/files/14097243/auditorLogs.txt)
./bin/bookkeeper shell listunderreplicated
2024-01-30T11:58:39,785+0000 [main] INFO
org.apache.bookkeeper.tools.cli.commands.autorecovery.ListUnderReplicatedCommand
- Under replicated ledger count: 1885
5. Nothing seem to happen on the ReplicationWorker side, I cannot find any
log for such thread. I will attach other logs from recovery pod though.
[recoveryPodLogs.txt](https://github.com/apache/pulsar/files/14097593/recoveryPodLogs.txt)
6. I also attach a thread dump of the recovery JVM.
[td00.txt](https://github.com/apache/pulsar/files/14097645/td00.txt)
Pulsar version : 3.0.0
It is complicated to upgrade to the latest version for now since threre are
other persons working on the same K8S cluster I use for my tests. If we cannot
investigate further, I will need to find a way to deploy a brand new cluster.
Sincerely,
Thomas @ TakeTurns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
