lordcheng10 opened a new pull request, #16420:
URL: https://github.com/apache/pulsar/pull/16420
### Motivation
In the log we found No such ledger exception:
07:31:55.118 [broker-topic-workers-OrderedScheduler-0-0] ERROR
org.apache.pulsar.broker.service.persistent.PersistentDispatcherSingleActiveConsumer
-
[persistent://tenant_g_cdg_cft_tdbank__cft/b_cdg_cft_dz_lct/lcttengan_item_dc_db_xx_t_item_dc_netvalue_x_tengan_hh-partition-2
/
sort_lj_cft_pulsar_to_thive_gz_cft_1_v2_9_lcttengan_item_dc_db_xx_t_item_dc_netvalue_x_tengan_hh_consumer_group-Consumer{subscription=PersistentSubscription{topic=persistent://tenant_g_cdg_cft_tdbank__cft/b_cdg_cft_dz_lct/lcttengan_item_dc_db_xx_t_item_dc_netvalue_x_tengan_hh-partition-2,
name=sort_lj_cft_pulsar_to_thive_gz_cft_1_v2_9_lcttengan_item_dc_db_xx_t_item_dc_netvalue_x_tengan_hh_consumer_group},
consumerId=62, consumerName=55bd8, address=/11.168.42.86:35383}] Error reading
entries at 16092024:0 : No such ledger exists on Metadata Server - Retrying to
read in 58.032 seconds
The reasons are as follows:
1. Frequent Full GC occurs on the broker, causing zk to time out：
22:29:06.119 [main-EventThread] ERROR
org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client
connection to the ZooKeeper server has expired!
2.When the create leadger is completed, the ledger will be put into ledgers
and update currentLedger，the updateLedgersListAfterRollover method is executed,
and the metadata to zookeeper fails to write：
<img width="1449" alt="image"
src="https://user-images.githubusercontent.com/19296967/177555986-96b40448-c831-4622-94d9-1385a4605d3e.png";>
https://github.com/apache/pulsar/blob/4c958a9a1a0468656e37851293cf1930041d2c93/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L1484-L1489
3.In the failure callback method operationFailed, the corresponding ledger
will be removed from the ledgers, but the currentLedger still points to the
ledger that failed to create, but the ledger will be deleted here through
bookKeeper.asyncDeleteLedger. When reading data, it will be read through the
currentLedger. Since the ledger has been deleted, the final error is reported:
No such ledger
https://github.com/apache/pulsar/blob/4c958a9a1a0468656e37851293cf1930041d2c93/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L1509-L1522
### Modifications
### Documentation
Check the box below or label this PR directly.
Need to update docs?
- [ ] `doc-required`
(Your PR needs to update docs and you will update later)
- [ ] `doc-not-needed`
(Please explain why)
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-complete`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
![https://user-images.githubusercontent.com/19296967/177555986-96b40448-c831-4622-94d9-1385a4605d3e.png"](https://user-images.githubusercontent.com/19296967/177555986-96b40448-c831-4622-94d9-1385a4605d3e.png"){kind=link}