dlg99 opened a new issue #12070:
URL: https://github.com/apache/pulsar/issues/12070
**Describe the bug**
Debezium connector hung (was not processing messages) and was not working
after restarts until the offset storage topic was deleted and recreated.
**To Reproduce**
I do not have a repro, this happened in prod and so far I cannot repro it.
Interesting log excerpts:
```
[tenant/namespace/debezium-postgres-source-0] ERROR
org.apache.pulsar.functions.instance.JavaInstanceRunnable - Source open
produced uncaught exception:
java.util.concurrent.CompletionException:
org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The
subscription reader-c791f27169 of the topic
persistent://tenant/namespace/debezium-postgres-source-debezium-offset-topic
gets the last message id was failed
Failed to get batch size for entry
org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException:
No such ledger exists on Metadata Server
```
```
[bookkeeper-ml-workers-OrderedExecutor-1-0] INFO
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl -
[replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic]
Removing ledger 375120 - size: 1049452
[bookkeeper-ml-workers-OrderedExecutor-1-0] INFO
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl -
[replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic]
End TrimConsumedLedgers. ledgers=1 totalSize=0
[bookkeeper-ml-workers-OrderedExecutor-1-0] INFO
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl -
[replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic]
Ledger 375120 contains the current last confirmed entry 375120:3160, and it is
going to be deleted
[bookkeeper-ml-workers-OrderedExecutor-1-0] WARN
org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl -
[replicated-tenant/gns/persistent/debezium-postgres-source-debezium-offset-topic]
Failed to mark delete while trimming data ledgers: Invalid mark deleted
position
```
retention:
```
bin/pulsar-admin namespaces get-retention tenant/namespace
{
"retentionTimeInMinutes" : -1,
"retentionSizeInMB" : 1
}
```
Past issues that look similar: https://github.com/apache/pulsar/issues/8677
PR: https://github.com/apache/pulsar/pull/8725
**Expected behavior**
connector not hanging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
