lxcid commented on issue #7395:
URL: https://github.com/apache/pulsar/issues/7395#issuecomment-784846835
I'm on 2.7.0 I have a kinda similar error
```
04:52:27.949 [pulsar-client-io-7-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Subscribing to topic on cnx [id: 0xbdcc8063, L:/127.0.0.1:40794 -
R:127.0.0.1/127.0.0.1:6650], consumerId 0
04:52:27.956 [pulsar-client-io-7-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Subscribed to topic on 127.0.0.1/127.0.0.1:6650 -- consumer: 0
04:52:28.006 [pulsar-client-io-7-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Successfully getLastMessageId 14:1
04:52:28.021 [pulsar-client-io-7-1] INFO
com.scurrilous.circe.checksum.Crc32cIntChecksum - SSE4.2 CRC32C provider
initialized
04:52:28.027 [pulsar-external-listener-8-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Get topic last message Id
04:52:28.031 [pulsar-client-io-7-1] INFO
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Successfully getLastMessageId 14:1
04:52:28.228 [pulsar-client-io-7-1] WARN
org.apache.pulsar.client.impl.ClientCnx - [id: 0xbdcc8063, L:/127.0.0.1:40794 -
R:127.0.0.1/127.0.0.1:6650] Received error from server: Failed to get batch
size for entry
org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException:
No such ledger exists on Metadata Server
04:52:28.229 [pulsar-client-io-7-1] ERROR
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Failed getLastMessageId command
04:52:28.229 [pulsar-client-io-7-1] ERROR
org.apache.pulsar.client.impl.ConsumerImpl -
[public/default/debezium-postgres-source-debezium-offset-topic][reader-7759ef4898]
Failed getLastMessageId command
04:52:28.244 [pulsar-client-io-1-2] INFO
org.apache.pulsar.client.impl.ProducerImpl - [debezium-postgres-topic]
[standalone-1-0] Closed Producer
04:52:28.247 [main] INFO org.apache.pulsar.functions.LocalRunner -
RuntimeSpawner quit because of
org.apache.kafka.connect.errors.ConnectException: Failed to fetch offsets.
at
org.apache.kafka.connect.storage.OffsetStorageReaderImpl.offsets(OffsetStorageReaderImpl.java:82)
~[?:?]
at
io.debezium.connector.common.BaseSourceTask.getPreviousOffset(BaseSourceTask.java:265)
~[?:?]
at
io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:86)
~[?:?]
at
io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:106)
~[?:?]
at
org.apache.pulsar.io.kafka.connect.AbstractKafkaConnectSource.open(AbstractKafkaConnectSource.java:126)
~[?:?]
at
org.apache.pulsar.io.kafka.connect.KafkaConnectSource.open(KafkaConnectSource.java:62)
~[?:?]
at
org.apache.pulsar.io.debezium.DebeziumSource.open(DebeziumSource.java:100)
~[?:?]
at
org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupInput(JavaInstanceRunnable.java:700)
~[org.apache.pulsar-pulsar-functions-instance-2.7.0.jar:2.7.0]
at
org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:214)
~[org.apache.pulsar-pulsar-functions-instance-2.7.0.jar:2.7.0]
at
org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:234)
~[org.apache.pulsar-pulsar-functions-instance-2.7.0.jar:2.7.0]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_282]
Caused by: java.util.concurrent.ExecutionException:
org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The
subscription reader-7759ef4898 of the topic
persistent://public/default/debezium-postgres-source-debezium-offset-topic gets
the last message id was failed
Failed to get batch size for entry
org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException:
No such ledger exists on Metadata Server
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
~[?:1.8.0_282]
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
~[?:1.8.0_282]
at
org.apache.kafka.connect.storage.OffsetStorageReaderImpl.offsets(OffsetStorageReaderImpl.java:79)
~[?:?]
... 10 more
Caused by:
org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: The
subscription reader-7759ef4898 of the topic
persistent://public/default/debezium-postgres-source-debezium-offset-topic gets
the last message id was failed
Failed to get batch size for entry
org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException:
No such ledger exists on Metadata Server
at
org.apache.pulsar.client.api.PulsarClientException.wrap(PulsarClientException.java:853)
~[org.apache.pulsar-pulsar-client-api-2.7.0.jar:2.7.0]
at
org.apache.pulsar.client.impl.ConsumerImpl.lambda$internalGetLastMessageIdAsync$31(ConsumerImpl.java:2134)
~[org.apache.pulsar-pulsar-client-original-2.7.0.jar:2.7.0]
at
java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884)
~[?:1.8.0_282]
at
java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866)
~[?:1.8.0_282]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
~[?:1.8.0_282]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990)
~[?:1.8.0_282]
at
org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:632)
~[org.apache.pulsar-pulsar-client-original-2.7.0.jar:2.7.0]
at
org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:186)
~[org.apache.pulsar-pulsar-common-2.7.0.jar:2.7.0]
```
The main error line that is of interest is
> Received error from server: Failed to get batch size for entry
org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException:
No such ledger exists on Metadata Server
I have test this on 2.5.2 and above and they all provide similar errors,
only working version is 2.5.1.
This can be tested with pulsar-standalone running debezium. if you make sure
to store pulsar data between restart:
- start pulsar-standalone
- run debezium postgres source
- stop pulsar-standalone
- restart pulsar-standalone
- run debezium postgres source
- error occurs, debezium postgres can't resume
Similar or alike issues:
- https://github.com/apache/pulsar/issues/7395
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
