wcc1433 opened a new issue, #5395: URL: https://github.com/apache/paimon/issues/5395
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version Paimon(on oss): 1.0.1 ### Compute Engine Flink: 1.20.1 ### Minimal reproduce step The cdc data binlog was interrupted abnormally at night. After restarting the task the next morning (new task, not restarting from checkpoints/savepoints, because the binlog file was lost), the data could not be written. The exception was always thrown: org.apache.paimon.shade.org.apache.avro.AvroRuntimeException: Unrecognized codec: snappy ### What doesn't meet your expectations? 2025-04-03 11:31:23 java.io.IOException: org.apache.paimon.shade.org.apache.avro.AvroRuntimeException: Unrecognized codec: snappy at org.apache.paimon.flink.sink.cdc.CdcRecordStoreMultiWriteOperator.processElement(CdcRecordStoreMultiWriteOperator.java:171) at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:238) at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:157) at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:114) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:638) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:973) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:917) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:949) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.paimon.shade.org.apache.avro.AvroRuntimeException: Unrecognized codec: snappy at org.apache.paimon.shade.org.apache.avro.file.CodecFactory.fromString(CodecFactory.java:159) at org.apache.paimon.shade.org.apache.avro.file.DataFileStream.resolveCodec(DataFileStream.java:158) at org.apache.paimon.shade.org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:144) at org.apache.paimon.shade.org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:143) at org.apache.paimon.shade.org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:134) at org.apache.paimon.shade.org.apache.avro.file.DataFileReader.openReader(DataFileReader.java:74) at org.apache.paimon.format.avro.AvroBulkFormat$AvroReader.createReaderFromPath(AvroBulkFormat.java:82) at org.apache.paimon.format.avro.AvroBulkFormat$AvroReader.<init>(AvroBulkFormat.java:68) at org.apache.paimon.format.avro.AvroBulkFormat$AvroReader.<init>(AvroBulkFormat.java:55) at org.apache.paimon.format.avro.AvroBulkFormat.createReader(AvroBulkFormat.java:52) at org.apache.paimon.utils.FileUtils.createFormatReader(FileUtils.java:130) at org.apache.paimon.utils.ObjectsFile.createIterator(ObjectsFile.java:167) at org.apache.paimon.utils.ObjectsFile.readWithIOException(ObjectsFile.java:140) at org.apache.paimon.utils.ObjectsFile.read(ObjectsFile.java:123) at org.apache.paimon.utils.ObjectsFile.read(ObjectsFile.java:97) at org.apache.paimon.utils.ObjectsFile.read(ObjectsFile.java:93) at org.apache.paimon.manifest.ManifestList.readDataManifests(ManifestList.java:86) at org.apache.paimon.operation.AbstractFileStoreScan.readManifests(AbstractFileStoreScan.java:403) at org.apache.paimon.operation.AbstractFileStoreScan.readManifests(AbstractFileStoreScan.java:394) at org.apache.paimon.operation.AbstractFileStoreScan.doPlan(AbstractFileStoreScan.java:276) at org.apache.paimon.operation.AbstractFileStoreScan.plan(AbstractFileStoreScan.java:225) at org.apache.paimon.operation.AbstractFileStoreWrite.scanExistingFileMetas(AbstractFileStoreWrite.java:441) at org.apache.paimon.operation.AbstractFileStoreWrite.createWriterContainer(AbstractFileStoreWrite.java:398) at org.apache.paimon.operation.AbstractFileStoreWrite.lambda$getWriterWrapper$2(AbstractFileStoreWrite.java:373) at java.base/java.util.HashMap.computeIfAbsent(Unknown Source) at org.apache.paimon.operation.AbstractFileStoreWrite.getWriterWrapper(AbstractFileStoreWrite.java:372) at org.apache.paimon.operation.AbstractFileStoreWrite.write(AbstractFileStoreWrite.java:139) at org.apache.paimon.table.sink.TableWriteImpl.writeAndReturn(TableWriteImpl.java:175) at org.apache.paimon.flink.sink.StoreSinkWriteImpl.write(StoreSinkWriteImpl.java:180) at org.apache.paimon.flink.sink.cdc.CdcRecordStoreMultiWriteOperator.processElement(CdcRecordStoreMultiWriteOperator.java:169) ... 13 more ### Anything else? When CDC synchronizes data, other tasks also write data. This may be the reason for the exception. ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
