rahil-c opened a new issue, #14014:
URL: https://github.com/apache/hudi/issues/14014
### Bug Description
**What happened:**
Currently the test method `testUpgradeDowngrade` in class
`TestUpgradeDowngrade` fails on the CI for the following.
**What you expected:**
Test should not fail, so need to investigate to see what is causing this NPE.
**Steps to reproduce:**
Running the test locally usually passes so may be hard to repro.
### Environment
**Hudi version:**
master
**Query engine:** (Spark/Flink/Trino etc)
Spark
**Relevant configs:**
Run test locally
### Logs and Stack Trace
```
Error: testUpgradeOrDowngrade{HoodieTableVersion, HoodieTableVersion}[4]
Time elapsed: 0.932 s <<< ERROR!
org.apache.hudi.exception.HoodieException: Failed to downgrade LSM timeline
to old archived format
at
org.apache.hudi.table.upgrade.EightToSevenDowngradeHandler.downgradeFromLSMTimeline(EightToSevenDowngradeHandler.java:233)
at
org.apache.hudi.table.upgrade.EightToSevenDowngradeHandler.downgrade(EightToSevenDowngradeHandler.java:132)
at
org.apache.hudi.table.upgrade.UpgradeDowngrade.downgrade(UpgradeDowngrade.java:328)
at
org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:230)
at
org.apache.hudi.table.upgrade.TestUpgradeDowngrade.testUpgradeOrDowngrade(TestUpgradeDowngrade.java:112)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.util.Optional.ifPresent(Optional.java:159)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at
java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:272)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at java.util.ArrayList.forEach(ArrayList.java:1259)
Caused by: org.apache.hudi.exception.HoodieCommitException: Failed to
archive commits
at
org.apache.hudi.client.timeline.versioning.v1.TimelineArchiverV1.flushArchiveEntries(TimelineArchiverV1.java:173)
at
org.apache.hudi.table.upgrade.EightToSevenDowngradeHandler$ArchiveEntryFlusher.close(EightToSevenDowngradeHandler.java:274)
at
org.apache.hudi.table.upgrade.EightToSevenDowngradeHandler.downgradeFromLSMTimeline(EightToSevenDowngradeHandler.java:229)
... 45 more
Caused by: java.lang.NullPointerException
at
org.apache.hudi.avro.HoodieAvroUtils.indexedRecordToBytesStream(HoodieAvroUtils.java:148)
at
org.apache.hudi.avro.HoodieAvroUtils.avroToBytesStream(HoodieAvroUtils.java:144)
at
org.apache.hudi.common.model.HoodieAvroIndexedRecord.getAvroBytes(HoodieAvroIndexedRecord.java:325)
at
org.apache.hudi.common.table.log.block.HoodieAvroDataBlock.serializeRecords(HoodieAvroDataBlock.java:120)
at
org.apache.hudi.common.table.log.block.HoodieDataBlock.getContentBytes(HoodieDataBlock.java:132)
at
org.apache.hudi.common.table.log.HoodieLogFormatWriter.appendBlocks(HoodieLogFormatWriter.java:155)
at
org.apache.hudi.common.table.log.HoodieLogFormatWriter.appendBlock(HoodieLogFormatWriter.java:132)
at
org.apache.hudi.client.timeline.versioning.v1.TimelineArchiverV1.writeToFile(TimelineArchiverV1.java:447)
at
org.apache.hudi.client.timeline.versioning.v1.TimelineArchiverV1.flushArchiveEntries(TimelineArchiverV1.java:171)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
