[
https://issues.apache.org/jira/browse/HUDI-7796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7796:
----------------------------
Description:
When running tests in Trino with Hudi MDT enabled, the following line in
HoodieAvroHFileWriter throws class cast exception, since Trino uses dependency
injection to provide the Hadoop file system instance, which may skip the
wrapper file system logic.
{code:java}
this.fs = (HoodieWrapperFileSystem) this.file.getFileSystem(conf); {code}
{code:java}
Caused by: java.lang.ClassCastException: class
io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper cannot be cast to class
org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem
(io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper and
org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem are in unnamed module of
loader 'app')
at
org.apache.hudi.io.hadoop.HoodieAvroHFileWriter.<init>(HoodieAvroHFileWriter.java:91)
at
org.apache.hudi.io.hadoop.HoodieAvroFileWriterFactory.newHFileFileWriter(HoodieAvroFileWriterFactory.java:108)
at
org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriterByFormat(HoodieFileWriterFactory.java:70)
at
org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriter(HoodieFileWriterFactory.java:53)
at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:108)
at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:77)
at
org.apache.hudi.io.CreateHandleFactory.create(CreateHandleFactory.java:45)
at
org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:101)
at
org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:44)
{code}
was:
When running tests in Trino with Hudi MDT enabled, the following line in
HoodieAvroHFileWriter throws class cast exception, since Trino uses dependency
injection and reflection to provide the Hadoop file system instance, which may
skip the wrapper file system logic.
{code:java}
this.fs = (HoodieWrapperFileSystem) this.file.getFileSystem(conf); {code}
{code:java}
Caused by: java.lang.ClassCastException: class
io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper cannot be cast to class
org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem
(io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper and
org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem are in unnamed module of
loader 'app')
at
org.apache.hudi.io.hadoop.HoodieAvroHFileWriter.<init>(HoodieAvroHFileWriter.java:91)
at
org.apache.hudi.io.hadoop.HoodieAvroFileWriterFactory.newHFileFileWriter(HoodieAvroFileWriterFactory.java:108)
at
org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriterByFormat(HoodieFileWriterFactory.java:70)
at
org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriter(HoodieFileWriterFactory.java:53)
at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:108)
at org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:77)
at
org.apache.hudi.io.CreateHandleFactory.create(CreateHandleFactory.java:45)
at
org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:101)
at
org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:44)
{code}
> Gracefully cast file system instance in Avro writers
> ----------------------------------------------------
>
> Key: HUDI-7796
> URL: https://issues.apache.org/jira/browse/HUDI-7796
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Ethan Guo
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.15.0, 1.0.0
>
>
> When running tests in Trino with Hudi MDT enabled, the following line in
> HoodieAvroHFileWriter throws class cast exception, since Trino uses
> dependency injection to provide the Hadoop file system instance, which may
> skip the wrapper file system logic.
> {code:java}
> this.fs = (HoodieWrapperFileSystem) this.file.getFileSystem(conf); {code}
> {code:java}
> Caused by: java.lang.ClassCastException: class
> io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper cannot be cast to class
> org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem
> (io.trino.hdfs.TrinoFileSystemCache$FileSystemWrapper and
> org.apache.hudi.hadoop.fs.HoodieWrapperFileSystem are in unnamed module of
> loader 'app')
> at
> org.apache.hudi.io.hadoop.HoodieAvroHFileWriter.<init>(HoodieAvroHFileWriter.java:91)
> at
> org.apache.hudi.io.hadoop.HoodieAvroFileWriterFactory.newHFileFileWriter(HoodieAvroFileWriterFactory.java:108)
> at
> org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriterByFormat(HoodieFileWriterFactory.java:70)
> at
> org.apache.hudi.io.storage.HoodieFileWriterFactory.getFileWriter(HoodieFileWriterFactory.java:53)
> at
> org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:108)
> at
> org.apache.hudi.io.HoodieCreateHandle.<init>(HoodieCreateHandle.java:77)
> at
> org.apache.hudi.io.CreateHandleFactory.create(CreateHandleFactory.java:45)
> at
> org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:101)
> at
> org.apache.hudi.execution.CopyOnWriteInsertHandler.consume(CopyOnWriteInsertHandler.java:44)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
