Hi openinx I do not get you. what do you mean by 'Looks like the line 112 in HadoopReadOptions is not the first line accessing the variables in ParquetInputFormat.'? The parquet file I want to read was wrote by iceberg table without any explicit specified, no file format and no parquet version was specified. I just want to read the parquet file by iceberg, when read, there was also no explicit file format and parquet version.
OpenInx <open...@gmail.com> 于2021年9月23日周四 下午12:34写道： > Hi Joshua > > Can you check what's the parquet version you are using ? Looks like the > line 112 in HadoopReadOptions is not the first line accessing the variables > in ParquetInputFormat. > > [image: image.png] > > On Wed, Sep 22, 2021 at 11:07 PM Joshua Fan <joshuafat...@gmail.com> > wrote: > >> Hi >> I am glad to use iceberg as table source in flink sql, flink version is >> 1.13.2, and iceberg version is 0.12.0. >> >> After changed the flink version from 1.12 to 1.13, and changed some codes >> in FlinkCatalogFactory, the project can be build successfully. >> >> First, I tried to write data into iceberg by flink sql, and it seems go >> well. And then I want to verify the data, so I want to read from iceberg >> table, I wrote a >> simple sql, like "select * from >> iceberg_catalog.catalog_database.catalog_table", the sql can be submitted, >> but the flink job kept restarting by 'java.lang.NoClassDefFoundError: >> org/apache/iceberg/shaded/org/apache/parquet/hadoop/ParquetInputFormat'. >> But, actually, ParquetInputFormat was in the >> iceberg-flink-runtime-0.12.0.jar. >> Has no idea why this can happen. >> The full stack trace is below: >> java.lang.NoClassDefFoundError: >> org/apache/iceberg/shaded/org/apache/parquet/hadoop/ParquetInputFormat >> at >> org.apache.iceberg.shaded.org.apache.parquet.HadoopReadOptions$Builder.<init>(HadoopReadOptions.java:112) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.shaded.org.apache.parquet.HadoopReadOptions$Builder.<init>(HadoopReadOptions.java:97) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.shaded.org.apache.parquet.HadoopReadOptions.builder(HadoopReadOptions.java:85) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.parquet.Parquet$ReadBuilder.build(Parquet.java:793) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.flink.source.RowDataIterator.newParquetIterable(RowDataIterator.java:135) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.flink.source.RowDataIterator.newIterable(RowDataIterator.java:86) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.flink.source.RowDataIterator.openTaskIterator(RowDataIterator.java:74) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.flink.source.DataIterator.updateCurrentIterator(DataIterator.java:102) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.flink.source.DataIterator.hasNext(DataIterator.java:84) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.iceberg.flink.source.FlinkInputFormat.reachedEnd(FlinkInputFormat.java:104) >> ~[iceberg-flink-runtime-0.12.0-qihoo.jar:?] >> at >> org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:89) >> ~[flink-dist_2.11-1.13.2.jar:1.13.2] >> at >> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) >> ~[flink-dist_2.11-1.13.2.jar:1.13.2] >> at >> org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) >> ~[flink-dist_2.11-1.13.2.jar:1.13.2] >> at >> org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:269) >> ~[flink-dist_2.11-1.13.2.jar:1.13.2] >> You can see that the HadoopReadOptions can be found. >> >> Any help will be appricated. Thank you. >> >> Yours sincerely >> >> Josh >> >
