[
https://issues.apache.org/jira/browse/SPARK-33255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221126#comment-17221126
]
Yang Jie commented on SPARK-33255:
----------------------------------
[~hyukjin.kwon] Got it ~
> Use new API to construct ParquetFileReader and read Parquet footer
> ------------------------------------------------------------------
>
> Key: SPARK-33255
> URL: https://issues.apache.org/jira/browse/SPARK-33255
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: Yang Jie
> Priority: Major
>
> {code:java}
> /**
> * @param configuration the Hadoop conf
> * @param fileMetaData fileMetaData for parquet file
> * @param filePath Path for the parquet file
> * @param blocks the blocks to read
> * @param columns the columns to read (their path)
> * @throws IOException if the file can not be opened
> * @deprecated will be removed in 2.0.0.
> */
> @Deprecated
> public ParquetFileReader(
> Configuration configuration, FileMetaData fileMetaData,
> Path filePath, List<BlockMetaData> blocks, List<ColumnDescriptor>
> columns) throws IOException {
> {code}
> {code:java}
> /**
> * Reads the meta data block in the footer of the file
> * @param configuration a configuration
> * @param file the parquet File
> * @param filter the filter to apply to row groups
> * @return the metadata blocks in the footer
> * @throws IOException if an error occurs while reading the file
> * @deprecated will be removed in 2.0.0;
> * use {@link ParquetFileReader#open(InputFile,
> ParquetReadOptions)}
> */
> @Deprecated
> public static final ParquetMetadata readFooter(Configuration configuration,
> FileStatus file, MetadataFilter filter) throws IOException
> {code}
> {code:java}
> /**
> * Reads the meta data in the footer of the file.
> * Skipping row groups (or not) based on the provided filter
> * @param configuration a configuration
> * @param file the Parquet File
> * @param filter the filter to apply to row groups
> * @return the metadata with row groups filtered.
> * @throws IOException if an error occurs while reading the file
> * @deprecated will be removed in 2.0.0;
> * use {@link ParquetFileReader#open(InputFile,
> ParquetReadOptions)}
> */
> public static ParquetMetadata readFooter(Configuration configuration, Path
> file, MetadataFilter filter) throws IOException{code}
> in ParquetFileReader were marked as deprecated, use
> {code:java}
> public ParquetFileReader(InputFile file, ParquetReadOptions options) throws
> IOException
> {code}
> {code:java}
> public ParquetMetadata getFooter()
> {code}
> to instead of them.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
