jerolba opened a new pull request, #3781:
URL: https://github.com/apache/parquet-java/pull/3781
### Rationale for this change
Reading the Parquet read configuration via `ParquetReadOptions` previously
referenced
`ParquetInputFormat.getFilter(...)` and statically imported its constants.
`ParquetInputFormat` extends
`org.apache.hadoop.mapreduce.lib.input.FileInputFormat`, so using the read
configuration forced the JVM
to initialize `ParquetInputFormat` and therefore `FileInputFormat` and its
whole
`org.apache.hadoop.mapreduce.*` transitive dependency graph, even though
only plain `String`/`boolean`
config properties were needed. The new `ParquetInputProperties` (constants)
and `ParquetInputFilters`
(filter resolution) centralise the `ParquetConfiguration`-based read
configuration so that read-only
consumers no longer transitively pull in the Hadoop `mapreduce` dependency.
### What changes are included in this PR?
#### New files
-
`parquet-hadoop/src/main/java/org/apache/parquet/conf/ParquetInputProperties.java`
- Declares all Parquet read-configuration constants previously on
`ParquetInputFormat`:
`READ_SUPPORT_CLASS`, `UNBOUND_RECORD_FILTER`, `STRICT_TYPE_CHECKING`,
`FILTER_PREDICATE`,
`RECORD_FILTERING_ENABLED`, `STATS_FILTERING_ENABLED`,
`DICTIONARY_FILTERING_ENABLED`,
`COLUMN_INDEX_FILTERING_ENABLED`, `PAGE_VERIFY_CHECKSUM_ENABLED`,
`BLOOM_FILTERING_ENABLED`,
`OFF_HEAP_DECRYPT_BUFFER_ENABLED`, `HADOOP_VECTORED_IO_ENABLED`,
`HADOOP_VECTORED_IO_DEFAULT`.
- Constants only; no `org.apache.hadoop.mapreduce` dependency.
-
`parquet-hadoop/src/main/java/org/apache/parquet/conf/ParquetInputFilters.java`
- `ParquetConfiguration`-based filter resolution:
`getFilter(ParquetConfiguration)`,
`getUnboundRecordFilter(ParquetConfiguration)` and
`getFilterPredicate(ParquetConfiguration)`.
- No `org.apache.hadoop.mapreduce` dependency.
#### Removed
-
`parquet-hadoop/src/main/java/org/apache/parquet/ParquetInputConfiguration.java`
(intermediate
class, split into the two classes above).
#### Modified
- `parquet-hadoop/.../ParquetReadOptions.java`
- Static imports now resolve to `ParquetInputProperties`, and the
`getFilter(...)` call to
`ParquetInputFilters` instead of `ParquetInputFormat`.
`ParquetReadOptions` no longer references
`ParquetInputFormat`.
- `parquet-hadoop/.../hadoop/InternalParquetRecordReader.java`
- Static imports of `RECORD_FILTERING_ENABLED` / `STRICT_TYPE_CHECKING`
now from
`ParquetInputProperties`.
- `parquet-hadoop/.../hadoop/ParquetInputFormat.java`
- Constants are now `@Deprecated`, delegating to
`ParquetInputProperties.*`.
- Kept the legacy Hadoop `Configuration`-based `getFilter(Configuration)` /
`getUnboundRecordFilter(Configuration)` overloads (used by the MapReduce
read path), delegating by
wrapping into `HadoopParquetConfiguration`. Internal callers use
`ParquetInputFilters` /
`ParquetInputProperties` directly.
- Tests updated accordingly (`DeprecatedInputFormatTest`,
`TestParquetFileWriter`,
`TestInputOutputFormat`, `TestInputFormatColumnProjection`,
`TestDataPageChecksums`,
`TestColumnChunkPageWriteStore`, `TestPropertiesDrivenEncryption`).
### Are these changes tested?
No change in behavior, just refactoring code location. Code compilation and
existing tests validate the change.
### Are there any user-facing changes?
No. Fully backward and binary compatible: the `ParquetInputFormat` constants
and methods are retained
(deprecated) and delegate to the new class; constant string/boolean values
are unchanged, so any
previously-serialised configuration still works.
Closes #3780
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
