vvysotskyi commented on a change in pull request #2026: DRILL-7330: Implement
metadata usage for all format plugins
URL: https://github.com/apache/drill/pull/2026#discussion_r392716591
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/metastore/store/MetastoreFileTableMetadataProvider.java
##########
@@ -291,124 +250,74 @@ public boolean checkMetadataVersion() {
}
}
- private ParquetTableMetadataProvider getFallbackTableMetadataProvider()
throws IOException {
- if (fallback == null) {
- fallback = fallbackBuilder == null ? null : fallbackBuilder.build();
- }
- return fallback;
- }
+ public static class Builder<T extends Builder<T>> implements
FileTableMetadataProviderBuilder<T> {
+ protected final MetastoreMetadataProviderManager metadataProviderManager;
- private void throwIfChanged() {
- if (basicTablesRequests.hasMetastoreTableInfoChanged(metastoreTableInfo)) {
- throw
MetadataException.of(MetadataException.MetadataExceptionType.INCONSISTENT_METADATA);
- }
- }
+ // builder for fallback ParquetFileTableMetadataProvider
+ // for the case when required metadata is absent in Metastore
+ protected final TableMetadataProviderBuilder fallback;
- public static class Builder implements
ParquetFileTableMetadataProviderBuilder {
- private final MetastoreMetadataProviderManager metadataProviderManager;
+ protected TupleMetadata schema;
- private List<ReadEntryWithPath> entries;
- private DrillFileSystem fs;
- private TupleMetadata schema;
+ protected List<String> paths;
private FileSelection selection;
- // builder for fallback ParquetFileTableMetadataProvider
- // for the case when required metadata is absent in Metastore
- private final ParquetFileTableMetadataProviderBuilder fallback;
+ private DrillFileSystem fs;
public Builder(MetastoreMetadataProviderManager source) {
- this.metadataProviderManager = source;
- this.fallback = new
ParquetTableMetadataProviderImpl.Builder(FileSystemMetadataProviderManager.init());
+ this(source, new
SimpleFileTableMetadataProvider.Builder(FileSystemMetadataProviderManager.init()));
}
- @Override
- public ParquetFileTableMetadataProviderBuilder
withEntries(List<ReadEntryWithPath> entries) {
- this.entries = entries;
- fallback.withEntries(entries);
- return this;
+ protected Builder(MetastoreMetadataProviderManager source,
TableMetadataProviderBuilder fallback) {
+ this.metadataProviderManager = source;
+ this.fallback = fallback;
}
@Override
- public ParquetFileTableMetadataProviderBuilder withSelectionRoot(Path
selectionRoot) {
- fallback.withSelectionRoot(selectionRoot);
- return this;
+ public T withSchema(TupleMetadata schema) {
Review comment:
Good question. Drill has only two kinds of storage formats: Parquet and
"easy". All formats except Parquet, including Avro, use `EasyGroupScan` and
current `MetastoreFileTableMetadataProvider` class. But since there is a lot of
common code for Parquet and "easy" format kinds, it was reasonable to inherit
parquet class from this one and use the common code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
