szlta commented on code in PR #3607:
URL: https://github.com/apache/hive/pull/3607#discussion_r979686170
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##########
@@ -176,10 +179,27 @@ public boolean shouldSkipCombine(Path path, Configuration
conf) {
}
@Override
- public VectorizedSupport.Support[] getSupportedFeatures() {
+ public VectorizedSupport.Support[] getSupportedFeatures(HiveConf hiveConf,
Set<String> tableNames) {
// disabling VectorizedSupport.Support.DECIMAL_64 as Parquet doesn't
support it, and we have no way of telling
// beforehand what kind of file format we're going to hit later
- return new VectorizedSupport.Support[]{ };
+ boolean onlyOrcFiles = true;
+ try {
+ Hive hiveDb = Hive.get(hiveConf, false);
Review Comment:
I agree that doing HMS calls from InputFormat is not a nice pattern, but I'm
not really fond of hardcoding Iceberg related stuff to hive-exec either.
Can't we refactor Vectorizer instead so that supplies the Table object or
its properties to getSupportedFeatures()? Perhaps we could get this info from
`verifyAndSetVectorPartDesc` through `addVectorizedInputFileFormatSupport` and
finally through `getVectorizedInputFormatSupports` ?
It's a bigger refactor but the responsibility of deciding decimal64 or other
vector features would remain solely within InputFormats.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
