deniskuzZ commented on code in PR #6793: URL: https://github.com/apache/hive/pull/6793#discussion_r4155066626
##########
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapIoImpl.java:
##########
@@ -122,6 +128,7 @@ public class LlapIoImpl implements
LlapIo<VectorizedRowBatch>, LlapIoDebugDump {
// TODO: later, we may have a map
private final ColumnVectorProducer orcCvp, genericCvp;
+ private final ColumnVectorProducer parquetCvp;
Review Comment:
could we refactor to keep class generic. llap-client sees LlapOrcIo /
LlapParquetIo from the earlier sketch
````
/** One natively served format: its input-format class and the producer
that decodes it. */
interface LlapFormatIo {
Class<? extends InputFormat> inputFormatClass();
@Nullable ColumnVectorProducer producer(); // null when this daemon
cannot serve it natively
}
final class LlapOrcIoImpl implements LlapOrcIo, LlapFormatIo {
private final ColumnVectorProducer cvp; // OrcColumnVectorProducer,
always present
// getTailFromCache(...), vectorizedReaderForPath(...) move here
unchanged
}
final class LlapParquetIoImpl implements LlapParquetIo, LlapFormatIo {
private final ColumnVectorProducer cvp; // null unless
useLowLevelCache (F5), optionally && daemon flag
// getFooterBuffersFromCache(...) moves here; vectorizedReaderForPath
arrives with its Iceberg caller
}
````
LlapIoImpl then holds orc, parquet, genericCvp and formats = List.of(orc,
parquet), and the instanceof chain collapses:
````
@Override
public InputFormat<NullWritable, VectorizedRowBatch>
getInputFormat(InputFormat<?, ?> src, Deserializer serde) {
ColumnVectorProducer cvp = serde != null ? genericCvp :
nativeProducerFor(src.getClass());
if (cvp == null) {
LOG.warn("No LLAP IO producer for {} (serde-based: {})",
src.getClass(), serde != null);
return null; // wrapForLlap falls to
injectLlapCaches as today
}
return new LlapInputFormat(src, serde, cvp, executor, daemonConf);
}
private ColumnVectorProducer nativeProducerFor(Class<?> ifClass) {
for (LlapFormatIo f : formats) {
if (f.inputFormatClass().isAssignableFrom(ifClass)) return
f.producer();
}
return null;
}
@Override public LlapOrcIo orc() { return orc; }
@Override public LlapParquetIo parquet() { return parquet; }
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
