jdere commented on a change in pull request #778: HIVE-22221: Llap external
client - Need to reduce LlapBaseInputFormat#getSplits() footprint
URL: https://github.com/apache/hive/pull/778#discussion_r327840568
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java
##########
@@ -192,9 +199,31 @@ public PlanFragment(TezWork work, Schema schema, JobConf
jc) {
@Override
public void process(Object[] arguments) throws HiveException {
+ initArgs(arguments);
+ try {
+ SplitResult splitResult = getSplitResult(false);
+ InputSplit[] splits = schemaSplitOnly ? new
InputSplit[]{splitResult.schemaSplit} : splitResult.actualSplits;
+ for (InputSplit s : splits) {
+ Object[] os = new Object[1];
+ bos.reset();
+ s.write(dos);
+ byte[] frozen = bos.toByteArray();
+ os[0] = frozen;
+ forward(os);
+ }
+ } catch (Exception e) {
+ throw new HiveException(e);
+ }
+ }
- String query = stringOI.getPrimitiveJavaObject(arguments[0]);
- int num = intOI.get(arguments[1]);
+ protected void initArgs(Object[] arguments) {
+ inputArgQuery = stringOI.getPrimitiveJavaObject(arguments[0]);
+ inputArgNumSplits = intOI.get(arguments[1]);
+ schemaSplitOnly = inputArgNumSplits == 0;
Review comment:
Under what circumstances does numSplits actually get set to 0, during actual
usage?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
