[
https://issues.apache.org/jira/browse/DRILL-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506735#comment-16506735
]
ASF GitHub Bot commented on DRILL-6340:
---------------------------------------
ilooner commented on a change in pull request #1302: DRILL-6340: Output Batch
Control in Project using the RecordBatchSizer
URL: https://github.com/apache/drill/pull/1302#discussion_r194206567
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
##########
@@ -497,30 +561,36 @@ private void setupNewSchemaFromInput(RecordBatch
incomingBatch) throws SchemaCha
final boolean useSetSafe = !(vector instanceof FixedWidthVector);
final ValueVectorWriteExpression write = new
ValueVectorWriteExpression(fid, expr, useSetSafe);
final HoldingContainer hc = cg.addExpr(write,
ClassGenerator.BlkCreateMode.TRUE_IF_BOUND);
+ memoryManager.addNewField(vector, write);
// We cannot do multiple transfers from the same vector. However we
still need to instantiate the output vector.
if (expr instanceof ValueVectorReadExpression) {
final ValueVectorReadExpression vectorRead =
(ValueVectorReadExpression) expr;
if (!vectorRead.hasReadPath()) {
final TypedFieldId id = vectorRead.getFieldId();
- final ValueVector vvIn =
incomingBatch.getValueAccessorById(id.getIntermediateClass(),
id.getFieldIds()).getValueVector();
+ final ValueVector vvIn =
incomingBatch.getValueAccessorById(id.getIntermediateClass(),
+ id.getFieldIds()).getValueVector();
vvIn.makeTransferPair(vector);
}
}
- logger.debug("Added eval for project expression.");
}
}
try {
CodeGenerator<Projector> codeGen = cg.getCodeGenerator();
codeGen.plainJavaCapable(true);
// Uncomment out this line to debug the generated code.
- // codeGen.saveCodeForDebugging(true);
+ //codeGen.saveCodeForDebugging(true);
this.projector = context.getImplementationClass(codeGen);
projector.setup(context, incomingBatch, this, transfers);
} catch (ClassTransformationException | IOException e) {
throw new SchemaChangeException("Failure while attempting to load
generated class", e);
}
+
+ long setupNewSchemaEndTime = System.currentTimeMillis();
+ if (logger.isTraceEnabled()) {
+ logger.trace("setupNewSchemaFromInput: time " + (setupNewSchemaEndTime -
setupNewSchemaStartTime) + " ms" + ", proj " + this + " incoming " +
incomingBatch);
Review comment:
{}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Output Batch Control in Project using the RecordBatchSizer
> ----------------------------------------------------------
>
> Key: DRILL-6340
> URL: https://issues.apache.org/jira/browse/DRILL-6340
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Reporter: Karthikeyan Manivannan
> Assignee: Karthikeyan Manivannan
> Priority: Major
> Fix For: 1.14.0
>
>
> This bug is for tracking the changes required to implement Output Batch
> Sizing in Project using the RecordBatchSizer. The challenge in doing this
> mainly lies in dealing with expressions that produce variable-length columns.
> The following doc talks about some of the design approaches for dealing with
> such variable-length columns.
> [https://docs.google.com/document/d/1h0WsQsen6xqqAyyYSrtiAniQpVZGmQNQqC1I2DJaxAA/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
