bitblender commented on a change in pull request #1442: DRILL-6688 Data batches
for Project operator exceed the maximum specified
URL: https://github.com/apache/drill/pull/1442#discussion_r212781844
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/OutputWidthVisitor.java
##########
@@ -270,8 +270,10 @@ public OutputWidthExpression
visitIfElseWidthExpr(IfElseWidthExpr ifElseWidthExp
private OutputWidthExpression getFixedLenExpr(MajorType majorType) {
MajorType type = majorType;
if (Types.isFixedWidthType(type)) {
- int fixedWidth =
ProjectMemoryManager.getWidthOfFixedWidthType(type);
- return new OutputWidthExpression.FixedLenExpr(fixedWidth);
+ // Use only the width of the data. Metadata width will be
accounted for at the end
+ // This is to avoid using metadata size in intermediate
calculations
+ int fixedDataWidth =
ProjectMemoryManager.getDataWidthOfFixedWidthType(type);
Review comment:
The MinorType.NULL check is required for handling cases where there can be a
function with a null argument. This can happen, for instance, in
convert_to_JSON like in TestComplexTypeReader testNonExistentFieldConverting()
when trying to convert a non existent field.
See below for the object graph.
this = {DrillFuncHolderExpr@8281}
holder = {DrillSimpleFuncHolder@8279} "DrillSimpleFuncHolder
[functionNames=[convert_toJSON, convert_toSIMPLEJSON],
returnType=MajorType[minor_type: VARBINARY mode: REQUIRED],
nullHandling=NULL_IF_NULL, parameters=[ValueReference
[type=MajorType[minor_type: LATE mode: REQUIRED], name=input]]]"
majorType = {TypeProtos$MajorType@8317} "minor_type: VARBINARY\nmode:
OPTIONAL\n"
interpreter = null
args = {SingletonImmutableList@8315} size = 1
0 = {NullExpression@8285}
t = {TypeProtos$MajorType@8287} "minor_type: NULL\nmode: OPTIONAL\n"
nameUsed = "convert_tojson"
fieldReference = null
pos = {ExpressionPosition@8322}
"org.apache.drill.common.expression.ExpressionPosition@748c42c3[charIndex = -1,
expression = --UNKNOWN EXPRESSION--]"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
