[
https://issues.apache.org/jira/browse/DRILL-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-7435:
-------------------------------
Description:
Run Drill with a fix for DRILL-7434. Now, another test fails:
{{TestJsonReader.testTypeCase()}} fails when it tries to set the value count.
Evidently the Project operator has added the {{LATE}} type to the Union vector.
However, there is no vector type associated with the {{LATE}} type. An attempt
to get the member or this type throws an exception.
The simple work around is to special-case this type when setting the value
count. The longer-term fix is to not add the {{LATE}} type to a union vector.
The problem appears to occur here:
{noformat}
Daemon Thread [2240a19e-344e-9a8b-f3d9-2a1550662b1b:frag:0:0] (Suspended
(breakpoint at line 2091 in TypeProtos$MajorType$Builder))
TypeProtos$MajorType$Builder.addSubType(TypeProtos$MinorType) line:
2091
DefaultReturnTypeInference.getType(List<LogicalExpression>,
FunctionAttributes) line: 58
FunctionTemplate$ReturnType.getType(List<LogicalExpression>,
FunctionAttributes) line: 195
DrillSimpleFuncHolder(DrillFuncHolder).getReturnType(List<LogicalExpression>)
line: 401
DrillFuncHolderExpr.<init>(String, DrillFuncHolder,
List<LogicalExpression>, ExpressionPosition) line: 39
DrillSimpleFuncHolder(DrillFuncHolder).getExpr(String,
List<LogicalExpression>, ExpressionPosition) line: 113
ExpressionTreeMaterializer.addCastExpression(LogicalExpression,
TypeProtos$MajorType, FunctionLookupContext, ErrorCollector, boolean) line: 235
ExpressionTreeMaterializer$MaterializeVisitor(ExpressionTreeMaterializer$AbstractMaterializeVisitor).visitIfExpression(IfExpression,
FunctionLookupContext) line: 638
ExpressionTreeMaterializer$MaterializeVisitor(ExpressionTreeMaterializer$AbstractMaterializeVisitor).visitIfExpression(IfExpression,
Object) line: 332
IfExpression.accept(ExprVisitor<T,V,E>, V) line: 65
ExpressionTreeMaterializer.materialize(LogicalExpression,
Map<VectorAccessible,BatchReference>, ErrorCollector, FunctionLookupContext,
boolean, boolean) line: 165
ExpressionTreeMaterializer.materialize(LogicalExpression,
VectorAccessible, ErrorCollector, FunctionLookupContext, boolean, boolean)
line: 143
ProjectRecordBatch.setupNewSchemaFromInput(RecordBatch) line: 482
ProjectRecordBatch.setupNewSchema() line: 571
ProjectRecordBatch(AbstractUnaryRecordBatch<T>).innerNext() line: 99
ProjectRecordBatch.innerNext() line: 144
...
{noformat}
This appears to be processing the if statement in the following test query:
{noformat}
.sqlQuery("select case when is_bigint(field1) " +
"then field1 when is_list(field1) then field1[0] " +
"when is_map(field1) then t.field1.inner1 end f1 from
cp.`jsoninput/union/a.json` t")
{noformat}
The problem appears to be that a function says it takes data of type LATE, and
then that data is added to the Union. Not sure of the exact solution, but
simply omitting the LATE value from the Union seems to work.
was:
Run Drill with a fix for DRILL-7434. Now, another test fails:
{{TestJsonReader.testTypeCase()}} fails when it tries to set the value count.
Evidently the JSON reader has added the {{LATE}} type to the Union vector.
However, there is no vector type associated with the {{LATE}} type. An attempt
to get the member or this type throws an exception.
The simple work around is to special-case this type when setting the value
count. The longer-term fix is to not add the {{LATE}} type to a union vector.
> Project operator incorrectly adds a LATE type to union vector
> -------------------------------------------------------------
>
> Key: DRILL-7435
> URL: https://issues.apache.org/jira/browse/DRILL-7435
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Paul Rogers
> Priority: Major
>
> Run Drill with a fix for DRILL-7434. Now, another test fails:
> {{TestJsonReader.testTypeCase()}} fails when it tries to set the value count.
> Evidently the Project operator has added the {{LATE}} type to the Union
> vector. However, there is no vector type associated with the {{LATE}} type.
> An attempt to get the member or this type throws an exception.
> The simple work around is to special-case this type when setting the value
> count. The longer-term fix is to not add the {{LATE}} type to a union vector.
> The problem appears to occur here:
> {noformat}
> Daemon Thread [2240a19e-344e-9a8b-f3d9-2a1550662b1b:frag:0:0] (Suspended
> (breakpoint at line 2091 in TypeProtos$MajorType$Builder))
> TypeProtos$MajorType$Builder.addSubType(TypeProtos$MinorType) line:
> 2091
> DefaultReturnTypeInference.getType(List<LogicalExpression>,
> FunctionAttributes) line: 58
> FunctionTemplate$ReturnType.getType(List<LogicalExpression>,
> FunctionAttributes) line: 195
>
> DrillSimpleFuncHolder(DrillFuncHolder).getReturnType(List<LogicalExpression>)
> line: 401
> DrillFuncHolderExpr.<init>(String, DrillFuncHolder,
> List<LogicalExpression>, ExpressionPosition) line: 39
> DrillSimpleFuncHolder(DrillFuncHolder).getExpr(String,
> List<LogicalExpression>, ExpressionPosition) line: 113
> ExpressionTreeMaterializer.addCastExpression(LogicalExpression,
> TypeProtos$MajorType, FunctionLookupContext, ErrorCollector, boolean) line:
> 235
>
> ExpressionTreeMaterializer$MaterializeVisitor(ExpressionTreeMaterializer$AbstractMaterializeVisitor).visitIfExpression(IfExpression,
> FunctionLookupContext) line: 638
>
> ExpressionTreeMaterializer$MaterializeVisitor(ExpressionTreeMaterializer$AbstractMaterializeVisitor).visitIfExpression(IfExpression,
> Object) line: 332
> IfExpression.accept(ExprVisitor<T,V,E>, V) line: 65
> ExpressionTreeMaterializer.materialize(LogicalExpression,
> Map<VectorAccessible,BatchReference>, ErrorCollector, FunctionLookupContext,
> boolean, boolean) line: 165
> ExpressionTreeMaterializer.materialize(LogicalExpression,
> VectorAccessible, ErrorCollector, FunctionLookupContext, boolean, boolean)
> line: 143
> ProjectRecordBatch.setupNewSchemaFromInput(RecordBatch) line: 482
> ProjectRecordBatch.setupNewSchema() line: 571
> ProjectRecordBatch(AbstractUnaryRecordBatch<T>).innerNext() line: 99
> ProjectRecordBatch.innerNext() line: 144
> ...
> {noformat}
> This appears to be processing the if statement in the following test query:
> {noformat}
> .sqlQuery("select case when is_bigint(field1) " +
> "then field1 when is_list(field1) then field1[0] " +
> "when is_map(field1) then t.field1.inner1 end f1 from
> cp.`jsoninput/union/a.json` t")
> {noformat}
> The problem appears to be that a function says it takes data of type LATE,
> and then that data is added to the Union. Not sure of the exact solution, but
> simply omitting the LATE value from the Union seems to work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
