[
https://issues.apache.org/jira/browse/DRILL-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16420107#comment-16420107
]
ASF GitHub Bot commented on DRILL-6223:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/1170#discussion_r178225930
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java
---
@@ -136,14 +138,28 @@ public void transferOut(VectorContainer containerOut)
{
public <T extends ValueVector> T addOrGet(final MaterializedField field,
final SchemaChangeCallBack callBack) {
final TypedFieldId id =
getValueVectorId(SchemaPath.getSimplePath(field.getName()));
final ValueVector vector;
- final Class<?> clazz =
TypeHelper.getValueVectorClass(field.getType().getMinorType(),
field.getType().getMode());
+
if (id != null) {
- vector = getValueAccessorById(id.getFieldIds()).getValueVector();
+ vector =
getValueAccessorById(id.getFieldIds()).getValueVector();
+ final Class<?> clazz =
TypeHelper.getValueVectorClass(field.getType().getMinorType(),
field.getType().getMode());
+
+ // Check whether incoming field and the current one are compatible;
if not then replace previous one with the new one
if (id.getFieldIds().length == 1 && clazz != null &&
!clazz.isAssignableFrom(vector.getClass())) {
final ValueVector newVector = TypeHelper.getNewVector(field,
this.getAllocator(), callBack);
replace(vector, newVector);
return (T) newVector;
}
+
+ // At this point, we know incoming and current fields are
compatible. Maps can have children,
+ // we need to ensure they have the same structure.
+ if (MinorType.MAP.equals(field.getType().getMinorType())
+ && vector != null
+ &&
!SchemaUtil.isSameSchemaIncludingOrder(vector.getField().getChildren(),
field.getChildren())) {
+
+ final ValueVector newVector = TypeHelper.getNewVector(field,
this.getAllocator(), callBack);
+ replace(vector, newVector);
+ return (T) newVector;
--- End diff --
We have two vectors, both maps. We found out their schemas differ. We are
throwing away the old map, replacing it with a new one with no members. Is
that what we want to do? Or, do we want to recursively merge the maps? Is this
done elsewhere? If so, how do we remember to do that merge?
> Drill fails on Schema changes
> ------------------------------
>
> Key: DRILL-6223
> URL: https://issues.apache.org/jira/browse/DRILL-6223
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.10.0, 1.12.0
> Reporter: salim achouche
> Assignee: salim achouche
> Priority: Major
> Fix For: 1.14.0
>
>
> Drill Query Failing when selecting all columns from a Complex Nested Data
> File (Parquet) Set). There are differences in Schema among the files:
> * The Parquet files exhibit differences both at the first level and within
> nested data types
> * A select * will not cause an exception but using a limit clause will
> * Note also this issue seems to happen only when multiple Drillbit minor
> fragments are involved (concurrency higher than one)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
