[
https://issues.apache.org/jira/browse/DRILL-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16492906#comment-16492906
]
ASF GitHub Bot commented on DRILL-6373:
---------------------------------------
paul-rogers commented on issue #1244: DRILL-6373: Refactor Result Set Loader
for Union, List support
URL: https://github.com/apache/drill/pull/1244#issuecomment-392594956
@Ben-Zvi, the stack trace indicates an error in `MaterializedField`, but
that file is not changed by this commit. Nor are any of the other classes in
the call stack.
The test is in an HBase test, which is a bit hard to set up. As you know, it
is not possible to run functional tests outside of MapR -- which is why only
MapR folks can do the actual commits.
The failure occurs inside the partitioned sender, generated code, when
attempting to clone a `MaterializedField` for a Map.
Looking at the code, the only way the `MaterializedField.clone()` method can
fail is if another thread is adding/removing members to the map while the
thread that failed is attempting to clone the map schema. I'm not familiar
enough with the partioned sender to track down the issue.
Looking at history, it appears we modified the partition sender back in
April. I wonder if that triggered this failure? Can someone who knows more
about that class check if there is some way that the same `MaterializedField`
is being used across threads, and mutated in one of them?
Otherwise, this may be a race condition; try running the test again and the
problem may go away. That may show that the problem is not in this particular
PR, but rather a pre-existing problem that requires a separate research effort.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Refactor the Result Set Loader to prepare for Union, List support
> -----------------------------------------------------------------
>
> Key: DRILL-6373
> URL: https://issues.apache.org/jira/browse/DRILL-6373
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.13.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
> Fix For: 1.14.0
>
>
> As the next step in merging the "batch sizing" enhancements, refactor the
> {{ResultSetLoader}} and related classes to prepare for Union and List
> support. This fix follows the refactoring of the column accessors for the
> same purpose. Actual Union and List support is to follow in a separate PR.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
