[
https://issues.apache.org/jira/browse/DRILL-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16545430#comment-16545430
]
ASF GitHub Bot commented on DRILL-6585:
---------------------------------------
sohami commented on issue #1367: DRILL-6585: PartitionSender clones vectors,
but shares field metdata
URL: https://github.com/apache/drill/pull/1367#issuecomment-405302803
> The reason for that change is that the result set loader code that clones
a vector needs to know the actual type. That code walks the vector tree, using
the MaterializedField to get the type. If a values vector (which has no bits
vector) reports its type as Nullable, then the clone will create a bits vector,
which causes havoc.
As of now NullableVectors takes care of creating it's internal `values` and
`bits` vector correctly. That is to say it creates correct `values` vector of
required type even though the MaterializedField of that vector points to parent
field.
When you traverse the MaterializedField tree to construct a new vector based
on it, for Nullable Vector without your change will not have any children
fields. So probably based on DataMode of Materialized Field you will know that
the vector is of Nullable Type and you will create a NullableValueVector.
Internally this NullableValueVector should create correct `values` vector (i.e.
without bit vector) even though it's field will be same as parent
NullableVector field. I don't think you have to create `values` vector
separately ? May be I am missing something, would be helpful if you can point
me to cloning code inside ResultSetLoader.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> PartitionSender clones vectors, but shares field metdata
> --------------------------------------------------------
>
> Key: DRILL-6585
> URL: https://issues.apache.org/jira/browse/DRILL-6585
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.13.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Major
>
> See the discussion for [PR #1244 for
> DRILL-6373|https://github.com/apache/drill/pull/1244].
> The PartitionSender clones vectors. But, it does so by reusing the
> {{MaterializedField}} from the original vector. Though the original authors
> of {{MaterializedField}} apparently meant it to be immutable, later changes
> for maps and unions ended up changing it to add members.
> When cloning a map, we get the original map materialized field, then start
> doctoring it up as we add the cloned map members. This screws up the original
> map vector's metadata.
> The solution is to clone an empty version of the materialized field when
> creating a new vector.
> But, since much code creates vectors by giving a perfectly valid, unique
> materialized field, we want to add a new method for use by the ill-behaved
> uses, such as PartitionSender, that ask to create a new vector without
> cloning the materialized field.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
