[
https://issues.apache.org/jira/browse/DRILL-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venki Korukanti updated DRILL-2088:
-----------------------------------
Attachment: 0001-DRILL-2088-ReAlloc-in-FixedWidthVector.setValueCount.patch
Attaching patch to fix the following issues:
1. In BitVector, find the valueCapacity from the buffer size (similar to other
fixed width vectors), not based on the allocation size requested. Currently we
store the valueCapacity in a member variable which is not transferred, causing
failures when setValueCount is called on vector that got the buffer transfer.
2. Change the FixedLengthVector.java template to generate setValueCount that
reallocs when requested valueCount falls beyond the available buffer capacity.
This is a problem for nullable vectors (which internally use fixed width
vectors).
Patch has tests for both changes.
> BitVector.Mutator.setValueCount() fails with invalid buffer access
> ------------------------------------------------------------------
>
> Key: DRILL-2088
> URL: https://issues.apache.org/jira/browse/DRILL-2088
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Venki Korukanti
> Assignee: Venki Korukanti
> Attachments:
> 0001-DRILL-2088-ReAlloc-in-FixedWidthVector.setValueCount.patch
>
>
> Run the following physical plan:
> {code}
> {
> head:{
> type:"APACHE_DRILL_PHYSICAL",
> version:"1",
> generator:{
> type:"manual"
> }
> },
> graph:[
> {
> @id:1,
> pop:"mock-scan",
> url: "http://apache.org";,
> entries:[
> {records: 100, types: [
> {name: "blue", type: "BIT", mode: "REQUIRED"},
> {name: "red", type: "BIGINT", mode: "REQUIRED"},
> {name: "green", type: "INT", mode: "OPTIONAL"}
> ]}
> ]
> },
> {
> @id:2,
> child: 1,
> pop:"selection-vector-remover"
> },
> {
> @id: 3,
> child: 2,
> pop: "screen"
> }
> ]
> }
> {code}
> It fails with:
> {code}
> Failure while running fragment., index: 0, length: 8192 (expected: range(0,
> 1024))
> {code}
> Couple of issues:
> 1. We don't transfer valueCapacity as part of TransferPair.transfer(). This
> causes the vector that got the buffers to have 0 as valueCapacity. During the
> realloc of vector that got the buffer, we rely on valueCapacity to decide
> whether to realloc or not.
> 2. When BitVector.Mutator.setValueCount() is called we try to reallocate the
> buffer as the valueCapacity is zero. As part of realloc, we allocate a buffer
> of size 1028bytes and try to copy the existing buffer. Problem is existing
> buffer has size more than 1028bytes and we fail in copying.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
