Github user amansinha100 commented on the pull request:
https://github.com/apache/drill/pull/208#issuecomment-149123146
@mehant , yes the underlying issue is simple: the null bit vector for the
NullableDateVector is allocated at 4096 bytes (actually, the UInt1Vector is
used for this, hence the error message shows length: 1). The 4096 is based on
the static constant BaseValueVector.INITIAL_VALUE_ALLOCATION. Since set() does
not do any reallocation, once this limit is reached, attempt to write to this
vector causes IOBE. setSafe() does realloc()s. I am pretty sure I can
create a test case with > 4096 files partitioned on date column that will repro
this issue. I will need to work with QA/performance folks on creating this
test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
