[
https://issues.apache.org/jira/browse/DRILL-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384393#comment-16384393
]
ASF GitHub Bot commented on DRILL-6126:
---------------------------------------
Github user ppadma commented on a diff in the pull request:
https://github.com/apache/drill/pull/1125#discussion_r171999148
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchSizer.java
---
@@ -418,11 +438,13 @@ private void measureColumn(ValueVector v, String
prefix) {
netRowWidthCap50 += ! colSize.isVariableWidth ? colSize.estSize :
8 /* offset vector */ +
roundUpToPowerOf2(Math.min(colSize.estSize,50));
// above change 8 to 4 after DRILL-5446 is fixed
+
+ return colSize;
}
- private void expandMap(AbstractMapVector mapVector, String prefix) {
+ private void expandMap(ColumnSize colSize, AbstractMapVector mapVector,
String prefix) {
for (ValueVector vector : mapVector) {
- measureColumn(vector, prefix);
+ colSize.childColumnSizes.put(prefix + vector.getField().getName(),
measureColumn(vector, prefix));
--- End diff --
I made the change to maintain trees of columns for trees of maps. Any one
who wants to access the lower level columns, they get top level column and have
to walk through the tree.
> Allocate memory for value vectors upfront in flatten operator
> -------------------------------------------------------------
>
> Key: DRILL-6126
> URL: https://issues.apache.org/jira/browse/DRILL-6126
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Priority: Critical
> Fix For: 1.12.0
>
>
> With recent changes to control batch size for flatten operator, we figure out
> row count in the output batch based on memory. Since we know how many rows we
> are going to include in the batch, we can also allocate the memory needed
> upfront instead of starting with initial value (4096) and doubling, copying
> every time we need more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
