----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30910/ -----------------------------------------------------------
(Updated Feb. 12, 2015, 9:52 p.m.) Review request for drill and Steven Phillips. Changes ------- One issue observed is: For vectors that have offsets (nullable, variable length), we allocate extra 1 record for offset vector. If you give 1024 as record capacity, offset vector is allocated for 1025 which may result in memory allocation for 2048 (next 2^x) number as netty tries to allocate in chunks of 2^x. So it is better to give initial capacity as number (2^x) - 1. This causes 1 record in values vector to be not utilized, but it is better than the memory wasted in offset vector when record capacity given is 2^x number. Updating the PartitionSender to allocate 1023 or 511 records instead of earlier numbers 1024 and 512. Our current default vector allocation also has the similar problem. I will update it in next patch. Repository: drill-git Description ------- Please see the JIRA DRILL-2178 for details Diffs (updated) ----- exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 52a3868 exec/java-exec/src/main/codegen/templates/NullableValueVectors.java ba7c629 exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java d39040e exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java aa5b702 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/materialize/QueryWritableBatch.java cef4101 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java f09acaa exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java 4292c09 exec/java-exec/src/main/java/org/apache/drill/exec/record/FragmentWritableBatch.java d122311 exec/java-exec/src/main/java/org/apache/drill/exec/record/MaterializedField.java bcc226f exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java f6644bd exec/java-exec/src/main/java/org/apache/drill/exec/vector/ObjectVector.java 3c15db3 exec/java-exec/src/main/java/org/apache/drill/exec/vector/RepeatedVector.java b23ee02 exec/java-exec/src/main/java/org/apache/drill/exec/vector/ValueVector.java df6a486 exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java c5dc5ba exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java 3078f4e exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java 6dce363 exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java 2bed433 Diff: https://reviews.apache.org/r/30910/diff/ Testing ------- Thanks, Venki Korukanti
