[
https://issues.apache.org/jira/browse/DRILL-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069343#comment-16069343
]
ASF GitHub Bot commented on DRILL-5517:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/840#discussion_r124937740
--- Diff: exec/vector/src/main/codegen/templates/FixedValueVectors.java ---
@@ -31,68 +31,98 @@
import org.apache.drill.exec.util.DecimalUtility;
/**
- * ${minor.class} implements a vector of fixed width values. Elements in
the vector are accessed
- * by position, starting from the logical start of the vector. Values
should be pushed onto the
- * vector sequentially, but may be randomly accessed.
- * The width of each element is ${type.width} byte(s)
- * The equivalent Java primitive is '${minor.javaType!type.javaType}'
+ * ${minor.class} implements a vector of fixed width values. Elements in
the vector are accessed
+ * by position, starting from the logical start of the vector. Values
should be pushed onto the
+ * vector sequentially, but may be accessed randomly.
+ * <ul>
+ * <li>The width of each element is {@link #VALUE_WIDTH} (= ${type.width})
byte<#if type.width != 1>s</#if>.</li>
+ * <li>The equivalent Java primitive is
'${minor.javaType!type.javaType}'.</li>
+ * </ul>
*
* NB: this class is automatically generated from ${.template_name} and
ValueVectorTypes.tdd using FreeMarker.
*/
-public final class ${minor.class}Vector extends BaseDataValueVector
implements FixedWidthVector{
+public final class ${minor.class}Vector extends BaseDataValueVector
implements FixedWidthVector {
private static final org.slf4j.Logger logger =
org.slf4j.LoggerFactory.getLogger(${minor.class}Vector.class);
+ /**
+ * Width of each fixed-width value.
+ */
+
+ public static final int VALUE_WIDTH = ${type.width};
+
+ /**
+ * Maximum number of values that this fixed-width vector can hold
+ * and stay below the maximum vector size limit. This is the limit
+ * enforced when the vector is used to hold values in a repeated
+ * vector.
+ */
+
+ public static final int MAX_VALUE_COUNT = MAX_BUFFER_SIZE / VALUE_WIDTH;
+
+ /**
+ * Maximum number of values that this fixed-width vector can hold
+ * and stay below the maximum vector size limit and/or stay below
+ * the maximum row count. This is the limit enforced when the
+ * vector is used to hold scalar (required or nullable) values.
+ */
+
+ public static final int MAX_SCALAR_COUNT = Math.min(MAX_ROW_COUNT,
MAX_VALUE_COUNT);
--- End diff --
@Added comment. For me, Eclipse sometimes opens the wrong version of this
file, preventing it from finding the declaration of symbols, so the explanation
is not entirely unnecessary...
> Provide size-aware set operations in value vectors
> --------------------------------------------------
>
> Key: DRILL-5517
> URL: https://issues.apache.org/jira/browse/DRILL-5517
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.11.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Labels: ready-to-commit
> Fix For: 1.11.0
>
>
> DRILL-5211 describes a memory fragmentation issue in Drill. The resolution is
> to limit vector sizes to 16 MB (the size of Netty memory allocation "slabs.")
> Effort starts by providing "size-aware" set operations in value vectors which:
> * Operate as {{setSafe()}} while vectors are below 16 MB.
> * Throw a new, specific exception ({{VectorOverflowException}}) if setting
> the value (and growing the vector) would exceed the vector limit.
> The methods in value vectors then become the foundation on which we can
> construct size-aware record batch "writers."
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
