[ https://issues.apache.org/jira/browse/DRILL-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625773#comment-15625773 ]
ASF GitHub Bot commented on DRILL-4823: --------------------------------------- Github user arina-ielchiieva commented on a diff in the pull request: https://github.com/apache/drill/pull/560#discussion_r85957731 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/interpreter/InterpreterEvaluator.java --- @@ -378,8 +388,14 @@ public ValueHolder visitDoubleConstant(ValueExpressions.DoubleExpression dExpr, } @Override - public ValueHolder visitQuotedStringConstant(ValueExpressions.QuotedString e, Integer value) throws RuntimeException { - return ValueHolderHelper.getVarCharHolder(getManagedBufferIfAvailable(), e.value); + public ValueHolder visitQuotedStringConstant(final ValueExpressions.QuotedString e, Integer value) throws RuntimeException { + return getConstantValueHolder(e.value, new Function<DrillBuf, ValueHolder>() { --- End diff -- @jinfengni, code does not produce the same keys for decimal and string. We pass different functions for each of them. Example: 1. return ValueHolderHelper.getDecimal38Holder(buffer, decExpr.getBigDecimal().toString()); 2. return ValueHolderHelper.getVarCharHolder(buffer, e.value); Only buffer is the same for string and decimal with the same value but this is acceptable. > Fix OOM while trying to prune partitions with reasonable data size > ------------------------------------------------------------------ > > Key: DRILL-4823 > URL: https://issues.apache.org/jira/browse/DRILL-4823 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill, Query Planning & Optimization > Affects Versions: 1.6.0 > Reporter: Arina Ielchiieva > Assignee: Arina Ielchiieva > Fix For: 1.9.0 > > > _Example query:_ > {code:sql} > select '/'||dir0||'/'||dir1||'/'||dir2||'/'||dir3 , count(*) > FROM dfs.`/path/to/parquet/files` > WHERE ('/'||dir0||'/'||dir1||'/'||dir2||'/'||dir3) IN ( > '/2015/11/30', > '//2015/09/01', > '/2015/09/02', > '/2015/09/03', > '/2015/09/04', > '/2015/09/09', > '/2016/03/30' > ) > group by '/'||dir0||'/'||dir1||'/'||dir2||'/'||dir3 > order by 1; > {code} > _Error:_ > OOM while trying to prune partitions: > {noformat} > org.apache.drill.exec.exception.OutOfMemoryException: Unable to allocate > buffer of size 256 due to memory limit. Current allocation: 5242880 > at > org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:216) > ~[drill-memory-base-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.ops.BufferManagerImpl.getManagedBuffer(BufferManagerImpl.java:60) > ~[drill-java-exec-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.ops.BufferManagerImpl.getManagedBuffer(BufferManagerImpl.java:56) > ~[drill-java-exec-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.ops.QueryContext.getManagedBuffer(QueryContext.java:241) > ~[drill-java-exec-1.6.0.jar:1.6.0] > at > org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.getManagedBufferIfAvailable(InterpreterEvaluator.java:158) > ~[drill-java-exec-1.6.0.jar:1.6.0] > {noformat} > _Cause:_ > Interpreter always asks for a new buffer to hold varchar/varbinary or decimal > constant values. That's why the memory size required would be proportion to # > of constant expressions multiplied by # of input rows (partition). This is > different from evaluation from run-time generated where constant expression > will be evaluated once and use only one buffer per value. > _Fix:_ > To use one buffer for each unique constant value in query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)