[
https://issues.apache.org/jira/browse/TRAFODION-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163913#comment-16163913
]
ASF GitHub Bot commented on TRAFODION-2733:
-------------------------------------------
Github user selvaganesang commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/1228#discussion_r138498212
--- Diff: core/sql/generator/GenPreCode.cpp ---
@@ -2154,34 +2154,10 @@ RelExpr * RelRoot::preCodeGen(Generator * generator,
// Compute the total available memory quota for BMOs
NADefaults &defs = ActiveSchemaDB()->getDefaults();
- // total per CPU
- double m = defs.getAsDouble(EXE_MEMORY_LIMIT_PER_CPU) * (1024*1024);
-
- // total memory usage for all nBMOs
- double m1 = (generator->getTotalNBMOsMemoryPerCPU()).value();
-
- // total memory limit for all BMOs
- double m2 = m-m1;
-
- double ratio =
- defs.getAsDouble(EXE_MEMORY_LIMIT_NONBMOS_PERCENT) / 100;
-
- if ( m2 < 0 ) {
- // EXE_MEMORY_LIMIT_PER_CPU is set too small, set the total
- // memory limit for BMOs to zero. When the memory quota for
- // each BMO is computed (via method
RelExpr::computeMemoryQuota()),
- // the lower-bound for each BMO will kick in and each will receive
- // a quota equal to the lower-bound value.
- m2 = 0;
- } else {
-
- // nBMOs use more memory than the portion, adjust m2 to
- // that of (1-ratio)*m
- if (m1 > m*ratio )
- m2 = m*(1-ratio);
- }
+ // total per node
+ double m = defs.getAsDouble(BMO_MEMORY_LIMIT_PER_NODE) * (1024*1024);
--- End diff --
Will do
> Provide an improved memory quota assignment for big memory operators (BMO)
> --------------------------------------------------------------------------
>
> Key: TRAFODION-2733
> URL: https://issues.apache.org/jira/browse/TRAFODION-2733
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp, sql-exe
> Affects Versions: 2.3-incubating
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Fix For: 2.3-incubating
>
>
> The big memory operators in Trafodion are HashJoin, HashGroupBy and Sort.
> Trafodion deploys multiple executor server processes (ESPs) to execute a
> query via its data flow architecture. Each ESPs can have an instance of this
> BMO operator. Currently, each instance of this operator can potentially have
> memory quota of 800 MB assigned to do its BMO operation. However, the memory
> allocation is usually limited by the memory pressure when this BMO attempts
> to allocate memory within the assigned quota. The assignment doesn't depend
> upon the estimation of memory needed by this operation.
> Improvement needed in BMO memory assignment are:
> 1. Limit the memory quota assignment for these BMO operations per node
> 2. Memory quota assigned taking into consideration estimated memory needed
> at every operator.
> 3. Ensure that the BMO gets the minimum memory needed at least to function
> smoothly
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
