[
https://issues.apache.org/jira/browse/TRAFODION-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163915#comment-16163915
]
ASF GitHub Bot commented on TRAFODION-2733:
-------------------------------------------
Github user zellerh commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/1228#discussion_r138498231
--- Diff: core/sql/cli/Globals.cpp ---
@@ -1099,6 +1100,52 @@ void CliGlobals::deleteContexts()
}
#endif // _DEBUG
+// The unused BMO memory quota can now be utilized by the other
+// BMO instances from the same or different fragment
+// In case of ESP process, the unused memory quota is maintained
+// at the defaultContext. In case of master process, the ununsed
+// memory quota is maintained in the context of the connection
+
+NABoolean CliGlobals::grabMemoryQuotaIfAvailable(ULng32 size)
+{
+ ContextCli *context;
+ if (espProcess_)
+ context = defaultContext_;
+ else
+ context = currContext();
+ return context->grabMemoryQuotaIfAvailable(size);
+}
+
+void CliGlobals::resetMemoryQuota()
+{
+ ContextCli *context;
+ if (espProcess_)
+ context = defaultContext_;
+ else
+ context = currContext();
+ return context->resetMemoryQuota();
+}
+
+ULng32 CliGlobals::unusedMemoryQuota()
+{
+ ContextCli *context;
+ if (espProcess_)
+ context = defaultContext_;
+ else
+ context = currContext();
+ return context->unusedMemoryQuota();
+}
+
+void CliGlobals::yieldMemoryQuota(ULng32 size)
+{
+ ContextCli *context;
+ if (espProcess_)
+ context = defaultContext_;
+ else
+ context = currContext();
--- End diff --
Thanks, sounds ok, although I don't quite understand why we wouldn't treat
both master and ESP the same. Otherwise, when we switch between serial and
parallel plans we may see an unexpected change in behavior? This leads me to a
more general comment: I wonder whether it would be better to set an overall
limit per connection, not per node. I wonder whether this would be easier to
manage, both for users as well as for the SQL engine.
> Provide an improved memory quota assignment for big memory operators (BMO)
> --------------------------------------------------------------------------
>
> Key: TRAFODION-2733
> URL: https://issues.apache.org/jira/browse/TRAFODION-2733
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: sql-cmp, sql-exe
> Affects Versions: 2.3-incubating
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Fix For: 2.3-incubating
>
>
> The big memory operators in Trafodion are HashJoin, HashGroupBy and Sort.
> Trafodion deploys multiple executor server processes (ESPs) to execute a
> query via its data flow architecture. Each ESPs can have an instance of this
> BMO operator. Currently, each instance of this operator can potentially have
> memory quota of 800 MB assigned to do its BMO operation. However, the memory
> allocation is usually limited by the memory pressure when this BMO attempts
> to allocate memory within the assigned quota. The assignment doesn't depend
> upon the estimation of memory needed by this operation.
> Improvement needed in BMO memory assignment are:
> 1. Limit the memory quota assignment for these BMO operations per node
> 2. Memory quota assigned taking into consideration estimated memory needed
> at every operator.
> 3. Ensure that the BMO gets the minimum memory needed at least to function
> smoothly
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
