[
https://issues.apache.org/jira/browse/HIVE-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662319#comment-14662319
]
Sergey Shelukhin commented on HIVE-11355:
-----------------------------------------
in GBO - why isn't it checking for flushes anymore if memoryNeeded is specified?
createEdgeProperty nit - memoryManagerEnabled is only used in one case block
but it is retrieved for all of them.
dummyStoreOp.setConf(new DummyStoreDesc()); - what is this for?
{noformat}
Invalid configuration could cause OutOfMemory issues at runtime. Configuration "
+ HiveConf.ConfVars.HIVECONVERTJOINNOCONDITIONALTASKTHRESHOLD + " =
"
+ mapJoinsTotalAvailableMemory + " is greater than configured
container size: "
+ totalAvailableMemory + ". Check your tez/yarn settings");
{noformat}
Will this happen to many people? When the threshold is set to some arbitrary
high value to force map join, this may happen, even though this value will not
actually be the table size in reality. IIRC I saw it in some tests.
To be continued
> Hive on tez: memory manager for sort buffers (input/output) and operators
> -------------------------------------------------------------------------
>
> Key: HIVE-11355
> URL: https://issues.apache.org/jira/browse/HIVE-11355
> Project: Hive
> Issue Type: Improvement
> Components: Tez
> Affects Versions: 2.0.0
> Reporter: Vikram Dixit K
> Assignee: Vikram Dixit K
> Attachments: HIVE-11355.1.patch, HIVE-11355.2.patch,
> HIVE-11355.3.patch
>
>
> We need to better manage the sort buffer allocations to ensure better
> performance. Also, we need to provide configurations to certain operators to
> stay within memory limits.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
