[
https://issues.apache.org/jira/browse/HIVE-24737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278959#comment-17278959
]
Panagiotis Garefalakis commented on HIVE-24737:
-----------------------------------------------
Hey David, seems like the obtainBufferSize values if afterward used as
[TEZ_RUNTIME_UNORDERED_OUTPUT_BUFFER_SIZE_MB|https://github.com/apache/tez/blob/bf4942b49a3d9c0d326f64715b3e76952b77382d/tez-runtime-library/src/main/java/org/apache/tez/runtime/library/api/TezRuntimeConfiguration.java#L203]
value.
I believe the conf is there in case the groupByOperator stats are completely
off -- creating an unreasonably large output buffer. Maybe we could remove and
have a large upper limit instead, not sure what that value could be though
(default is 100MB) -- any ideas here [~abstractdog] ?
> Remove Configuration TEZ_SIMPLE_CUSTOM_EDGE_TINY_BUFFER_SIZE_MB
> ---------------------------------------------------------------
>
> Key: HIVE-24737
> URL: https://issues.apache.org/jira/browse/HIVE-24737
> Project: Hive
> Issue Type: Improvement
> Reporter: David Mollitor
> Priority: Minor
>
> Please remove {{TEZ_SIMPLE_CUSTOM_EDGE_TINY_BUFFER_SIZE_MB}}
>
> It is never in practice actually used. Can it just be assigned a sensible
> hard-coded value?
>
> This seem like an over optimization at the cost of yet another configuration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
