[jira] (TEZ-3244) Allow overlap of input and output memory when they are not concurrent

[Rohini Palaniswamy (JIRA)]

Title: Message Title

Rohini Palaniswamy commented on TEZ-3244   Re: Allow overlap of input and output memory when they are not concurrent ConfigurationScope.Vertex instead of AM? That is a good catch. When Jason Lowe did the patch initially, TEZ-3286 was not done and so all WeightedScaling configs were all Scope.AM. Not absolutely convinced about the Input special handling - should we fix TEZ-3195? We should fix TEZ-3195. But it is still good to have the Input special handling config. For cases like replicated join, were the input would be read and cached in memory the processor will also require more memory. In those cases giving more memory to inputs can cause OOM and so will have to be turned off. Rohini Palaniswamy - what value will Pig set for this? Hoping to put tez.task.scale.memory.input-output-concurrent=false in pig-default.properties in Pig 0.17. Will go for tez.task.scale.memory.non-concurrent-inputs-disable=false in a later release for specific vertices in code once TEZ-3195 is fixed. Jason Lowe, Should we rename tez.task.scale.memory.non-concurrent-inputs-disable to tez.task.scale.memory.non-concurrent-inputs.enabled to be similar to other Tez config names? Add Comment   This message was sent by Atlassian JIRA (v6.3.15#6346-sha1:dbc023d)