Hey Everyone, This I have a question somewhat related to SAMZA-109 and this line in run-class.sh: # Check if a max-heap size is specified. If not - set a 768M heap [[ $JAVA_OPTS != *-Xmx* ]] && JAVA_OPTS="$JAVA_OPTS -Xmx768M"
If I were to set the container.memory.mb for yarn to 4GB ( yarn.container.memory.mb = 4096) the above JAVA_OPTS check would cause this to be ignored right? I can understand why preventing the Heap from going crazy is important in these long running jobs but to me this might cause some confusion especially when trying to debug a thrashing GC and Java isn't committing the memory amount set in yarn.container.memory.mb. I was trying to follow why this was decided on in SAMZA-109 but nothing stuck out to me. Would it make more sense to just not specifying a max heap and letting Yarn kill the job if it goes over it's specified allotment and/or let the user explicitly set these opts? Thanks! - Jordan
