> even that setting is not being applied after the hive shell is started and a > query is executed.
Are you increasing it or decreasing it? Tez will reuse existing larger containers, instead of releasing them - reducing the parameter has almost no effect without a session restart. Also that's rounded out by YARN internally by yarn.scheduler.minimum-allocation-mb, so the actual allocation will always be in multiples of yarn.scheduler.minimum-allocation-mb. This often causes confusion - when the YARN config says 1536, while the Hive config says 2048 it will default to 3072 for all values between 3072-1537. Point to note, if you configure the hive.tez.java.opts without the -Xmx parameter, tez-0.8+ will fill that in with 80% of the allocated container returned by YARN instead of using a fixed param. > Btw, what's the difference between hive.tez.container.size and > tez.task.resource.memory.mb? Should they always be the same? The hive parameter is the one encoded into the DAG and the tez parameter is the fall-back if it is -1. Cheers, Gopal
