Thank you guys for your answers. Really helpful. @Bikas: Nice that Tez has these APIs to define container size.
@Hitesh: Yeah, I think Hive is using a single size for all its vertices, at least this happens for all the queries I am running, unfortunately (or good for my scenario). Thanks, Robert On Tuesday, September 9, 2014 9:32 PM, Hitesh Shah <[email protected]> wrote: Hi Robert, From a Tez point of view, a user of the Tez APIs can define the container size on a per vertex basis. I believe currently, Hive, when using Tez, uses a single size for all its vertices. thanks — Hitesh On Sep 9, 2014, at 6:37 PM, Grandl Robert <[email protected]> wrote: > Hi guys, > > It seems the size of a container allocated to a task in Tez is static ? > Please correct me if I am wrong. > > Is there a way such that Tez is dynamically choosing the task container size > based on various things ? (like a map or reduce vertex, size of data to > process, etc.) > > At least for the tpcds/h queries I am running it looks containers are fixed > size. > > Thanks, > Robert
