Hello Dharmesh, The tez staging dir is where scratch data is kept for the lifetime of the Tez session. i.e. data which can be deleted once the application completes. Staging data includes the following: - recovery logs used by the Tez AM for checkpointing state - Configs and/or dag plan payloads that are sent across to the AM via the staging dir.
This staging directory location is configurable and overridable by the upper layer application. In the case of Hive, Hive uses the scratch dir as the Tez staging dir for the lifetime of the Hive session. For the actual usage of the hive staging dir and scratch dir, I suggest trying out the user@hive mailing list. thanks — Hitesh > On Oct 31, 2016, at 2:41 PM, Dharmesh Kakadia <dhkaka...@gmail.com> wrote: > > Hi, > > I am trying to understand meaning and relation between following > configurations when running Hive on Tez. > > hive.exec.stagingdir > tez.staging-dir > hive.exec.scratchdir > > Thanks, > Dharmesh
