Tez uses the standard yarn shuffle service (per node) that comes packaged with YARN.
-----Original Message----- From: Andrew Or [mailto:andrewo...@gmail.com] Sent: Thursday, September 18, 2014 5:25 PM To: dev@tez.apache.org Subject: General question about Tez shuffle Hi all, I have a general question of how Tez shuffle works on a yarn cluster. I am familiar with how Yarn shuffle uses ShuffleHandler as an auxiliary service of the node manager to serve spilled files. Does Tez use a similar interface? I am running an HDP 2.1 cluster with Tez installed, and I noticed that my "yarn.nodemanager.aux-services" does not include anything related to Tez. Does Tez run its own HTTP server to serve these files? If so, are these servers per-node or per-application? Thanks a lot in advance, Andrew -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
