Hi all, We are going to move to production with an 8 node Spark cluster. Request some help for below
We are running on YARN cluster manager.That means YARN is installed with SSH between the nodes. When we run a standalone Spark program with spark-submit, YARN initializes a resource manager followed by application master per application. This is allocated randomely with arbitrary port. So, would we be opening all ports in between the nodes in a production implementation ? Best, Passion
