On Wed, Feb 10, 2016 at 5:20 PM, Steve Loughran <ste...@hortonworks.com> wrote:
> > On 10 Feb 2016, at 04:42, praveen S <mylogi...@gmail.com> wrote: > > Hi, > > I have 2 questions when running the spark jobs on yarn in client mode : > > 1) Where is the AM(application master) created : > > > in the cluster > > > A) is it created on the client where the job was submitted? i.e driver and > AM on the same client? > > no > > Or B) yarn decides where the the AM should be created? > > yes > > 2) Driver and AM run in different processes : is my assumption correct? > > > yes. the driver runs on your local system, which had better be close to > the cluster and stay up for the duration of the work > This is not correct. In yarn-cluster mode driver is what runs inside the application master and the node on which application master gets allocated is decided by yarn. In yarn-client mode, there is no application master and driver runs in the context of the same unix process as the spark-submit. > Regards, > Praveen > > >
