One last question: running multiple jobs mean that each one has its own JobManager, right?
On Mon, Mar 7, 2016 at 3:14 PM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > Good, thank you for the explanation! > > On Mon, Mar 7, 2016 at 2:38 PM, Maximilian Michels <m...@apache.org> wrote: > >> Hi Stefano, >> >> Essentially the Yarn Session is not much different from a per-job Yarn >> cluster. In either case, a Flink cluster is brought up with resources >> provided by Yarn. In case of the Yarn session this cluster doesn't do >> anything until a job is submitted. In case of the per-job Yarn >> cluster, a job is immediately submitted after startup and the cluster >> is shutdown once that job has been completed. That's all. >> >> We're currently working on integrating proper resource allocation into >> the JobManager. As of now, everything is static, i.e. the JobManager >> won't allocate more than the initial requested resources. >> >> Cheers, >> Max >> >> On Mon, Mar 7, 2016 at 1:38 PM, Stefano Baghino >> <stefano.bagh...@radicalbit.io> wrote: >> > Hello everybody, >> > >> > I'm currently studying how the Flink/YARN integration works. Right now >> I'm a >> > little confused about the practical difference in having a long-running >> > session on which one deploys several jobs or deploying these jobs >> > individually. >> > >> > My intuition (which may not be correct) is that in the former ase a >> pool of >> > resources is allocated to the session and then the usage of these is >> handled >> > by the Flink JobManager, while in the latter case the resource >> allocation is >> > handled directly by YARN on a per-job basis. Am I right? >> > >> > If what I said is (more or less) right and apart from security concerns >> > (which have been discussed in a previous thread), are there any further >> > practical differences between having a long-running session or letting >> YARN >> > handle jobs? >> > >> > Thank you in advance! >> > >> > -- >> > BR, >> > Stefano Baghino >> > >> > Software Engineer @ Radicalbit >> > > > > -- > BR, > Stefano Baghino > > Software Engineer @ Radicalbit > -- BR, Stefano Baghino Software Engineer @ Radicalbit