RE: YARN : Different cutoff for job and task managers
This works; I had some issues modifying my scripts, but it’s OK and I could confirm by JMX that env.java.opts.jobmanager had priority over the “normal” heap size (calculated from cutoff). Thanks ! From: Yang Wang Sent: mercredi 20 novembre 2019 03:52 To: Gwenhael Pasquiers Cc: user@flink.apache.org Subject: Re: YARN : Different cutoff for job and task managers Hi Gwenhael, I'm afraid that we could not set different cut-off to jobmanager and taskmanager. You could set the jvm args manually to work around. For example, 'env.java.opts.jobmanager=-Xms3072m -Xmx3072m'. In most jvm implementation, the rightmost Xmx Xms will take effect. So i think it should work. Please have a try. Best, Yang Gwenhael Pasquiers mailto:gwenhael.pasqui...@ericsson.com>> 于2019年11月19日周二 下午10:56写道： Hello, In a setup where we allocate most of the memory to rocksdb (off-heap) we ha= ve an important cutoff. Our issue is that the same cutoff applies to both task and job managers : the heap size of the job manager then becomes too low. Is there a way to apply different cutoffs to job and task managers ? Regards,
RE: YARN : Different cutoff for job and task managers
I see, good idea, I’ll try that and tell you the result. Thanks, From: Yang Wang Sent: mercredi 20 novembre 2019 03:52 To: Gwenhael Pasquiers Cc: user@flink.apache.org Subject: Re: YARN : Different cutoff for job and task managers Hi Gwenhael, I'm afraid that we could not set different cut-off to jobmanager and taskmanager. You could set the jvm args manually to work around. For example, 'env.java.opts.jobmanager=-Xms3072m -Xmx3072m'. In most jvm implementation, the rightmost Xmx Xms will take effect. So i think it should work. Please have a try. Best, Yang Gwenhael Pasquiers mailto:gwenhael.pasqui...@ericsson.com>> 于2019年11月19日周二 下午10:56写道： Hello, In a setup where we allocate most of the memory to rocksdb (off-heap) we ha= ve an important cutoff. Our issue is that the same cutoff applies to both task and job managers : the heap size of the job manager then becomes too low. Is there a way to apply different cutoffs to job and task managers ? Regards,
Re: YARN : Different cutoff for job and task managers
Hi Gwenhael, I'm afraid that we could not set different cut-off to jobmanager and taskmanager. You could set the jvm args manually to work around. For example, 'env.java.opts.jobmanager=-Xms3072m -Xmx3072m'. In most jvm implementation, the rightmost Xmx Xms will take effect. So i think it should work. Please have a try. Best, Yang Gwenhael Pasquiers 于2019年11月19日周二 下午10:56写道： > Hello, > > > > In a setup where we allocate most of the memory to rocksdb (off-heap) we > ha= ve an important cutoff. > > > > Our issue is that the same cutoff applies to both task and job managers : > the heap size of the job manager then becomes too low. > > > > Is there a way to apply different cutoffs to job and task managers ? > > > > Regards, >
YARN : Different cutoff for job and task managers
Hello, In a setup where we allocate most of the memory to rocksdb (off-heap) we ha= ve an important cutoff. Our issue is that the same cutoff applies to both task and job managers : the heap size of the job manager then becomes too low. Is there a way to apply different cutoffs to job and task managers ? Regards,
