Awesome, thanks! On Wed, Jan 1, 2020 at 6:17 PM Xintong Song <tonysong...@gmail.com> wrote:
> Hi Li, > > Regarding your questions: > > 1. Is there actually a way to pass in the heap size via arguments to >> taskmanager.sh? Is passing -Dtaskmanager.heap.size supposed to work? > > > No, '-Dtaskmanager.heap.size' is not supposed to work. The '-D' > configurations are only parsed after the JVM is started, while > 'taskmanager.heap.size' will be used for starting the JVM. > > 2. If not, is there a recommended way to set the heap size by >> environment, like environmental variables? > > > Yes, these is an equivalent environment variable FLINK_TM_HEAP. > > > 3. Also, the maximum heap size logged and -Xms and -Xmx is always a little >> smaller than the configured size (i.e. configuring 3000m results 2700m in >> the jvm arguments, 1024m results in 922), why is that? > > > The configuration key 'taskmanager.heap.size' is a bit inaccurate. The > config option actually also account for some off-heap memory, such as > network direct buffers and off-heap managed memory (if used). That's way > you see the java heap size is always slightly smaller than the configured > 'taskmanager.heap.size'. > > > Thank you~ > > Xintong Song > > > > On Wed, Jan 1, 2020 at 3:10 AM Li Peng <li.p...@doordash.com> wrote: > >> Hey folks, we've been running a k8 flink application, using the >> taskmanager.sh script and passing in the -Djobmanager.heap.size=9000m and >> -Dtaskmanager.heap.size=7000m as options to the script. I noticed from the >> logs, that the Maximum heap size logged completely ignores these arguments, >> and just sets the heap to the default of 922M. >> >> I tested setting the taskmanager.heap.size and jobmanager.heap.size >> manually in flink-conf.yaml, and it does work as expected (minus the heap >> being set a little lower than configured). But since we want the >> application to pick up different memory settings based on the environment >> (local/staging/prod/etc), setting it in flink-conf isn't ideal. >> >> So my questions are: >> >> 1. Is there actually a way to pass in the heap size via arguments to >> taskmanager.sh? Is passing -Dtaskmanager.heap.size supposed to work? >> 2. If not, is there a recommended way to set the heap size by >> environment, like environmental variables? >> 3. Also, the maximum heap size logged and -Xms and -Xmx is always a >> little smaller than the configured size (i.e. configuring 3000m results >> 2700m in the jvm arguments, 1024m results in 922), why is that? >> >> Thanks, and happy new year! >> Li >> >>
