Hi, Rex, Can you share more logs for it. Did you see something like "The configuration option taskmanager.cpu.cores required for local execution is not set, setting it to" in your logs?
Best, Yangze Guo Best, Yangze Guo On Sat, Dec 5, 2020 at 6:53 PM David Anderson <dander...@apache.org> wrote: > > taskmanager.cpu.cores is intended for internal use only -- you aren't meant > to set this option. What happens if you leave it alone? > > Regards, > David > > > On Sat, Dec 5, 2020 at 8:04 AM Rex Fenley <r...@remind101.com> wrote: >> >> We're running this in a local environment so that may be contributing to >> what we're seeing. >> >> On Fri, Dec 4, 2020 at 10:41 PM Rex Fenley <r...@remind101.com> wrote: >>> >>> Hello, >>> >>> I'm tuning flink for parallelism right now and when I look at the >>> JobManager I see >>> taskmanager.cpu.cores1.7976931348623157E308 >>> Which looks like the maximum double number. >>> >>> We have 8 cpu cores, so we figured we'd bump to 16 for hyper threading. We >>> have 37 operators so we rounded up and set 40 task slots. >>> >>> Here is our configuration >>> >>> "vmArgs": "-Xmx16g -Xms16g -XX:MaxDirectMemorySize=1207959552 >>> -XX:MaxMetaspaceSize=268435456 -Dlog.file=/tmp/flink.log >>> -Dtaskmanager.memory.framework.off-heap.size=134217728b >>> -Dtaskmanager.memory.network.max=1073741824b >>> -Dtaskmanager.memory.network.min=1073741824b >>> -Dtaskmanager.memory.framework.heap.size=134217728b >>> -Dtaskmanager.memory.managed.size=6335076856b >>> -Dtaskmanager.memory.task.heap.size=8160437768b >>> -Dtaskmanager.memory.task.off-heap.size=0b >>> -Dtaskmanager.numberOfTaskSlots=40 -Dtaskmanager.cpu.cores=16.0" >>> >>> We then tried with -Dtaskmanager.cpu.cores=7.0 and still ended up with that >>> very odd value for cpu cores. >>> >>> How do we correctly adjust this? >>> >>> Thanks! >>> -- >>> >>> Rex Fenley | Software Engineer - Mobile and Backend >>> >>> >>> Remind.com | BLOG | FOLLOW US | LIKE US >> >> >> >> -- >> >> Rex Fenley | Software Engineer - Mobile and Backend >> >> >> Remind.com | BLOG | FOLLOW US | LIKE US
