ODP: docker, error NoResourceAvailableException..

[Dominik Wosiński]

Hey, 
The problem is that your command does start Job Manager container, but it does 
not start the Task Manager . That is why you have 0 slots. Currently, the 
default numberOfTaskSlots is set to the number of CPUs avaialbe on the machine.

So, You generally can to do 2 things: 

1) Start Job Manager and 2 Task Managers. If you have Docker Compose available, 
you can paste this to your docker-compose.yml : 
services:
  jobmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6123"
    ports:
      - "8081:8081"
    command: jobmanager
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6121"
      - "6122"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

  taskmanager1:
    image: ${FLINK_DOCKER_IMAGE_NAME:-flink}
    expose:
      - "6190"
      - "6120"
    depends_on:
      - jobmanager
    command: taskmanager
    links:
      - "jobmanager:jobmanager"
    environment:
      - JOB_MANAGER_RPC_ADDRESS=jobmanager

This will give you 1 Job Manager and 2 Task Managers with one task slot each, 
so 2 Task slots in general.

2) You can deploy 1 Job Manager and 1 Task Manager.Then you need to modify 
flink-conf.yml and set the following setting : 

taskmanager.numberOfTaskSlots: 2

This will give you 2 Task Slots with only 1 Task Manager. But you will need to 
somehow override config in the container, possibly using : 
https://docs.docker.com/storage/volumes/

Regards,
Dominik.
Od: shyla deshpande
Wysłano: środa, 15 sierpnia 2018 07:23
Do: user
Temat: docker, error NoResourceAvailableException..

Hello all,

Trying to use docker as a single node flink cluster.

docker run --name flink_local -p 8081:8081 -t flink local

I submited a job to the cluster using the Web UI. The job failed. I see this 
error message in the docker logs.

org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: 
Could not allocate all requires slots within timeout of 300000 ms. Slots 
required: 2, slots allocated: 0

The Web UI, shows 0 taskmanagers and 0 task slots on the Flink dashboard.
How do I start the docker with 2 Task slots?

Appreciate any help.

Thanks