Hi Borys,
I remember that another user reported a similar issue recently [1] –
attached
to the ticket you can find his log file. If I recall correctly, we concluded
that YARN returned the containers very quickly. At the time, Flink's debug
level logs were inconclusive because we did not log the
Hey guys,
thanks for the replies.
1. "Requesting new TaskExecutor" looks fine as it's exactly 32 as is jobs'
parallelism set.
The weird thing is that after those 32 containers requested and received we
have this "flood" of 'Received new container/Returning excess container`
(and as shown below
Hi Borys,
if possible the complete logs of the JM (DEBUG log level) would be helpful
to further debug the problem. Have there been recovery operations lately?
Cheers,
Till
On Sat, Oct 6, 2018 at 11:15 AM Gary Yao wrote:
> Hi Borys,
>
> To debug how many containers Flink is requesting, you can
Hi Borys,
To debug how many containers Flink is requesting, you can look out for the
log
statement below [1]:
Requesting new TaskExecutor container with resources [...]
If you need help debugging, can you attach the full JM logs (preferably on
DEBUG level)? Would it be possible for you to
Hey,
We're running Flink 1.5.2 (I know there's 1.5.4 and 1.6.1) on YARN for some
jobs we're processing. It's a "long running" container to which we're
submitting jobs - all jobs submitted to that container have got parallelism of
32 (to be precise: in this job there are 8 subtasks with
