[
https://issues.apache.org/jira/browse/FLINK-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-10434:
-----------------------------------
Component/s: Runtime / Coordination
> Blob file not removed after job execution.
> ------------------------------------------
>
> Key: FLINK-10434
> URL: https://issues.apache.org/jira/browse/FLINK-10434
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.5.2
> Environment: Flink container running on Yarn.
> Reporter: Piotr Szczepanek
> Priority: Major
>
> We have Flink container running on Yarn, and we're using YarnClusterClient
> for job submission. After successfully/failed job execution it looks like
> blob file for that job is deleted, but there is still handle from Flink
> process to that file. As a result the file is
> not removed from machine, until we restart the whole Flink container.
> From the size comparison it looks like mentioned file is actually our jar
> with Flink job.
> This is quite big problem for us, as we are submitting many jobs, every
> submission upload new jar and created new blob file, which is never removed
> from disc until we restart container. We already faced out of disc space.
> Results of lsof are:
> During job execution:
> lsof /flinkDir | grep job_dbafb671b0d60ed8a8ec2651fe59303b
> java 11883 yarn mem REG 253,2 112384928 109973177
> /flinkDir/yarn/../application_1536668870638_5555/blobStore-a1bcdbd4-5388-4c56-8052-6051f5af38dd/job_dbafb671b0d60ed8a8ec2651fe59303b/blob_p-8771d9ccac35e28d8571ac8957feaaecdebaeadd-7748aec7fe7369ca26181d0f94b1a578
> java 11883 yarn 1837r REG 253,2 112384928 109973177
> /flinkDir/yarn/../application_1536668870638_5555/blobStore-a1bcdbd4-5388-4c56-8052-6051f5af38dd/job_dbafb671b0d60ed8a8ec2651fe59303b/blob_p-8771d9ccac35e28d8571ac8957feaaecdebaeadd-7748aec7fe7369ca26181d0f94b1a578
> After job execution:
> lsof /flinkDir | grep job_dbafb671b0d60ed8a8ec2651fe59303b
> java 11883 yarn DEL REG 253,2 109973177
> /flinkDir/yarn/../application_1536668870638_5555/blobStore-a1bcdbd4-5388-4c56-8052-6051f5af38dd/job_dbafb671b0d60ed8a8ec2651fe59303b/blob_p-8771d9ccac35e28d8571ac8957feaaecdebaeadd-7748aec7fe7369ca26181d0f94b1a578
> java 11883 yarn 1837r REG 253,2 112384928 109973177
> /flinkDir/yarn/../application_1536668870638_5555/blobStore-a1bcdbd4-5388-4c56-8052-6051f5af38dd/job_dbafb671b0d60ed8a8ec2651fe59303b/blob_p-8771d9ccac35e28d8571ac8957feaaecdebaeadd-7748aec7fe7369ca26181d0f94b1a578
> *(deleted)*
> After restarting Flink container this handle disappeared.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
