[
https://issues.apache.org/jira/browse/FLINK-25581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477868#comment-17477868
]
Leonid Ilyevsky commented on FLINK-25581:
-----------------------------------------
[~chesnay] Yes, I found something interesting.
This morning I checked the storage directory again, and I saw files there, so
it looks like it does archive jobs, but not all of them. I could tell because I
did increase the jobstore timeout to 24 hours, and I saw that some completed
jobs certainly not archived.
Then I checked the jobmanager logs (I have 5 instances). In one of them I found
exceptions related to the archiving. See the attached fragment in error.log.
This is just one example, there are more errors like that in the log,
complaining that the file already exists. The file indeed exists, I checked.
But other jobs completed around that time were not archived.
> Jobmanager does not archive completed jobs
> ------------------------------------------
>
> Key: FLINK-25581
> URL: https://issues.apache.org/jira/browse/FLINK-25581
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.13.5
> Environment: RHEL 7
> Reporter: Leonid Ilyevsky
> Priority: Major
> Attachments: error.log
>
>
> Jobmanager does not archive completed jobs.
> I configured the upload directory like this:
> jobmanager.archive.fs.dir: file:///liquidnet/shared/flink/completed-jobs
>
> After the job was completed, nothing appeared in that directory.
> The job info was visible in the jobmanager console for one hour, then it
> disappeared, and still there was no files in the configured directory.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
