[
https://issues.apache.org/jira/browse/FLINK-28477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aitozi updated FLINK-28477:
---------------------------
Description:
In the {{HistoryServerArchiveFetcher}}, we only list the archives specified by
the historyserver.archive.fs.dir, but we have an extra hierarchy for the jobs,
like:
/base/job_platform_id1/jobid1
/base/job_platform_id1/jobid2
/base/job_platform_id2/jobid3
For this use case, we can not provide the full dir for history server to scan.
So, we want to set the dir {{/base}} to scan. Can we support the HistoryServer
to list the dir with max depth, I think it will make it more flexible for
different usage.
was:
In the {{HistoryServerArchiveFetcher}}, we only list the archives specified by
the historyserver.archive.fs.dir, but we have an extra hierarchy for the jobs,
like:
/base/job_platform_id1/jobid1
/base/job_platform_id1/jobid2
/base/job_platform_id2/jobid3
For this use case, we can not provide the full dir for history server to scan.
So, we want to set the dir {{/base}} to scan. Can we support the HistoryServer
to list the dir with max depth, I think it will make it more flexible with
different usage.
> Support to scan the history server dir with max depth
> -----------------------------------------------------
>
> Key: FLINK-28477
> URL: https://issues.apache.org/jira/browse/FLINK-28477
> Project: Flink
> Issue Type: Improvement
> Reporter: Aitozi
> Priority: Major
>
> In the {{HistoryServerArchiveFetcher}}, we only list the archives specified
> by the historyserver.archive.fs.dir, but we have an extra hierarchy for the
> jobs, like:
> /base/job_platform_id1/jobid1
> /base/job_platform_id1/jobid2
> /base/job_platform_id2/jobid3
> For this use case, we can not provide the full dir for history server to
> scan. So, we want to set the dir {{/base}} to scan. Can we support the
> HistoryServer to list the dir with max depth, I think it will make it more
> flexible for different usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
