[
https://issues.apache.org/jira/browse/FLINK-36540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-36540:
-----------------------------------
Labels: pull-request-available (was: )
> Add Support for Hadoop Caller Context when using Flink on yarn
> --------------------------------------------------------------
>
> Key: FLINK-36540
> URL: https://issues.apache.org/jira/browse/FLINK-36540
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Task
> Environment: Hadoop 3.3.4
> Flink 1.14.4
> Reporter: liang yu
> Priority: Major
> Labels: pull-request-available
>
> *Description:* As HDFS mentioned in HDFS-9184, for a given HDFS operation,
> it's
> very helpful to track which upper level job issues it. But now Flink does not
> support to set callerContext when trying to operate on Hadoop fileSystem.
>
> *implementation:* Add a new HadoopOption which can be set in flink.yaml. By
> setting this option to true, flink can search for the classpath if we can
> invoke the method
> {code:java}
> callerContext.Builder().build().setCurrent(context){code}
> and then set the jobId and TaskName with subtask id so that we can find
> exactly who did the operation to Hadoop file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
