[
https://issues.apache.org/jira/browse/FLINK-16666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073593#comment-17073593
]
Dian Fu commented on FLINK-16666:
---------------------------------
Hi [~aljoscha], as explained by [~zhongwei], the requirement is that we need to
process the Python dependency management related config options and register
the dependencies to the distributed cache to make sure that the dependencies
could be accessed during execution. In the `configure` method of
ExecutionEnvironment / StreamExecutionEnvironment, it will perform some
pre-processing according to the configuration object, e.g. setup the execution
config and checkpoint config according to the give configuration object. So
conceptually, this seems a good place for this requirement, e.g. register the
Python dependencies to the distributed cache according to the given
configuration object. Besides, it will also avoid code duplication as this
requirements not only apply for PyFlink Table API, but also for any other kinds
of API, such as PyFlink DataStream API, etc which maybe introduced in the near
future.
What's your thought?
> Support new Python dependency configuration options in flink-java,
> flink-streaming-java and flink-table
> -------------------------------------------------------------------------------------------------------
>
> Key: FLINK-16666
> URL: https://issues.apache.org/jira/browse/FLINK-16666
> Project: Flink
> Issue Type: Sub-task
> Components: API / Python
> Reporter: Wei Zhong
> Assignee: Wei Zhong
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
