[
https://issues.apache.org/jira/browse/FLINK-14319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-14319:
-----------------------------------
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Register user jar files in {Stream}ExecutionEnvironment
> --------------------------------------------------------
>
> Key: FLINK-14319
> URL: https://issues.apache.org/jira/browse/FLINK-14319
> Project: Flink
> Issue Type: New Feature
> Components: API / DataSet, API / DataStream
> Reporter: Leo Zhang
> Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I see that there are some use cases in which people want to implement
> applications based on loading external jars for now, not just SQL but also
> streaming ones. And the related API proposals have been issued in the
> sub-task FLINK-14055 under task FLINK-10232 Add a SQL DDL.
> To support this sub-task FLINK-14055 , we need the other new task for
> \{Stream}ExecutionEnvironment::registerUserJarFile() interface which will be
> addressed in this issue.
> Here is the plan.
> *Design*
> * Add interface
> {code:java}
> void registerUserJarFile(String jarFile)
> {code}
> into _StreamExecutionEnvironment_ ( in module flink-streaming-java). The
> affected classes are _StreamGraph_, _StreamGraphGenerator_,
> _StreamingJobGraphGenerator_ to support getting and setting a list of user
> jars. And all they are in module flink-streaming-java.
> * Add interface
> {code:java}
> void registerUserJarFile(String jarFile)
> {code}
> into _ExecutionEnvironment_ (in module flink-java). The affected classes is
> _Plan_, in module flink-core, to support getting and setting a list of user
> jars.
> * Add interface
> {code:java}
> void addUserJars(List<Path> userJars, JobGraph jobGraph)
> {code}
> into _JobGraphGenerator_ and add the user jars within the method _JobGraph_
> _compileJobGraph(OptimizedPlan program, JobID jobId)_ so that user jars can
> be shipped with user's program and submitted to cluster. _JobGraphGenerator_
> is in module flink-optimizer.
> * Add interface
> {code:java}
> void registerUserJarFile(String jarFile)
> {code}
> into \{Stream}ExecutionEnvironment (in module flink-scala and
> flink-streaming-scala) and just use the wrapped _javaEnv_ to achieve
> registration.
> *Testing*
> * One test case for adding local user jars both in the streaming and batch
> jobs. We need to process test classes into a jar before testing. For this
> purpose, we can add a goal in process-test-classes for this testing case in
> the pom file. The affected module is flink-tests.
> * Another test case for adding use jars in HDFS. The same idea with the
> previous one. The affected module is flink-fs-tests.
> * Note that python API is not included in this issue just as registering
> cached files. But we still need to modify some python test cases in order to
> avoid building error as lacking some methods declared in java. The affected
> files are
> _flink-python/pyflink/dataset/tests/test_execution_environment_completeness.py_
> and
> _flink-python/pyflink/datastream/tests/test_stream_execution_environment_completeness.py_.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
