[
https://issues.apache.org/jira/browse/FLINK-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839961#comment-15839961
]
Geoffrey Mon edited comment on FLINK-5183 at 1/26/17 5:21 PM:
--------------------------------------------------------------
I'm currently working on this at
https://github.com/geofbot/flink/tree/FLINK-5183. So far I've managed to get it
to work except for a few issues:
* -Execution environments need to be able to get a unique ID that can be used
to identify it between Java and Python. At the moment, these IDs are manually
assigned.-
** I've achieved this using global variables in the Environment module of the
Python API.
* PythonPlanBinder does not exit when Python process exits, but instead waits
indefinitely for more jobs
** Although there is a check in a while loop to continue waiting for more
Python jobs until the Python process ends, there is a race condition where the
Python process will usually end right after Java checks if it is running
(unless you pause it in a debugger, giving time for Python to exit). This
causes Java to expect more jobs and wait indefinitely.
* Global variables used to differentiate between execution environments and to
run operators in Python for skipping over execution environments that do not
contain the operator to be run (there should be a better solution)
was (Author: hydronium):
I'm currently working on this at
https://github.com/geofbot/flink/tree/FLINK-5183. So far I've managed to get it
to work except for a few issues:
* Execution environments need to be able to get a unique ID that can be used to
identify it between Java and Python. At the moment, these IDs are manually
assigned.
* PythonPlanBinder does not exit when Python process exits, but instead waits
indefinitely for more jobs
** Although there is a check in a while loop to continue waiting for more
Python jobs until the Python process ends, there is a race condition where the
Python process will usually end right after Java checks if it is running
(unless you pause it in a debugger, giving time for Python to exit). This
causes Java to expect more jobs and wait indefinitely.
* Global variables used to run operators in Python for skipping over execution
environments that do not contain the operator to be run (there should be a
better solution)
> [py] Support multiple jobs per Python plan file
> -----------------------------------------------
>
> Key: FLINK-5183
> URL: https://issues.apache.org/jira/browse/FLINK-5183
> Project: Flink
> Issue Type: Improvement
> Components: Python API
> Affects Versions: 1.1.3
> Reporter: Geoffrey Mon
> Priority: Minor
>
> Support running multiple jobs per Python plan file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
