GitHub user GEOFBOT opened a pull request:
https://github.com/apache/flink/pull/3232
[FLINK-5183] [py] Support mulitple jobs per plan file
Modifies the Python API to support running multiple jobs in one Python plan
file.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/GEOFBOT/flink FLINK-5183
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3232.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3232
----
commit e2f4a4acda8c180e69a72709dfd777786ac6650b
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-26T14:15:55Z
[FLINK-5183] [py] Support mulitple jobs per plan file
Issues to be resolved:
* Execution environments need to be able to get a unique ID that can be
used to identify it between languages. At the moment, these IDs are
manually assigned.
* PythonPlanBinder does not exit when Python process exits, waiting
indefinitely for more jobs
* Global variables used to run operators in Python (there should be a
better solution)
commit ee500403b62ebf9c8b923d5940aefb2f928a5c13
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-26T14:27:47Z
Remove debug print statements
commit afdeddc03f9817a74927a727733ef5c0025af0c7
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-26T17:04:23Z
Auto assign environment IDs; rm Python alive checks that didn't help
commit 5dac565d89a27cc113cd4abce7fbdd57485e4401
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-26T21:15:15Z
Set 5 second timeout for PythonPlanStreamer socket
After the timeout, we check to see if the Python process has exited or
not. This allows PythonPlanBinder to exit properly when the Python
process has exited.
commit e34522e54921989db6ab37ce3b134672b6980f67
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-26T22:26:03Z
Multiple jobs per Python plan file unit test
commit 64c013f8a9942b05f1b3000149a66e34b7b366b1
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-27T14:05:25Z
Fix support for operators using multiple data sets
commit 782f69ac1d17f8ff5812216c69fa4cfd119d5a60
Author: Geoffrey Mon <[email protected]>
Date: 2017-01-29T06:02:36Z
Create Python ExecutionEnvironment after checking if Python has job
This prevents ExecutionEnvironment#getLastJobExecutionResult from being
null.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
