>From the Python point of view, I think it makes sense to configure the "shim" runner with either (1) a command to run to launch and tear down the service or (2) an already existing endpoint. We could make (1) more convenient by providing some ready-made commands in some namespace that use the existing containers and/or set up good defaults.
On Thu, Feb 8, 2018 at 4:23 PM, Kenneth Knowles (JIRA) <[email protected]> wrote: > > [ > https://issues.apache.org/jira/browse/BEAM-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357748#comment-16357748 > ] > > Kenneth Knowles commented on BEAM-2591: > --------------------------------------- > > I think probably best to get feedback on idiomatic uses from [~robertwb], > [~altay], [~jgrier], etc. > > Basically I imagine a user will > 1. test bounded smoke test with the Python direct runner > 2. try the same on a local flink (or whatever is a one-liner to launch and > teardown) and also some unbounded data at that point > 3. go big by: > a. point the pipeline at a production endpoint where they don't know or > care what tech is behind it > b. or launch their pipeline main alongside flink as a complete app via > k8s or a bespoke setup > c. ??? > > This ticket had been filed for 2 with the idea that it would be reusable for > 3 but actually I think it should just be split into separate tasks and this > focus on 2. > >> Python shim for submitting to FlinkRunner >> ----------------------------------------- >> >> Key: BEAM-2591 >> URL: https://issues.apache.org/jira/browse/BEAM-2591 >> Project: Beam >> Issue Type: Sub-task >> Components: runner-flink, sdk-py-core >> Reporter: Kenneth Knowles >> Priority: Major >> Labels: portability >> >> Whatever the result of https://s.apache.org/beam-job-api, Python users will >> need to be able to pass --runner=FlinkRunner and have it work. > > > > -- > This message was sent by Atlassian JIRA > (v7.6.3#76005)
