[
https://issues.apache.org/jira/browse/BEAM-12836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-12836:
---------------------------------
Labels: (was: stale-P2)
> Flink JobServer should allow setting the default environment for the
> expansion service
> --------------------------------------------------------------------------------------
>
> Key: BEAM-12836
> URL: https://issues.apache.org/jira/browse/BEAM-12836
> Project: Beam
> Issue Type: Improvement
> Components: jobserver
> Reporter: Jeremy Lewi
> Priority: P3
>
> I think the
> [FlinkJobServer|https://github.com/apache/beam/blob/3a7b8e757d0c220b5838ca6a0805ee463b24f85d/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkJobServerDriver.java#L72]
> needs to expose command line arguments to allow configuring the default
> environment for the expansion service.
> IIUC the expansion service in the job server is needed when submitting jobs
> using the portable runner that have cross language transforms (e.g. a Python
> pipeline that uses KafkaIO which is a Java language transform). In this case,
> we may need to configure the expansion service to change the default
> environment (e.g. the docker image used for Java transforms).
> It looks like [https://github.com/apache/beam/pull/15082/files] added
> appropriate command line flags to the main entry point for a standalone
> expansion service. However, I don't believe this code path is used when
> running the expansion service as part of the job server.
> I believe the
> [FlinkJobServer|https://github.com/apache/beam/blob/3a7b8e757d0c220b5838ca6a0805ee463b24f85d/runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkJobServerDriver.java#L72]
> would need to take similar command line arguments and use them to configure
> the expansion service.
> More discussion in this
> [thread|https://lists.apache.org/thread.html/r3cf56464c157da840f46e777c1b3f307f65ec2cf6b2278728f2cd1e1%40%3Cuser.beam.apache.org%3E]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
