chamikaramj opened a new issue, #22859: URL: https://github.com/apache/beam/issues/22859
### What would you like to happen? Many Python transforms used by Beam pipelines need to use new dependencies. Such dependencies usually need to be available both at job submission as well as job execution (Python SDK harness containers). For cross-language Python transforms (for example, when used by a Java pipeline), we have following workarounds for doing this. For job submission: * Setup a custom virtual environment with correct dependencies and startup a manual expansion service. For job execution: * Use a custom Beam Python container that contains correct dependencies and specify that via the `sdkHarnessContainerImageOverrides` option. * Setting dependencies via an environment variable (TODO: add details and document). We should simplify this by allowing users of PythonExternalTransform class to specify new dependencies. ### Issue Priority Priority: 2 ### Issue Component Component: cross-language -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
