[
https://issues.apache.org/jira/browse/BEAM-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847968#comment-16847968
]
Valentyn Tymofieiev edited comment on BEAM-6955 at 5/24/19 11:56 PM:
---------------------------------------------------------------------
[~danl], please note that Dataflow service cannot guarantee compatibility
between Dataflow python workers and the modified SDK. Also I encourage you to
contribute your fixes upstream. Beam community has been producing releases
every 6 weeks, and you can expect your changes to be released within 3-9 weeks
once a PR is merged.
That said, we plan to make a very similar change to one you suggest, to support
pre-released versions of the SDK, for example 2.14.0rc2. The gist of the change
will be identical to PR-8189 but we need to add tests and warnings. If you
don't mind, I can take over this JIRA and CC you on my PR once it is ready.
What do you think?
was (Author: tvalentyn):
[~danl], please note that Dataflow service cannot guarantee compatibility
between Dataflow python workers and the modified SDK. Also I encourage you to
contribute your fixes upstream. Beam community has been producing releases
every 6 weeks, and you can expect your changes to be released withing 3-9 weeks
once a PR is merged.
That said, we plan to make a very similar change to one you suggest, to support
pre-released versions of the SDK, for example 2.14.0rc2. The gist of the change
will be identical to PR-8189 but we need to add tests and warnings. If you
don't mind, I can take over this JIRA and CC you on my PR once it is ready.
What do you think?
> Support Dataflow --sdk_location with modified version number
> ------------------------------------------------------------
>
> Key: BEAM-6955
> URL: https://issues.apache.org/jira/browse/BEAM-6955
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.11.0
> Reporter: Daniel Lescohier
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Support Dataflow --sdk_location with modified version number
> Determine the version tag to use for the Google Container Registry, for the
> service image versions to use on the Dataflow worker nodes. Users of Dataflow
> may be using a locally-modified version of Apache Beam, which they submit to
> Dataflow with the --sdk_location option. Those users would most likely modify
> the version number of Apache Beam, so they can distinguish it from the public
> distribution of Apache Beam. However, the remote nodes in Dataflow still need
> to bootsrap the worker service with a Docker image that a version tag exists
> for.
> The most appropriate way for system integrators to modify the Apache Beam
> version number would be to add a Local Version Identifier:
> https://www.python.org/dev/peps/pep-0440/#local-version-identifiers
> If people only use Local Version Identifiers, then we could use the "public"
> attribute of the pkg_resources version object.
> If people instead use a post-release version identifier:
> https://www.python.org/dev/peps/pep-0440/#post-releases then only the
> "base_version" attribute would work both of these version number changes.
> Since Dataflow documentation does not specify how to modify version numbers,
> I am choosing to use "base_version" attribute.
> Will shortly submit a PR with the change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
