[
https://issues.apache.org/jira/browse/BEAM-6955?focusedWorklogId=262510&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-262510
]
ASF GitHub Bot logged work on BEAM-6955:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Jun/19 17:30
Start Date: 18/Jun/19 17:30
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #8885: [BEAM-6955]
Use base version component of Beam Python SDK version when choosing Dataflow
container image to use.
URL: https://github.com/apache/beam/pull/8885
When Beam SDK version is modified, for example, when current version of the
SDK is a release candidate, Dataflow runner should use the base version
component of the SDK, which we can parse via `pkg_resources.parse_version()`,
for example:
```
>>> pkg_resources.parse_version("2.14.0").base_version
'2.14.0'
>>> pkg_resources.parse_version("2.14.0.rc1").base_version
'2.14.0'
```
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python3_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
<br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/)
| --- | ---
See
[.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md)
for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 262510)
Time Spent: 40m (was: 0.5h)
> Support Dataflow --sdk_location with modified version number
> ------------------------------------------------------------
>
> Key: BEAM-6955
> URL: https://issues.apache.org/jira/browse/BEAM-6955
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Affects Versions: 2.11.0
> Reporter: Daniel Lescohier
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Support Dataflow --sdk_location with modified version number
> Determine the version tag to use for the Google Container Registry, for the
> service image versions to use on the Dataflow worker nodes. Users of Dataflow
> may be using a locally-modified version of Apache Beam, which they submit to
> Dataflow with the --sdk_location option. Those users would most likely modify
> the version number of Apache Beam, so they can distinguish it from the public
> distribution of Apache Beam. However, the remote nodes in Dataflow still need
> to bootsrap the worker service with a Docker image that a version tag exists
> for.
> The most appropriate way for system integrators to modify the Apache Beam
> version number would be to add a Local Version Identifier:
> https://www.python.org/dev/peps/pep-0440/#local-version-identifiers
> If people only use Local Version Identifiers, then we could use the "public"
> attribute of the pkg_resources version object.
> If people instead use a post-release version identifier:
> https://www.python.org/dev/peps/pep-0440/#post-releases then only the
> "base_version" attribute would work both of these version number changes.
> Since Dataflow documentation does not specify how to modify version numbers,
> I am choosing to use "base_version" attribute.
> Will shortly submit a PR with the change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
