[
https://issues.apache.org/jira/browse/BEAM-5334?focusedWorklogId=143000&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-143000
]
ASF GitHub Bot logged work on BEAM-5334:
----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Sep/18 09:12
Start Date: 11/Sep/18 09:12
Worklog Time Spent: 10m
Work Description: lgajowy closed pull request #6355: [BEAM-5334] Fix java
performance test - No Beam python setup for other language
URL: https://github.com/apache/beam/pull/6355
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.test-infra/jenkins/CommonJobProperties.groovy
b/.test-infra/jenkins/CommonJobProperties.groovy
index 081216e5aac..3fdbdba7e33 100644
--- a/.test-infra/jenkins/CommonJobProperties.groovy
+++ b/.test-infra/jenkins/CommonJobProperties.groovy
@@ -314,7 +314,7 @@ class CommonJobProperties {
}
// Adds the standard performance test job steps.
- static def buildPerformanceTest(def context, def argMap) {
+ static def buildPerformanceTest(def context, def argMap, def language =
"DEFAULT") {
def pkbArgs = genPerformanceArgs(argMap)
// Absolute path of project root and virtualenv path of Beam and Perfkit.
@@ -326,29 +326,30 @@ class CommonJobProperties {
context.steps {
// Clean up environment.
shell("rm -rf ${perfkit_root}")
- shell("rm -rf ${beam_env}")
shell("rm -rf ${perfkit_env}")
- // create new VirtualEnv, inherit already existing packages
- shell("virtualenv ${beam_env} --system-site-packages")
- shell("virtualenv ${perfkit_env} --system-site-packages")
+ // create new VirtualEnv
+ shell("virtualenv ${perfkit_env}")
// update setuptools and pip
- shell("${beam_env}/bin/pip install --upgrade setuptools pip
grpcio-tools==1.3.5")
shell("${perfkit_env}/bin/pip install --upgrade setuptools pip")
// Clone appropriate perfkit branch
shell("git clone
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git ${perfkit_root}")
- // Install job requirements for Python SDK.
- shell("${beam_env}/bin/pip install -e
${beam_root}/sdks/python/[gcp,test]")
-
- // Build PythonSDK tar ball.
- shell("(cd ${beam_root}/sdks/python && ${beam_env}/bin/python setup.py
sdist --dist-dir=target)")
-
// Install Perfkit benchmark requirements.
shell("${perfkit_env}/bin/pip install -r
${perfkit_root}/requirements.txt")
+ // Install Beam Python SDK requirements.
+ if (language == "PYTHON") {
+ shell("rm -rf ${beam_env}")
+ shell("virtualenv ${beam_env}")
+ shell("${beam_env}/bin/pip install --upgrade setuptools pip
grpcio-tools==1.3.5")
+ shell("${beam_env}/bin/pip install -e
${beam_root}/sdks/python/[gcp,test]")
+ // Build PythonSDK tar ball.
+ shell("(cd ${beam_root}/sdks/python && ${beam_env}/bin/python
setup.py sdist --dist-dir=target)")
+ }
+
// Launch performance test.
shell("${perfkit_env}/bin/python ${perfkit_root}/pkb.py ${pkbArgs}")
}
diff --git a/.test-infra/jenkins/job_PerformanceTests_Python.groovy
b/.test-infra/jenkins/job_PerformanceTests_Python.groovy
index 63b9cc95819..0b25f7098e6 100644
--- a/.test-infra/jenkins/job_PerformanceTests_Python.groovy
+++ b/.test-infra/jenkins/job_PerformanceTests_Python.groovy
@@ -55,5 +55,5 @@ job('beam_PerformanceTests_Python'){
beam_it_args : pipelineArgsJoined,
]
- commonJobProperties.buildPerformanceTest(delegate, argMap)
+ commonJobProperties.buildPerformanceTest(delegate, argMap, 'PYTHON')
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 143000)
Time Spent: 20m (was: 10m)
> [beam_PerformanceTests_Python] Incompatible packages in dependencies
> --------------------------------------------------------------------
>
> Key: BEAM-5334
> URL: https://issues.apache.org/jira/browse/BEAM-5334
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Mikhail Gryzykhin
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Failing job url:
> https://builds.apache.org/job/beam_PerformanceTests_Python/1390/console
> Relevant log:
> 05:18:36 Obtaining
> file:///home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Python/src/sdks/python
> 05:18:37 Complete output from command python setup.py egg_info:
> 05:18:37
> /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Python/env/.beam_env/local/lib/python2.7/site-packages/setuptools/dist.py:398:
> UserWarning: Normalizing '2.8.0.dev' to '2.8.0.dev0'
> 05:18:37 normalized_version,
> 05:18:37 running egg_info
> 05:18:37 Requirement already satisfied: future==0.16.0 in
> /home/jenkins/.local/lib/python2.7/site-packages (0.16.0)
> 05:18:37 gapic-google-cloud-pubsub-v1 0.15.4 has requirement
> oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is
> incompatible.
> 05:18:37 proto-google-cloud-datastore-v1 0.90.4 has requirement
> oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is
> incompatible.
> 05:18:37 proto-google-cloud-pubsub-v1 0.15.4 has requirement
> oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is
> incompatible.
> 05:18:37 googledatastore 7.0.1 has requirement httplib2<0.10,>=0.9.1, but
> you'll have httplib2 0.11.3 which is incompatible.
> 05:18:37 googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1,
> but you'll have oauth2client 4.1.2 which is incompatible.
> 05:18:37 You are using pip version 10.0.1, however version 18.0 is available.
> 05:18:37 You should consider upgrading via the 'pip install --upgrade pip'
> command.
> 05:18:37 error: [Errno 2] No such file or directory
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
