[
https://issues.apache.org/jira/browse/BEAM-4836?focusedWorklogId=127720&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-127720
]
ASF GitHub Bot logged work on BEAM-4836:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Jul/18 09:56
Start Date: 26/Jul/18 09:56
Worklog Time Spent: 10m
Work Description: lgajowy closed pull request #6056: [BEAM-4836 ]
separate virtualenvs for Beam and PerfKit
URL: https://github.com/apache/beam/pull/6056
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.test-infra/jenkins/CommonJobProperties.groovy
b/.test-infra/jenkins/CommonJobProperties.groovy
index 8ca6a4ed84a..99745f83b6a 100644
--- a/.test-infra/jenkins/CommonJobProperties.groovy
+++ b/.test-infra/jenkins/CommonJobProperties.groovy
@@ -260,6 +260,7 @@ class CommonJobProperties {
bigquery_table: 'beam_performance.pkb_results',
k8s_get_retry_count: 36, // wait up to 6 minutes for K8s LoadBalancer
k8s_get_wait_interval: 10,
+ python_binary: '$WORKSPACE/.beam_env/bin/python',
temp_dir: '$WORKSPACE',
// Use source cloned by Jenkins and not clone it second time
(redundantly).
beam_location: '$WORKSPACE/src',
@@ -304,25 +305,28 @@ class CommonJobProperties {
context.steps {
// Clean up environment.
shell('rm -rf PerfKitBenchmarker')
- shell('rm -rf .env')
+ shell('rm -rf .beam_env')
+ shell('rm -rf .perfkit_env')
// create new VirtualEnv, inherit already existing packages
- shell('virtualenv .env --system-site-packages')
+ shell('virtualenv .beam_env --system-site-packages')
+ shell('virtualenv .perfkit_env --system-site-packages')
// update setuptools and pip
- shell('.env/bin/pip install --upgrade setuptools pip')
+ shell('.beam_env/bin/pip install --upgrade setuptools pip')
+ shell('.perfkit_env/bin/pip install --upgrade setuptools pip')
// Clone appropriate perfkit branch
shell('git clone
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git')
// Install job requirements for Python SDK.
- shell('.env/bin/pip install -e ' + CommonJobProperties.checkoutDir +
'/sdks/python/[gcp,test]')
+ shell('.beam_env/bin/pip install -e ' +
CommonJobProperties.checkoutDir + '/sdks/python/[gcp,test]')
// Install Perfkit benchmark requirements.
- shell('.env/bin/pip install -r PerfKitBenchmarker/requirements.txt')
+ shell('.perfkit_env/bin/pip install -r
PerfKitBenchmarker/requirements.txt')
// Launch performance test.
- shell(".env/bin/python PerfKitBenchmarker/pkb.py $pkbArgs")
+ shell(".perfkit_env/bin/python PerfKitBenchmarker/pkb.py $pkbArgs")
}
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 127720)
Time Spent: 2h (was: 1h 50m)
> IOIT tests fails on Jenkins because of numpy version
> ----------------------------------------------------
>
> Key: BEAM-4836
> URL: https://issues.apache.org/jira/browse/BEAM-4836
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Kasia Kucharczyk
> Assignee: Kasia Kucharczyk
> Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Perfkit requires 1.13 numpy version. But probably because of
> [https://github.com/apache/beam/pull/5565] in beam there is numpy-1.14.
> Failure details (more details here
> [https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/452/] ):
> perfkitbenchmarker.errors.PythonPackageRequirementUnfulfilled: A Python
> package requirement was not met while checking
> "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_MongoDBIO_IT/PerfKitBenchmarker/requirements.txt":
> numpy 1.14.5 is installed but numpy==1.13.3 is required
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
