[
https://issues.apache.org/jira/browse/BEAM-9797?focusedWorklogId=426850&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-426850
]
ASF GitHub Bot logged work on BEAM-9797:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Apr/20 04:09
Start Date: 24/Apr/20 04:09
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11502:
URL: https://github.com/apache/beam/pull/11502#discussion_r414277855
##########
File path: sdks/java/container/license_scripts/license_script.sh
##########
@@ -15,39 +15,33 @@
# limitations under the License.
set -e
+ROOT_DIR=$(pwd)
+CONTAINER_DIR="$ROOT_DIR/sdks/java/container"
+
# reports are generated at ~/beam/java_third_party_licenses
./gradlew generateLicenseReport --rerun-tasks
-# install packages needed for pull_licenses_java.py
-pip install "beautifulsoup4>=4.9.0,<5.0"
-pip install "future>=0.16.0,<1.0.0"
-pip install "pyyaml>=3.12,<6.0.0"
-pip install "tenacity>=5.0.2,<6.0"
+# pip install 'tox>=3.14.6,<3.15.0'
+pip install --no-cache-dir tox==3.11.1
Review comment:
I tried with virtureenv. It works fine locally, but fails with Jenkins.
**Try1:**
```
python3 -m venv env && source env/bin/activate
```
Error:
```
21:01:17 The virtual environment was not created successfully because
ensurepip is not
21:01:17 available. On Debian/Ubuntu systems, you need to install the
python3-venv
21:01:17 package using the following command.
21:01:17
21:01:17 apt-get install python3-venv
21:01:17
21:01:17 You may need to use sudo with that command. After installing the
python3-venv
21:01:17 package, recreate your virtual environment.
21:01:17
21:01:17 Failing command:
['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit/src/env/bin/python3',
'-Im', 'ensurepip', '--upgrade', '--default-pip']
```
It failed to create virtueenv and ran with default local python env, which
is py2.7.
**Try2:**
```
python3 -m venv env && source env/bin/activate || pip install virtualenv &&
virtualenv env && source env/bin/activate
```
Ok, this time, if virtueenv with py3 fails, create it with py2. Both
commands failed and ran at default py env.
Error:
[link](https://builds.apache.org/job/beam_PreCommit_Java_Commit/11004/consoleFull)
```
20:24:40 Using base prefix '/usr'
20:24:40 New python executable in
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit/src/env/bin/python3
20:24:41 Traceback (most recent call last):
20:24:41 File "/usr/local/bin/virtualenv", line 11, in <module>
20:24:41 sys.exit(main())
20:24:41 File "/usr/local/lib/python3.5/dist-packages/virtualenv.py", line
793, in main
20:24:41 symlink=options.symlink,
20:24:41 File "/usr/local/lib/python3.5/dist-packages/virtualenv.py", line
1071, in create_environment
20:24:41 install_python(home_dir, lib_dir, inc_dir, bin_dir,
site_packages=site_packages, clear=clear, symlink=symlink)
20:24:41 File "/usr/local/lib/python3.5/dist-packages/virtualenv.py", line
1457, in install_python
20:24:41 shutil.copyfile(executable, py_executable)
20:24:41 File "/usr/lib/python3.5/shutil.py", line 98, in copyfile
20:24:41 raise SameFileError("{!r} and {!r} are the same
file".format(src, dst))
20:24:41 shutil.SameFileError: '/usr/bin/python3' and
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit/src/env/bin/python3'
are the same file
```
**Try3:**
```
apt-get install python3-venv
python3 -m venv env && source env/bin/activate
```
In order to apt-get install, need to run with sudo.
**Try4:**
```
sudo apt-get install python3-venv
python3 -m venv env && source env/bin/activate
```
Added sudo, and it asks to input password and failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 426850)
Time Spent: 2h 40m (was: 2.5h)
> license_script.sh calls pip install/uninstall in local env
> ----------------------------------------------------------
>
> Key: BEAM-9797
> URL: https://issues.apache.org/jira/browse/BEAM-9797
> Project: Beam
> Issue Type: Bug
> Components: build-system
> Reporter: Udi Meiri
> Assignee: Hannah Jiang
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> File is:
> https://github.com/apache/beam/blob/master/sdks/java/container/license_scripts/license_script.sh
> The problem is with the code that does pip install and uninstall.
> 1. It is not okay to modify the local environment.
> 2. Running this script in parallel with itself (on Jenkins) has a chance to
> cause a race.
> The solution is to use a tox environment to run this script in. Tox will take
> care of creating a virtualenv with the required dependencies.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
