[
https://issues.apache.org/jira/browse/BEAM-1755?focusedWorklogId=104714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-104714
]
ASF GitHub Bot logged work on BEAM-1755:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/May/18 17:33
Start Date: 22/May/18 17:33
Worklog Time Spent: 10m
Work Description: pabloem closed pull request #5400: [BEAM-1755] Add a
directory with build-specific scripts to clear up the Python SDK dir
URL: https://github.com/apache/beam/pull/5400
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/build.gradle b/sdks/python/build.gradle
index 0eea94d92fb..ff7e6dd0491 100644
--- a/sdks/python/build.gradle
+++ b/sdks/python/build.gradle
@@ -201,7 +201,7 @@ task postCommitVRTests(dependsOn: 'installGcpTest') {
doLast {
exec {
executable 'sh'
- args '-c', ". ${envdir}/bin/activate && ./run_postcommit.sh IT"
+ args '-c', ". ${envdir}/bin/activate && ./scripts/run_postcommit.sh IT"
}
}
}
@@ -210,7 +210,7 @@ task validatesRunnerTests(dependsOn: 'installGcpTest') {
doLast {
exec {
executable 'sh'
- args '-c', ". ${envdir}/bin/activate && ./run_postcommit.sh
ValidatesRunner"
+ args '-c', ". ${envdir}/bin/activate && ./scripts/run_postcommit.sh
ValidatesRunner"
}
}
}
diff --git a/sdks/python/generate_pydoc.sh
b/sdks/python/scripts/generate_pydoc.sh
similarity index 95%
rename from sdks/python/generate_pydoc.sh
rename to sdks/python/scripts/generate_pydoc.sh
index 68b7006a1b6..dfb87a13f11 100755
--- a/sdks/python/generate_pydoc.sh
+++ b/sdks/python/scripts/generate_pydoc.sh
@@ -22,6 +22,17 @@
#
# The exit-code of the script indicates success or a failure.
+# Check that the script is running in a known directory.
+if [[ $PWD != *sdks/python* ]]; then
+ echo 'Unable to locate Apache Beam Python SDK root directory'
+ exit 1
+fi
+
+# Go to the Apache Beam Python SDK root
+if [[ "*sdks/python" != $PWD ]]; then
+ cd $(pwd | sed 's/sdks\/python.*/sdks\/python/')
+fi
+
# Quit on any errors
set -e
diff --git a/sdks/python/run_mini_py3lint.sh
b/sdks/python/scripts/run_mini_py3lint.sh
similarity index 82%
rename from sdks/python/run_mini_py3lint.sh
rename to sdks/python/scripts/run_mini_py3lint.sh
index 0fd9ef7f507..63e71771d73 100755
--- a/sdks/python/run_mini_py3lint.sh
+++ b/sdks/python/scripts/run_mini_py3lint.sh
@@ -22,6 +22,17 @@
#
# The exit-code of the script indicates success or a failure.
+# Check that the script is running in a known directory.
+if [[ $PWD != *sdks/python* ]]; then
+ echo 'Unable to locate Apache Beam Python SDK root directory'
+ exit 1
+fi
+
+# Go to the Apache Beam Python SDK root
+if [[ "*sdks/python" != $PWD ]]; then
+ cd $(pwd | sed 's/sdks\/python.*/sdks\/python/')
+fi
+
set -o errexit
set -o pipefail
diff --git a/sdks/python/run_postcommit.sh
b/sdks/python/scripts/run_postcommit.sh
similarity index 85%
rename from sdks/python/run_postcommit.sh
rename to sdks/python/scripts/run_postcommit.sh
index afcdb3be6b2..d6e31aaccff 100755
--- a/sdks/python/run_postcommit.sh
+++ b/sdks/python/scripts/run_postcommit.sh
@@ -23,8 +23,19 @@
# PROJECT -> Project name to use for service jobs.
#
+# Check that the script is running in a known directory.
+if [[ $PWD != *sdks/python* ]]; then
+ echo 'Unable to locate Apache Beam Python SDK root directory'
+ exit 1
+fi
+
+# Go to the Apache Beam Python SDK root
+if [[ "*sdks/python" != $PWD ]]; then
+ cd $(pwd | sed 's/sdks\/python.*/sdks\/python/')
+fi
+
if [ -z "$1" ]; then
- printf "Usage: \n$> ./run_postcommit.sh <test_type> [gcp_location]
[gcp_project]"
+ printf "Usage: \n$> ./scripts/run_postcommit.sh <test_type> [gcp_location]
[gcp_project]"
printf "\n\ttest_type: ValidatesRunner or IT"
printf "\n\tgcp_location: A gs:// path to stage artifacts and output results"
printf "\n\tgcp_project: A GCP project to run Dataflow pipelines\n"
diff --git a/sdks/python/run_pylint.sh b/sdks/python/scripts/run_pylint.sh
similarity index 92%
rename from sdks/python/run_pylint.sh
rename to sdks/python/scripts/run_pylint.sh
index 4e7dc9d8218..17f6aa706c1 100755
--- a/sdks/python/run_pylint.sh
+++ b/sdks/python/scripts/run_pylint.sh
@@ -23,6 +23,17 @@
#
# The exit-code of the script indicates success or a failure.
+# Check that the script is running in a known directory.
+if [[ $PWD != *sdks/python* ]]; then
+ echo 'Unable to locate Apache Beam Python SDK root directory'
+ exit 1
+fi
+
+# Go to the Apache Beam Python SDK root
+if [[ "*sdks/python" != $PWD ]]; then
+ cd $(pwd | sed 's/sdks\/python.*/sdks\/python/')
+fi
+
set -o errexit
set -o pipefail
diff --git a/sdks/python/run_pylint_2to3.sh
b/sdks/python/scripts/run_pylint_2to3.sh
similarity index 91%
rename from sdks/python/run_pylint_2to3.sh
rename to sdks/python/scripts/run_pylint_2to3.sh
index 9a074368dba..56c87606dc6 100755
--- a/sdks/python/run_pylint_2to3.sh
+++ b/sdks/python/scripts/run_pylint_2to3.sh
@@ -22,6 +22,17 @@
#
# The exit-code of the script indicates success or a failure.
+# Check that the script is running in a known directory.
+if [[ $PWD != *sdks/python* ]]; then
+ echo 'Unable to locate Apache Beam Python SDK root directory'
+ exit 1
+fi
+
+# Go to the Apache Beam Python SDK root
+if [[ "*sdks/python" != $PWD ]]; then
+ cd $(pwd | sed 's/sdks\/python.*/sdks\/python/')
+fi
+
set -o errexit
set -o pipefail
diff --git a/sdks/python/run_tox_cleanup.sh
b/sdks/python/scripts/run_tox_cleanup.sh
similarity index 78%
rename from sdks/python/run_tox_cleanup.sh
rename to sdks/python/scripts/run_tox_cleanup.sh
index 76b683cf588..87081da8e17 100755
--- a/sdks/python/run_tox_cleanup.sh
+++ b/sdks/python/scripts/run_tox_cleanup.sh
@@ -19,6 +19,17 @@
# This script is used to remove generated files in previous Tox runs so that
# they are not picked up by tests.
+# Check that the script is running in a known directory.
+if [[ $PWD != *sdks/python* ]]; then
+ echo 'Unable to locate Apache Beam Python SDK root directory'
+ exit 1
+fi
+
+# Go to the Apache Beam Python SDK root
+if [[ "*sdks/python" != $PWD ]]; then
+ cd $(pwd | sed 's/sdks\/python.*/sdks\/python/')
+fi
+
set -e
for dir in apache_beam target/build; do
diff --git a/sdks/python/tox.ini b/sdks/python/tox.ini
index 61dc12c3017..b40133e21f6 100644
--- a/sdks/python/tox.ini
+++ b/sdks/python/tox.ini
@@ -48,10 +48,10 @@ list_dependencies_command = {envbindir}/python
{envbindir}/pip freeze
commands =
python --version
pip --version
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
python apache_beam/examples/complete/autocomplete_test.py
python setup.py nosetests
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
[testenv:py27-cython]
# cython tests are only expected to work in linux (2.x and 3.x)
@@ -62,20 +62,20 @@ platform = linux2
commands =
python --version
pip --version
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
python apache_beam/examples/complete/autocomplete_test.py
python setup.py nosetests
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
[testenv:py27-gcp]
extras = test,gcp
commands =
python --version
pip --version
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
python apache_beam/examples/complete/autocomplete_test.py
python setup.py nosetests
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
[testenv:py27-lint]
deps =
@@ -87,7 +87,7 @@ deps =
commands =
python --version
pip --version
- time {toxinidir}/run_pylint.sh
+ time {toxinidir}/scripts/run_pylint.sh
[testenv:py27-lint3]
# Checks for py2/3 compatibility issues
@@ -103,7 +103,7 @@ modules =
commands =
python --version
pip --version
- time {toxinidir}/run_pylint_2to3.sh {[testenv:py27-lint3]modules}
+ time {toxinidir}/scripts/run_pylint_2to3.sh {[testenv:py27-lint3]modules}
[testenv:py3-lint]
@@ -118,7 +118,7 @@ setenv =
commands =
python --version
pip --version
- time {toxinidir}/run_mini_py3lint.sh
+ time {toxinidir}/scripts/run_mini_py3lint.sh
[testenv:docs]
extras = test,gcp,docs
@@ -128,7 +128,7 @@ deps =
commands =
python --version
pip --version
- time {toxinidir}/generate_pydoc.sh
+ time {toxinidir}/scripts/generate_pydoc.sh
[testenv:cover]
# This is not included in envlist which is defined in [tox].
@@ -137,7 +137,7 @@ deps =
commands =
python --version
pip --version
- {toxinidir}/run_tox_cleanup.sh
+ {toxinidir}/scripts/run_tox_cleanup.sh
# Clean up previously collected data.
coverage erase
coverage run setup.py test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 104714)
Time Spent: 2h 20m (was: 2h 10m)
> Python-SDK: Move build specific scripts to a dedicated folder
> -------------------------------------------------------------
>
> Key: BEAM-1755
> URL: https://issues.apache.org/jira/browse/BEAM-1755
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Tibor Kiss
> Assignee: Pablo Estrada
> Priority: Minor
> Labels: newbie, starter
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> There are numerous build related files (run_*.sh, generate_pydoc.sh and most
> recently findSupportedPython.groovy) are located now in Python-SDK's root.
> We should create a dedicated {{build_utils}} directory and relocate the
> scripts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
