Lunderberg commented on a change in pull request #8576:
URL: https://github.com/apache/tvm/pull/8576#discussion_r678589727
##########
File path: tests/scripts/setup-pytest-env.sh
##########
@@ -32,7 +32,33 @@ export PYTHONPATH="${TVM_PATH}/python"
export TVM_PYTEST_RESULT_DIR="${TVM_PATH}/build/pytest-results"
mkdir -p "${TVM_PYTEST_RESULT_DIR}"
+if [ -n "${CI_PYTEST_NUM_CPUS-}" ]; then
+ PYTEST_NUM_CPUS=${CI_PYTEST_NUM_CPUS}
+else
+ PYTEST_NUM_CPUS=$(nproc)
+ if [ -z "${PYTEST_NUM_CPUS}" ]; then
+ PYTEST_NUM_CPUS=1
+ elif [ ${PYTEST_NUM_CPUS} -gt 1 ]; then
+ PYTEST_NUM_CPUS=$(expr ${PYTEST_NUM_CPUS} - 1) # Don't nuke
interactive work.
+ fi
+fi
+
+if [ ${PYTEST_NUM_CPUS} -gt 8 ]; then
+ PYTEST_NUM_CPUS=8 # It usually doesn't make sense to launch > 8 workers
+fi
+
+# DNS: remove after we actually fix up CI_PYTEST_NUM_CPUS in Jenkinsfile
+if [ ${PYTEST_NUM_CPUS} -gt 2 ]; then
+ PYTEST_NUM_CPUS=2 # Fix to 2 CPUs for Jenkins
+fi
+
+
function run_pytest() {
+ local extra_args=( )
+ if [ "$1" == "--parallel" ]; then
Review comment:
Since the documentation [currently
recommends](https://github.com/apache/tvm/blob/main/docs/contribute/pull_request.rst#L121)
using `task_python_unittest.sh` to run the unit tests, we should either make
sure that location also has `pytest-xdist` in the list of packages to install
in order to run the tests, or check whether `pytest-xdist` is installed by
wrapping this in `if python3 -c "import xdist" > /dev/null 2>&1; then`.
##########
File path: tests/scripts/task_python_unittest.sh
##########
@@ -31,9 +31,9 @@ if [ -z "${TVM_UNITTEST_TESTSUITE_NAME:-}" ]; then
fi
# First run minimal test on both ctypes and cython.
-run_pytest ctypes ${TVM_UNITTEST_TESTSUITE_NAME}-platform-minimal-test
tests/python/all-platform-minimal-test
-run_pytest cython ${TVM_UNITTEST_TESTSUITE_NAME}-platform-minimal-test
tests/python/all-platform-minimal-test
+run_pytest --parallel ctypes
${TVM_UNITTEST_TESTSUITE_NAME}-platform-minimal-test
tests/python/all-platform-minimal-test
Review comment:
Looks like this applies to everything in the unittest directory,
including `test_tvm_testing_features.py`. I don't see the [explicit
ordering](https://github.com/apache/tvm/pull/8343#discussion_r660038825) that
will be required for some of those tests, so we should add it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
