GitHub user WeichenXu123 opened a pull request:
https://github.com/apache/spark/pull/13275
[SPARK-15499][PySpark][Tests] Add python testsuite with remote debug and
single test parameter to help developer debug code easier
## What changes were proposed in this pull request?
To python/run-tests.py script, I add the following parameters:
--single-test=SINGLE_TEST
specify a python module to run single python test.
--debug-server=DEBUG_SERVER
debug server host, only used in single test.
--debug-port=DEBUG_PORT
debug server port, only used in single test.
Now, for example, I want to debug only pyspark.tests, I can use the
following command:
(first startup debug server in your python IDE such as pycharm or pydev,
your IED on machine which has host MY_DEV_MACHINE and using debug port 5678,
and make sure the machine running test can connect to MY_DEV_MACHINE)
python/run-tests --python-executables=python2.7 --single-test=pyspark.tests
--debug-server=MY_DEV_MACHINE --debug-port=5678
the parameter --single-test specify the single python testsuite you want to
test, currently you can use the following value:
`
pyspark.tests
pyspark.ml.tests
pyspark.mllib.tests
pyspark.sql.tests
pyspark.streaming.tests
`
## How was this patch tested?
Existing test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/WeichenXu123/spark add_python_remote_debug
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13275.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13275
----
commit 872ebdedc80a76b9bf064bda52af6074d16e7efc
Author: WeichenXu <[email protected]>
Date: 2016-05-24T05:32:35Z
add python test with single test and remote debug param
commit e694d10f27d2da9607c6f52bb7b2a2e2fd50e0fd
Author: WeichenXu <[email protected]>
Date: 2016-05-24T05:41:53Z
remove useless code which used for debug by myself
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
