[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/7031#discussion_r33524819
--- Diff: python/run-tests.py ---
@@ -43,34 +50,43 @@ def print_red(text):
LOG_FILE = os.path.join(SPARK_HOME, python/unit-tests.log)
+LOG_FILE_LOCK = Lock()
+LOGGER = logging.getLogger()
def run_individual_python_test(test_name, pyspark_python):
env = {'SPARK_TESTING': '1', 'PYSPARK_PYTHON': which(pyspark_python)}
-print(Running test: %s ... % test_name, end='')
+LOGGER.info(Starting test(%s): %s, pyspark_python, test_name)
--- End diff --
We could remove this line, to not having them interleaved.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116787481 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116795411 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116787900 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116787968 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116790208 [Test build #36011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36011/consoleFull) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116795337 [Test build #36011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36011/console) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/7031#discussion_r33521735
--- Diff: python/run-tests.py ---
@@ -43,34 +50,43 @@ def print_red(text):
LOG_FILE = os.path.join(SPARK_HOME, python/unit-tests.log)
+LOG_FILE_LOCK = Lock()
+LOGGER = logging.getLogger()
def run_individual_python_test(test_name, pyspark_python):
env = {'SPARK_TESTING': '1', 'PYSPARK_PYTHON': which(pyspark_python)}
-print(Running test: %s ... % test_name, end='')
+LOGGER.info(Starting test(%s): %s, pyspark_python, test_name)
start_time = time.time()
-with open(LOG_FILE, 'a') as log_file:
-retcode = subprocess.call(
-[os.path.join(SPARK_HOME, bin/pyspark), test_name],
-stderr=log_file, stdout=log_file, env=env)
+per_test_output = tempfile.TemporaryFile()
+retcode = subprocess.Popen(
+[os.path.join(SPARK_HOME, bin/pyspark), test_name],
+stderr=per_test_output, stdout=per_test_output, env=env).wait()
duration = time.time() - start_time
+with LOG_FILE_LOCK:
+with open(LOG_FILE, 'ab') as log_file:
+per_test_output.seek(0)
--- End diff --
TODO: only need to print the log when the test fails.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116876711 LGTM, once you finished all the TODOs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116887073 [Test build #36056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36056/consoleFull) for PR 7031 at commit [`d4ded73`](https://github.com/apache/spark/commit/d4ded7368f6c3d9c7e7c958ee2925f3867116aa9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116886922 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116886948 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116887324 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116887322 [Test build #36056 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36056/console) for PR 7031 at commit [`d4ded73`](https://github.com/apache/spark/commit/d4ded7368f6c3d9c7e7c958ee2925f3867116aa9). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116206123 [Test build #35918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35918/consoleFull) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116207504 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116206924 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116207547 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208416 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208411 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208419 [Test build #35925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35925/consoleFull) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208897 Looks like that didn't help: ``` Build step 'Publish JUnit test result report' changed build result to UNSTABLE Finished: UNSTABLE ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116218969 [Test build #35927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35927/console) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116219009 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116206065 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116206054 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116207939 [Test build #35923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35923/consoleFull) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208281 [Test build #35923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35923/console) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116213031 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208387 I wonder if I can just add a `touch core/target/test-reports/dummy-test-report.xml` to Jenkins in order to prevent the build from failing for PRs that don't run JVM tests. Let me try... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208400 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116213754 [Test build #35927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35927/consoleFull) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116213673 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116213704 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116205997 Now that my other patch has been merged, I've rebased this so that the changes are easier to review (less than 100 lines of diff now). This still isn't quite ready to merge (see TODO checklist in the PR description), but I think it's now at a state where I'm ready for some initial review feedback. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116206905 [Test build #35918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35918/console) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116206908 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208282 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208808 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-116208806 [Test build #35925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35925/console) for PR 7031 at commit [`110cd9d`](https://github.com/apache/spark/commit/110cd9dd9877eb5b6934639b5e5adde15748cd57). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115546971 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115546980 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115547296 [Test build #35834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35834/consoleFull) for PR 7031 at commit [`3064eb1`](https://github.com/apache/spark/commit/3064eb16caa33979ac4c8cf63ce23344a0269cb5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115551753 [Test build #35834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35834/console) for PR 7031 at commit [`3064eb1`](https://github.com/apache/spark/commit/3064eb16caa33979ac4c8cf63ce23344a0269cb5). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115551816 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115558468 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115558386 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115558885 [Test build #35838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35838/consoleFull) for PR 7031 at commit [`d81119e`](https://github.com/apache/spark/commit/d81119e5b760e660e1de435c7788aa0c91d1d6f8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115561536 [Test build #35838 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35838/console) for PR 7031 at commit [`d81119e`](https://github.com/apache/spark/commit/d81119e5b760e660e1de435c7788aa0c91d1d6f8). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115561567 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115561796 Interesting, it looks like the test failures are not a direct consequence of the parallelization since they still occur even if I only use one thread: ```python == ERROR: test_termination_sigterm (__main__.DaemonTests) Ensure that daemon and workers terminate on SIGTERM. -- Traceback (most recent call last): File /home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/tests.py, line 1451, in test_termination_sigterm self.do_termination_test(lambda daemon: os.kill(daemon.pid, SIGTERM)) File /home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/tests.py, line 1424, in do_termination_test daemon = Popen([sys.executable, daemon_path], stdin=PIPE, stdout=PIPE) File /usr/lib64/python2.6/subprocess.py, line 642, in __init__ errread, errwrite) File /usr/lib64/python2.6/subprocess.py, line 1234, in _execute_child raise child_exception OSError: [Errno 13] Permission denied == ERROR: test_termination_stdin (__main__.DaemonTests) Ensure that daemon and workers terminate when stdin is closed. -- Traceback (most recent call last): File /home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/tests.py, line 1446, in test_termination_stdin self.do_termination_test(lambda daemon: daemon.stdin.close()) File /home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/tests.py, line 1424, in do_termination_test daemon = Popen([sys.executable, daemon_path], stdin=PIPE, stdout=PIPE) File /usr/lib64/python2.6/subprocess.py, line 642, in __init__ errread, errwrite) File /usr/lib64/python2.6/subprocess.py, line 1234, in _execute_child raise child_exception OSError: [Errno 13] Permission denied -- Ran 97 tests in 102.677s FAILED (errors=2) Random listing order was used ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115562141 Also, it looks like my attempt to disable the Web UI in the Python tests was unsuccessful, since the `unit-tests.log` file still contains a bunch of port-contention errors while starting the UI: ``` java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:316) at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:265) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at org.eclipse.jetty.server.Server.doStart(Server.java:293) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at org.apache.spark.ui.JettyUtils$.org$apache$spark$ui$JettyUtils$$connect$1(JettyUtils.scala:228) at org.apache.spark.ui.JettyUtils$$anonfun$2.apply(JettyUtils.scala:238) at org.apache.spark.ui.JettyUtils$$anonfun$2.apply(JettyUtils.scala:238) at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1985) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1976) at org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:238) at org.apache.spark.ui.WebUI.bind(WebUI.scala:117) at org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:448) at org.apache.spark.SparkContext$$anonfun$13.apply(SparkContext.scala:448) at scala.Option.foreach(Option.scala:236) at org.apache.spark.SparkContext.init(SparkContext.scala:448) at org.apache.spark.api.java.JavaSparkContext.init(JavaSparkContext.scala:61) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) at py4j.Gateway.invoke(Gateway.java:214) at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79) at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68) at py4j.GatewayConnection.run(GatewayConnection.java:207) at java.lang.Thread.run(Thread.java:745) ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115568760 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115568790 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115585484 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115569672 [Test build #35841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35841/consoleFull) for PR 7031 at commit [`29ab78d`](https://github.com/apache/spark/commit/29ab78d82c2e024c240a24a403352e41dde08dbf). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115574293 [Test build #35841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35841/console) for PR 7031 at commit [`29ab78d`](https://github.com/apache/spark/commit/29ab78d82c2e024c240a24a403352e41dde08dbf). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115574310 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115585374 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115586604 [Test build #35847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35847/consoleFull) for PR 7031 at commit [`2b51724`](https://github.com/apache/spark/commit/2b517248078201d1c68caf6c76239155471cfea3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115755863 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115757621 [Test build #35866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35866/consoleFull) for PR 7031 at commit [`c022b47`](https://github.com/apache/spark/commit/c022b4730b7394c76c92f11aaa775930f39f114c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115762312 [Test build #35866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35866/console) for PR 7031 at commit [`c022b47`](https://github.com/apache/spark/commit/c022b4730b7394c76c92f11aaa775930f39f114c). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115748089 These tests are now failing since we're now attempting to run the PySpark ML tests with PyPy, but that won't work because we don't have numpy for PyPy. Let me see if I can come up with a reasonably clean way to handle this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115755956 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115762388 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115828029 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115784499 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115815691 [Test build #35871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35871/consoleFull) for PR 7031 at commit [`f960ee5`](https://github.com/apache/spark/commit/f960ee5757c04b4dc64dbb5182fe995d41c8380a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115812080 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115767375 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115814708 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115814785 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115828017 [Test build #35871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35871/console) for PR 7031 at commit [`f960ee5`](https://github.com/apache/spark/commit/f960ee5757c04b4dc64dbb5182fe995d41c8380a). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115767443 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115842050 [Test build #35877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35877/consoleFull) for PR 7031 at commit [`3279c34`](https://github.com/apache/spark/commit/3279c34c8440bfd9f90e80cdc111dd5e4e588415). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115876652 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115876640 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115877669 [Test build #35883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35883/consoleFull) for PR 7031 at commit [`facfafe`](https://github.com/apache/spark/commit/facfafe856fe73081ce87853f794f20ddd256a93). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115883884 [Test build #35883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35883/console) for PR 7031 at commit [`facfafe`](https://github.com/apache/spark/commit/facfafe856fe73081ce87853f794f20ddd256a93). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115883907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115845288 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115845511 [Test build #35880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35880/consoleFull) for PR 7031 at commit [`5f2d295`](https://github.com/apache/spark/commit/5f2d295431271f8223e304a0976c80019c600ce6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115849867 [Test build #35880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35880/console) for PR 7031 at commit [`5f2d295`](https://github.com/apache/spark/commit/5f2d295431271f8223e304a0976c80019c600ce6). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115849901 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115840872 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115845306 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115852418 [Test build #35877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35877/console) for PR 7031 at commit [`3279c34`](https://github.com/apache/spark/commit/3279c34c8440bfd9f90e80cdc111dd5e4e588415). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115852460 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115613905 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115613897 [Test build #35847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35847/console) for PR 7031 at commit [`2b51724`](https://github.com/apache/spark/commit/2b517248078201d1c68caf6c76239155471cfea3). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115934404 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115934402 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115934516 [Test build #35890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35890/consoleFull) for PR 7031 at commit [`17e52c3`](https://github.com/apache/spark/commit/17e52c3af46da5270fe6bef0aa045546bd77ebe5). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115936098 [Test build #35890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35890/console) for PR 7031 at commit [`17e52c3`](https://github.com/apache/spark/commit/17e52c3af46da5270fe6bef0aa045546bd77ebe5). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class Module(object):` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115936103 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115957716 It's also worth figuring out why the Spark UI doesn't seem to be disabled properly according to `unit-tests.logs`, since problems there could be a source of flakiness once we aggressively bump up the parallelism. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115957527 Yay, this passed tests! I think that we should merge my other PR, #6967, first, then loop back to review the parallelism-specific changes here. With 4 threads / processes of parallelism, the Python tests ran in ~10 minutes, whereas they usually take 30+ minutes in Jenkins. In principle, + our tests will support a much higher level of parallelism, so I bet we could cut this down to 5 minutes or less. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115519294 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5161] [WIP] Parallelize Python test exe...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7031#issuecomment-115519286 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
