[GitHub] spark issue #21996: [SPARK-24888][CORE] spark-submit --master spark://host:p...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21996 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21996: [SPARK-24888][CORE] spark-submit --master spark://host:p...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21996 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21987: [SPARK-25015][BUILD] Update Hadoop 2.7 to 2.7.7
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21987 **[Test build #4232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4232/testReport)** for PR 21987 at commit [`654b918`](https://github.com/apache/spark/commit/654b918fc536de67c88e1bc4d618d27aa4fb76f1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94191/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94191/testReport)** for PR 21977 at commit [`2fb3d79`](https://github.com/apache/spark/commit/2fb3d790dac74a8db6fffac7f34c89880b53e11f). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21996: [SPARK-24888][CORE] spark-submit --master spark:/...
GitHub user devaraj-kavali opened a pull request:
https://github.com/apache/spark/pull/21996
[SPARK-24888][CORE] spark-submit --master spark://host:port --status
driver-id does not work
## What changes were proposed in this pull request?
In `SparkSubmit.scala` (`val uninitLog = initializeLogIfNecessary(true,
silent = true)`) -> `Logging.scala` (`val replLevel =
Option(replLogger.getLevel()).getOrElse(Level.WARN)`), the log level for
`rootLogger ` is overiri to `WARN ` but the status of the driver and kill
driver commands status have been logging with `INFO ` log level, so there is
nothing printing on the console for status and kill commands. This PR overrides
the `logInfo()` for `RestSubmissionClient` and redirects the msg to
`printStream`.
## How was this patch tested?
I verified it manually by running status and kill commands, these are the
results with and without the PR change.
- Without the PR change
```
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --status driver-20180803165641-
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --kill driver-20180803165641-
```
- With the PR change
```
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --kill driver-20180803165641-
Submitting a request to kill submission driver-20180803165641- in
spark://user1-work-pc:6066.
Server responded with KillSubmissionResponse:
{
"action" : "KillSubmissionResponse",
"message" : "Driver driver-20180803165641- has already finished or
does not exist",
"serverSparkVersion" : "2.4.0-SNAPSHOT",
"submissionId" : "driver-20180803165641-",
"success" : false
}
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --status driver-20180803165641-
Submitting a request for the status of submission
driver-20180803165641- in spark://user1-work-pc:6066.
Server responded with SubmissionStatusResponse:
{
"action" : "SubmissionStatusResponse",
"driverState" : "FINISHED",
"serverSparkVersion" : "2.4.0-SNAPSHOT",
"submissionId" : "driver-20180803165641-",
"success" : true,
"workerHostPort" : "xx.x.xx.xxx:42040",
"workerId" : "worker-20180803165615-10.3.66.149-42040"
}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/devaraj-kavali/spark SPARK-24888
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21996.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21996
commit 1e8000ffacabac742d16efee72ec1e421225a272
Author: Devaraj K
Date: 2018-08-04T00:47:23Z
[SPARK-24888][CORE] spark-submit --master spark://host:port --status
driver-id does not work
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21911 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21911 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94173/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21911 **[Test build #94173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94173/testReport)** for PR 21911 at commit [`acda5e2`](https://github.com/apache/spark/commit/acda5e281f1e3e8fd87faf4c8582d39ea857d215). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94191/testReport)** for PR 21977 at commit [`2fb3d79`](https://github.com/apache/spark/commit/2fb3d790dac74a8db6fffac7f34c89880b53e11f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1785/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94190/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #94190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94190/testReport)** for PR 21847 at commit [`02a86b9`](https://github.com/apache/spark/commit/02a86b9703ce5468c717550ea4e2a24f0f3cdbee). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #94190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94190/testReport)** for PR 21847 at commit [`02a86b9`](https://github.com/apache/spark/commit/02a86b9703ce5468c717550ea4e2a24f0f3cdbee). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94189/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94189/testReport)** for PR 21977 at commit [`f427120`](https://github.com/apache/spark/commit/f42712055a017f270b58576acd3efa3272d90767). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94189/testReport)** for PR 21977 at commit [`f427120`](https://github.com/apache/spark/commit/f42712055a017f270b58576acd3efa3272d90767). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1784/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21978 **[Test build #94188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94188/testReport)** for PR 21978 at commit [`00295ee`](https://github.com/apache/spark/commit/00295ee6b3713995641c90a9b3b7cd4a6b79ded6). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21978 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21977 @rdblue I'll take a look at that test on Monday during one of my streams after I get something else done :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21978 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1783/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...
Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207692117
--- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala
---
@@ -51,6 +52,17 @@ private[spark] class PythonRDD(
val bufferSize = conf.getInt("spark.buffer.size", 65536)
val reuseWorker = conf.getBoolean("spark.python.worker.reuse", true)
+ val memoryMb = {
--- End diff --
I think there might be a misunderstanding on what `reuseWorker` means
perhaps. The workers will be reused but the decision on if we fork in Python or
not is based on if we are in Windows or not. How about we both go and read the
code path there and see if we reach the same understanding? I could be off too.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21978: SPARK-25006: Add CatalogTableIdentifier.
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21978 FYI @jzhuge --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21403 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94178/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17185: [SPARK-19602][SQL] Support column resolution of fully qu...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17185 **[Test build #94187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94187/testReport)** for PR 17185 at commit [`90cd6d3`](https://github.com/apache/spark/commit/90cd6d33f59fdae16a3a386ed14cefb3f28d35a8). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21403: [SPARK-24341][SQL] Support only IN subqueries with the s...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21403 **[Test build #94178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94178/testReport)** for PR 21403 at commit [`a6114a6`](https://github.com/apache/spark/commit/a6114a655305f318230bf1bbd25394e952793a94). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class InSubquery(values: Seq[Expression], query: ListQuery)` * `case class ListQuery(` * `case class Exists(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17185: [SPARK-19602][SQL] Support column resolution of fully qu...
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/17185 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21994 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94177/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21994 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21994: [SPARK-24529][Build][test-maven][follow-up] Add spotbugs...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21994 **[Test build #94177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94177/testReport)** for PR 21994 at commit [`3501209`](https://github.com/apache/spark/commit/3501209badca1c4d38e4ca7d0ddc45d2977e34e0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17185: [SPARK-19602][SQL] Support column resolution of fully qu...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17185 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17185: [SPARK-19602][SQL] Support column resolution of fully qu...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17185 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94172/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17185: [SPARK-19602][SQL] Support column resolution of fully qu...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17185 **[Test build #94172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94172/testReport)** for PR 17185 at commit [`90cd6d3`](https://github.com/apache/spark/commit/90cd6d33f59fdae16a3a386ed14cefb3f28d35a8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 @holdenk, I attempted to write a YARN unit test for this, but evidently the MiniYARNCluster doesn't run python workers. The task is run, but a worker is never started. If you have any idea how to fix this, I think we could have an easy test. Here's what I have so far: https://gist.github.com/rdblue/9848a00f49eaad6126fbbcfa1b039e19 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94186/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94186/testReport)** for PR 21977 at commit [`68e9141`](https://github.com/apache/spark/commit/68e9141f14ccb68847ea30cda98c5e4f9b74ad52). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #94186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94186/testReport)** for PR 21977 at commit [`68e9141`](https://github.com/apache/spark/commit/68e9141f14ccb68847ea30cda98c5e4f9b74ad52). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/21977#discussion_r207690569
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala
---
@@ -81,6 +82,17 @@ case class AggregateInPandasExec(
val bufferSize = inputRDD.conf.getInt("spark.buffer.size", 65536)
val reuseWorker =
inputRDD.conf.getBoolean("spark.python.worker.reuse", defaultValue = true)
+val memoryMb = {
--- End diff --
I went ahead with the refactor.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1782/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21911: [SPARK-24940][SQL] Coalesce and Repartition Hint for SQL...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21911 **[Test build #94185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94185/testReport)** for PR 21911 at commit [`2dcab4b`](https://github.com/apache/spark/commit/2dcab4b491e38766889c9cb83b8bebd59aa507fd). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94184/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #94184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94184/testReport)** for PR 21847 at commit [`5cf20df`](https://github.com/apache/spark/commit/5cf20dfa89a74503ad9befed2c7e0c791aea7763). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...
Github user Achuth17 commented on the issue: https://github.com/apache/spark/pull/21608 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21982: [SPARK-23911][SQL] Add aggregate function.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21982 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94169/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21982: [SPARK-23911][SQL] Add aggregate function.
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21982 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21898 **[Test build #94183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94183/testReport)** for PR 21898 at commit [`16ee90e`](https://github.com/apache/spark/commit/16ee90e5bbecbb94b54d61703e7d7d17a58b3bcb). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21847: [SPARK-24855][SQL][EXTERNAL]: Built-in AVRO support shou...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21847 **[Test build #94184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94184/testReport)** for PR 21847 at commit [`5cf20df`](https://github.com/apache/spark/commit/5cf20dfa89a74503ad9befed2c7e0c791aea7763). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21898 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1781/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21982: [SPARK-23911][SQL] Add aggregate function.
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21982 **[Test build #94169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94169/testReport)** for PR 21982 at commit [`26bf379`](https://github.com/apache/spark/commit/26bf37960a1534da8e2119181dce5794e3b48172). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class ArrayAggregate(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21995 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21898 test this please (after #21995 ) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21986 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94170/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21986 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21995 LGTM Merging to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21986 **[Test build #94170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94170/testReport)** for PR 21986 at commit [`3f88e2a`](https://github.com/apache/spark/commit/3f88e2a927c22f4fc509b8ca96027ef381f7fe84). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait UnaryHigherOrderFunction extends HigherOrderFunction with ExpectsInputTypes ` * `trait ArrayBasedUnaryHigherOrderFunction extends UnaryHigherOrderFunction ` * `trait MapBasedUnaryHigherOrderFunction extends UnaryHigherOrderFunction ` * `case class MapFilter(` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94182/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...
Github user Achuth17 commented on the issue: https://github.com/apache/spark/pull/21608 @maropu @gatorsmile The tests are failing in places unrelated to my changes. How can I resolve this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21995 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21995 **[Test build #94182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94182/testReport)** for PR 21995 at commit [`517a48e`](https://github.com/apache/spark/commit/517a48eba9996481b61870879aeeedb1260f82f9). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21898 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21991 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94161/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21991 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21991 **[Test build #94161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94161/testReport)** for PR 21991 at commit [`2563311`](https://github.com/apache/spark/commit/2563311b2b52fb7ca886f7feca1b3d5dcf786e9c). * This patch **fails from timeout after a configured wait of \`300m\`**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21608 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21608 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94167/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #94167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94167/testReport)** for PR 21608 at commit [`253af70`](https://github.com/apache/spark/commit/253af70cd4cd525db4951db67fce01a5ca1f0014). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21992: Little typo
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21992 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21992: Little typo
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21992 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21816: [SPARK-24794][CORE] Driver launched through rest should ...
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21816 Jenkins OK to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21995 **[Test build #94182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94182/testReport)** for PR 21995 at commit [`517a48e`](https://github.com/apache/spark/commit/517a48eba9996481b61870879aeeedb1260f82f9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1780/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94164/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21995 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21632 **[Test build #94164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94164/testReport)** for PR 21632 at commit [`a34b3cd`](https://github.com/apache/spark/commit/a34b3cde5b6a1e426d153b7b4f3b2b4d7d70ba8c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21995: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client ...
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/21995 [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version from 0.10.0.1 to 2.0.0 ## What changes were proposed in this pull request? Increase ZK timeout and harmonize configs across Kafka tests to resolâ¦ve potentially flaky test failure ## How was this patch tested? Existing tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/srowen/spark SPARK-18057.3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21995.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21995 commit 517a48eba9996481b61870879aeeedb1260f82f9 Author: Sean Owen Date: 2018-08-03T22:27:06Z Increase ZK timeout and harmonize configs across Kafka tests to resolve potentially flaky test failure --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21955: [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21955 Let's continue at https://github.com/apache/spark/pull/21995 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21992: Little typo
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21992 **[Test build #4233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4233/testReport)** for PR 21992 at commit [`d15ab4b`](https://github.com/apache/spark/commit/d15ab4b4833487d1c3f87f4461fafc2f9a13cd49). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21991 **[Test build #94181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94181/testReport)** for PR 21991 at commit [`6732913`](https://github.com/apache/spark/commit/6732913905906743fda9842999d6c9c45dd6134a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `S...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/21991#discussion_r207680824
--- Diff: dev/merge_spark_pr.py ---
@@ -154,20 +154,22 @@ def merge_pr(pr_num, target_ref, title, body,
pr_repo_desc):
# to people every time someone creates a public fork of Spark.
merge_message_flags += ["-m", body.replace("@", "")]
-authors = "\n".join(["Author: %s" % a for a in distinct_authors])
-
-merge_message_flags += ["-m", authors]
+committer_name = run_cmd("git config --get user.name").strip()
+committer_email = run_cmd("git config --get user.email").strip()
if had_conflicts:
-committer_name = run_cmd("git config --get user.name").strip()
-committer_email = run_cmd("git config --get user.email").strip()
message = "This patch had conflicts when merged, resolved
by\nCommitter: %s <%s>" % (
committer_name, committer_email)
merge_message_flags += ["-m", message]
# The string "Closes #%s" string is required for GitHub to correctly
close the PR
merge_message_flags += ["-m", "Closes #%s from %s." % (pr_num,
pr_repo_desc)]
+authors = "\n".join(["Co-authored-by: %s" % a for a in
distinct_authors])
--- End diff --
For single author commit, should we do `Authored-by` instead of
`Co-authored-by`?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21991 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1779/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21991: [SPARK-25018] [Infra] Use `Co-authored-by` and `Signed-o...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21991 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21632 **[Test build #94180 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94180/testReport)** for PR 21632 at commit [`6326bdf`](https://github.com/apache/spark/commit/6326bdf8856a93e193b320c6579009d75db9409a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1778/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 looks like a random failure --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/21632 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94165/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21632: [SPARK-19591][ML][MLlib] Add sample weights to decision ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21632 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
