[GitHub] spark pull request: [SPARK-2693][SQL] Supported for UDAF Hive Aggr...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2620#issuecomment-57588061 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57588026 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21174/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57588021 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21174/consoleFull) for PR 2388 at commit [`99945ce`](https://github.com/apache/spark/commit/99945ce52e7559728191226fbc21a2a592591ceb). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class TopicModelingKryoRegistrator extends KryoRegistrator ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-57588159 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2693][SQL] Supported for UDAF Hive Aggr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2620#issuecomment-57588338 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21176/consoleFull) for PR 2620 at commit [`caf25c6`](https://github.com/apache/spark/commit/caf25c6633751f5418864a484304b17cf7a18b1a). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-57588340 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21177/consoleFull) for PR 1486 at commit [`338d4f8`](https://github.com/apache/spark/commit/338d4f8fedd68b64a7fdfaf078afcc2623072501). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2391 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP] SPARK-2450: Add YARN executor log links ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1375 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: Merge pull request #1 from apache/master
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2502 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: JIRA issue: [SPARK-1405] Gibbs sampling based ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/476 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1297 Upgrade HBase dependency to 0.98
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1893#issuecomment-57588918 @pwendell can you take a look at this when you have a chance --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57589288 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21175/consoleFull) for PR 789 at commit [`1be3fa5`](https://github.com/apache/spark/commit/1be3fa53c4daf29d5b0153f2ac39e6d221f9bc56). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` throw new SparkException(Failed to load class to register with Kryo, e)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57589292 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21175/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-57589474 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21178/consoleFull) for PR 2501 at commit [`65ed04a`](https://github.com/apache/spark/commit/65ed04afdc49f96d5f66257cb003f1e8e345095c). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3223 runAsSparkUser cannot change HDFS w...
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/2126#issuecomment-57589522 @tgravescs This code only apples in mesos mode, so another mode - yarn and standalone - is not affected. +1 @timothysc, val fwInfo = FrameworkInfo.newBuilder().setUser(sc.sparkUser).setName(sc.appName).build() --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57589906 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21179/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-57590286 Hey Patrick, thanks for looking at this. I did not say it is not possible. I just said the best(easiest ) way I could come up was to modify the maven install plugin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-57590353 And this https://github.com/ScrapCodes/scala-install-plugin plugin takes care of publishing correct poms too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...
GitHub user ScrapCodes reopened a pull request: https://github.com/apache/spark/pull/2357 [SPARK-3437][BUILD] Support crossbuilding in maven. With new scala-install-plugin. Since this plugin is not deployed anywhere, for anyone trying this patch has to publish it locally by cloning the following repo https://github.com/ScrapCodes/scala-install-plugin. And then running `mvn install`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ScrapCodes/spark-1 maven-improvements Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2357 commit 4d2d30cc9dcbe2b929ed8ed124d6719430ec6fae Author: Prashant Sharma prashan...@imaginea.com Date: 2014-09-11T08:09:46Z Supported new scala install plugin. Which can let us cross build for scala. commit 609dd98b7de77397dfc490c1c0a12bb9349830e5 Author: Prashant Sharma prashan...@imaginea.com Date: 2014-09-11T10:00:10Z Changed to newly updated with cross build support branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2377] Python API for Streaming
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2538#issuecomment-57590494 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21181/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57590495 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21180/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57590497 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21182/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2357#issuecomment-57590659 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21183/consoleFull) for PR 2357 at commit [`609dd98`](https://github.com/apache/spark/commit/609dd98b7de77397dfc490c1c0a12bb9349830e5). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3371][SQL] Renaming a function expressi...
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2511#issuecomment-57590958 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3371][SQL] Renaming a function expressi...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2511 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2693][SQL] Supported for UDAF Hive Aggr...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2620#issuecomment-57591440 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21176/consoleFull) for PR 2620 at commit [`caf25c6`](https://github.com/apache/spark/commit/caf25c6633751f5418864a484304b17cf7a18b1a). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2693][SQL] Supported for UDAF Hive Aggr...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2620#issuecomment-57591446 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21176/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2612#issuecomment-57591816 @pwendell Actually, CRLF is already contained in compute-classpath.cmd so I think, we should unify CRLF or LF in *.cmd. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3762] clear reference of SparkEnv after...
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/2624 [SPARK-3762] clear reference of SparkEnv after stop SparkEnv is cached in ThreadLocal object, so after stop and create a new SparkContext, old SparkEnv is still used by some threads, it will trigger many problems, for example, pyspark will have problem after restart SparkContext, because py4j use thread pool for RPC. This patch will clear all the references after stop a SparkEnv. You can merge this pull request into a Git repository by running: $ git pull https://github.com/davies/spark env Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2624.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2624 commit 4d0ea8bf5df513d5d1f4250286ca328192018f08 Author: Davies Liu davies@gmail.com Date: 2014-10-02T07:10:38Z clear reference of SparkEnv after stop --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2563#discussion_r18324999
--- Diff: python/pyspark/sql.py ---
@@ -205,6 +234,16 @@ def __str__(self):
return ArrayType(%s,%s) % (self.elementType,
str(self.containsNull).lower())
+simpleString = 'array'
+
+def jsonValue(self):
+return {
+self.simpleString: {
+'type': self.elementType.jsonValue(),
+'containsNull': self.containsNull
+}
+}
--- End diff --
Any suggestions about indenting and wrapping complex nested Python data
structure like this? I checked PEP8 while adding these lines, but didn't find
useful guidelines for this case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57592234 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3762] clear reference of SparkEnv after...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2624#issuecomment-57592595 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21184/consoleFull) for PR 2624 at commit [`4d0ea8b`](https://github.com/apache/spark/commit/4d0ea8bf5df513d5d1f4250286ca328192018f08). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57592605 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21185/consoleFull) for PR 789 at commit [`2def654`](https://github.com/apache/spark/commit/2def654da91e0cfc3a976d487566262c285b4b03). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-57592627 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21177/consoleFull) for PR 1486 at commit [`338d4f8`](https://github.com/apache/spark/commit/338d4f8fedd68b64a7fdfaf078afcc2623072501). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-57592639 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21178/consoleFull) for PR 2501 at commit [`65ed04a`](https://github.com/apache/spark/commit/65ed04afdc49f96d5f66257cb003f1e8e345095c). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-57592632 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21177/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-3212][SQL] Use logical plan matchi...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2501#issuecomment-57592643 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21178/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-57592942 Okay - gonna merge this. Glad it's in good shape now. Thanks @cmccabe for the contribution. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3645][SQL] Makes table caching eager by...
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2513#discussion_r18325314
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/CachedTableSuite.scala ---
@@ -69,23 +78,50 @@ class CachedTableSuite extends HiveComparisonTest {
test(correct error on uncache of non-cached table) {
intercept[IllegalArgumentException] {
- TestHive.uncacheTable(src)
+ uncacheTable(src)
}
}
test('CACHE TABLE' and 'UNCACHE TABLE' HiveQL statement) {
-TestHive.sql(CACHE TABLE src)
-TestHive.table(src).queryExecution.executedPlan match {
+sql(CACHE TABLE src)
+table(src).queryExecution.executedPlan match {
case _: InMemoryColumnarTableScan = // Found evidence of caching
case _ = fail(sTable 'src' should be cached)
}
assert(TestHive.isCached(src), Table 'src' should be cached)
-TestHive.sql(UNCACHE TABLE src)
-TestHive.table(src).queryExecution.executedPlan match {
+sql(UNCACHE TABLE src)
+table(src).queryExecution.executedPlan match {
case _: InMemoryColumnarTableScan = fail(sTable 'src' should not
be cached)
case _ = // Found evidence of uncaching
}
-assert(!TestHive.isCached(src), Table 'src' should not be cached)
+assert(!isCached(src), Table 'src' should not be cached)
+ }
+
+ test(CACHE TABLE tableName AS SELECT * FROM anotherTable) {
+sql(CACHE TABLE testCacheTable AS SELECT * FROM src)
+assert(isCached(testCacheTable), Table 'testCacheTable' should be
cached)
+assert(
+ isMaterialized(testCacheTable),
+ Eagerly cached in-memory table should have already been
materialized)
+uncacheTable(testCacheTable)
+ }
+
+ test(CACHE TABLE tableName AS SELECT ...) {
+sql(CACHE TABLE testCacheTable AS SELECT key FROM src LIMIT 10)
+assert(isCached(testCacheTable), Table 'testCacheTable' should be
cached)
+assert(
+ isMaterialized(testCacheTable),
+ Eagerly cached in-memory table should have already been
materialized)
+uncacheTable(testCacheTable)
+ }
+
+ test(LAZY CACHE TABLE tableName) {
--- End diff --
Agree, I wasn't very sure about the syntax either when add this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2461] Add a toString method to Generali...
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/2625 [SPARK-2461] Add a toString method to GeneralizedLinearModel Add a toString method to GeneralizedLinearModel, also change `__str__` to `__repr__` for some classes, to provide better message in repr. This PR is based on #1388, thanks to @sryza! closes #1388 You can merge this pull request into a Git repository by running: $ git pull https://github.com/davies/spark string Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2625.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2625 commit f7b58ed25d8228c25cac0fc72616fb6ebb392959 Author: Sandy Ryza sa...@cloudera.com Date: 2014-07-12T19:46:35Z SPARK-2461. Add a toString method to GeneralizedLinearModel commit fac1bc44d57db1c0fb23f0b246b21373b397781e Author: Sandy Ryza sa...@cloudera.com Date: 2014-08-31T17:51:03Z Fix PEP8 error commit 1aa17f57bc9ac74d68ca5a5235c3a0d0cc197993 Author: Sandy Ryza sa...@cloudera.com Date: 2014-09-02T06:52:36Z Match existing conventions commit a0c5041910dcfc1491c01954d9234336172e0f7b Author: Sandy Ryza sa...@cloudera.com Date: 2014-09-02T18:46:10Z Add labels back in commit aa9e962de5b56a71bbd67ef9efe9aad018f1d73c Author: Sandy Ryza sa...@cloudera.com Date: 2014-09-04T08:43:01Z Switch __str__ to __repr__ commit 1ce5c2d12c5fd4e078907903a138232e67f24335 Author: Sandy Ryza sa...@cloudera.com Date: 2014-09-10T20:59:32Z __repr__ back to __str__ in a couple places commit 0bcd642e1ad77a64de8ec9cde5405f1907099b93 Author: Davies Liu davies@gmail.com Date: 2014-10-02T07:19:40Z Merge branch 'sandy-spark-2461' of github.com:sryza/spark Conflicts: python/pyspark/mllib/linalg.py python/pyspark/mllib/regression.py python/pyspark/mllib/tree.py commit 3544aad2bd5fe691d72db930112252914e63cc5b Author: Davies Liu davies@gmail.com Date: 2014-10-02T07:23:07Z fix LinearModel --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2461] [PySpark] Add a toString method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2625#issuecomment-57593293 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21186/consoleFull) for PR 2625 at commit [`3544aad`](https://github.com/apache/spark/commit/3544aad2bd5fe691d72db930112252914e63cc5b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-2461. Add a toString method to Generaliz...
Github user sryza closed the pull request at: https://github.com/apache/spark/pull/1388 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-2461. Add a toString method to Generaliz...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/1388#issuecomment-57593494 Closing this in favor of #2625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1767: Prefer HDFS-cached replicas when s...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/1486#issuecomment-57593579 w00t! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2461] [PySpark] Add a toString method t...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/2625#issuecomment-57593551 Thanks for picking this up @davies. This looks good to me if Jenkins OKs it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3713][SQL] Uses JSON to serialize DataT...
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/2563#discussion_r18325514
--- Diff: python/pyspark/sql.py ---
@@ -205,6 +234,16 @@ def __str__(self):
return ArrayType(%s,%s) % (self.elementType,
str(self.containsNull).lower())
+simpleString = 'array'
+
+def jsonValue(self):
+return {
+self.simpleString: {
+'type': self.elementType.jsonValue(),
+'containsNull': self.containsNull
+}
+}
--- End diff --
I'd like this one:
```
{self.simpleString: {'type': self.elementType.jsonValue(),
'containsNull': self.containsNull}}}
```
it will be better if it has one layer:
```
{'type': self.simpleString,
'type': self.elementType.jsonValue(),
'containsNull': self.containsNull}
```
I prefer fewer lines personally, then I can read more codes in one screen.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-57593822 So now to try this patch just `mvn install - [this plugin](https://github.com/ScrapCodes/scala-install-plugin). After that look at the published poms. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57594202 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21187/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57594574 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21189/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-57594573 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21188/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3654][SQL] Implement all extended HiveQ...
Github user ravipesala commented on the pull request: https://github.com/apache/spark/pull/2590#issuecomment-57595734 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3755][Core] Do not bind port 1 - 1024 t...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2623#discussion_r18326718
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1437,7 +1437,13 @@ private[spark] object Utils extends Logging {
val serviceString = if (serviceName.isEmpty) else s '$serviceName'
for (offset - 0 to maxRetries) {
// Do not increment port if startPort is 0, which is treated as a
special port
- val tryPort = if (startPort == 0) startPort else (startPort +
offset) % 65536
+ val _tryPort = if (startPort == 0) startPort else (startPort +
offset) % 65536
--- End diff --
Doesn't this still have a similar issue? If I request port 80 (and am
running as root, yes), this will allocate port 1104. You could say this should
disallow privileged ports, but then that would be done explicitly.
What about `... else ((startPort + offset - 1024) % (65536 - 1024)) + 1024`
?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2357#issuecomment-57596973 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21183/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3437][BUILD] Support crossbuilding in m...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2357#issuecomment-57596967 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21183/consoleFull) for PR 2357 at commit [`609dd98`](https://github.com/apache/spark/commit/609dd98b7de77397dfc490c1c0a12bb9349830e5). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-57597447 I think you mean to try `replaceFile` when you said maven-shade-plugin. Let me try that as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-57597737 Ahh wait, I would still need to alter maven-install-plugin. Because, there has to be someway to tell install plugin that it has to install at location which has _2.10 at the end. Which means we will anyway substitute install plugin with something that does this. And I guess then best thing to do is use scala-install-plugin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3007][SQL] (This is a debugging PR to t...
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/2626 [SPARK-3007][SQL] (This is a debugging PR to test Jenkins build) **PLEASE DON'T REVIEW/MERGE THIS PR** Although #2226 passes Jenkins build, Jenkins master SBT build fails deterministically after #2226 is merged. This debugging PR is exactly the same as #2226, just used to double check whether it fails Jenkins or not. You can merge this pull request into a Git repository by running: $ git pull https://github.com/liancheng/spark dp-debug Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2626.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2626 commit dc24c41ed094c1d46c3f3752f17cbeeaa5ce890a Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-12T17:27:54Z Update HiveQl.scala commit 701a814c0fbb2310ddbb02a02b8730fa52dc42b1 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-12T17:29:36Z Update SparkHadoopWriter.scala commit a2374a89a3bdf927919e7f4c40fe9a65e7c1fbe7 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-12T17:31:01Z Update InsertIntoHiveTable.scala commit 491c7d095cde3c1e2ba9fa83d33d909d0b816d27 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-12T17:53:04Z Update InsertIntoHiveTable.scala commit 0a50db92ae6d1272381de1626ec9a737dd194eff Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-18T06:29:21Z Update HiveCompatibilitySuite.scala commit 60f70aa2e5956249107d8a4ee92bec58fb702628 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-18T06:34:53Z Update InsertIntoHiveTable.scala commit 0e18496e0d57b393f5b7c62cbd22f6fc3e4b57c4 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-18T06:36:58Z Update HiveQuerySuite.scala commit c1a59dd89cfd39aca979f1485eb81c916c0910f9 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-18T07:18:07Z Update Cast.scala commit f7467d0d527226e39485728b5fa8e2f0481af152 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-18T07:44:07Z Update InsertIntoHiveTable.scala commit a3961d9b6f7750dc0dfe6001f943ddc75330fd18 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-08-18T07:57:20Z Update Cast.scala commit 88d01102dd261fea95f49b63105173476c8efd8a Author: baishuo vc_j...@hotmail.com Date: 2014-08-19T16:01:49Z update file after test commit 66e33fcbf300d286691f47a6739ba95d89af504b Author: baishuo vc_j...@hotmail.com Date: 2014-08-19T17:14:53Z do a little modify commit 37c1c43650cacf04b5e4036e25d4235ddd9f6d56 Author: baishuo vc_j...@hotmail.com Date: 2014-09-03T03:28:29Z delete a empty else branch commit 7ce2d9f32dbe3e648833b126939b5f3e26c874fb Author: baishuo vc_j...@hotmail.com Date: 2014-09-03T06:03:34Z modify code to pass scala style checks commit c3ab36d01f11f4724882c852068e04aa420cc7f3 Author: baishuo vc_j...@hotmail.com Date: 2014-09-03T08:22:19Z modify for some bad indentation commit b47c9bf5cdf81a07519935180fcd82e0d1d12165 Author: baishuo vc_j...@hotmail.com Date: 2014-09-09T09:57:48Z modify according micheal's advice commit 2201c757cdae061e4ad8cbe5d32d4e7d0a0a3702 Author: baishuo vc_j...@hotmail.com Date: 2014-09-10T02:51:20Z use HiveConf.DEFAULTPARTITIONNAME to replace hive.exec.default.partition.name commit 5033928ee8709c3d1b758856efe2c6cb951ea574 Author: baishuo vc_j...@hotmail.com Date: 2014-09-10T03:09:20Z pass check style commit c464b262b0fc5cec014f39c9f55b319d7e18951d Author: Cheng Lian lian.cs@gmail.com Date: 2014-09-16T07:06:25Z Refactors dynamic partitioning support Conflicts: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala commit 528e84c44ce885accbb6a953fa604e5ad664032d Author: Cheng Lian lian.cs@gmail.com Date: 2014-09-17T01:32:12Z Fixes typo in test name, regenerated golden answer files commit fae9eff4f8d68e2ba0816d859fb7f4171c56b995 Author: Cheng Lian lian.cs@gmail.com Date: 2014-09-17T05:13:36Z Refactors InsertIntoHiveTable to a Command commit 50045422630711cf92768b30e305d0dde4ee04c8 Author: Cheng Lian lian.cs@gmail.com Date: 2014-09-17T05:41:56Z Minor refactoring commit 1093c2074ddd81b07c28bfb5369ff0b39be8f424 Author: Cheng Lian lian.cs@gmail.com Date: 2014-09-17T07:28:57Z Adds more tests commit 096bbbc261e6e5015d7ff4b90f299ea321ba7651 Author: baishuo(白硕) vc_j...@hotmail.com Date: 2014-09-17T09:15:36Z Merge pull request #1 from liancheng/refactor-dp Refactors dynamic partitioning support commit b20a3dcfaa5be0cfa925f11512c882c163070424 Author: Cheng Lian lian.cs@gmail.com Date: 2014-09-18T01:08:01Z Addresses @yhuai's comments commit e69ce883ee9d337a81d4aae3a63943937f771e84 Author: Cheng Lian lian.cs@gmail.com
[GitHub] spark pull request: [SPARK-3762] clear reference of SparkEnv after...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2624#issuecomment-57598250 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21184/consoleFull) for PR 2624 at commit [`4d0ea8b`](https://github.com/apache/spark/commit/4d0ea8bf5df513d5d1f4250286ca328192018f08). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3762] clear reference of SparkEnv after...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2624#issuecomment-57598254 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21184/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57598297 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21185/consoleFull) for PR 789 at commit [`2def654`](https://github.com/apache/spark/commit/2def654da91e0cfc3a976d487566262c285b4b03). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` throw new SparkException(Failed to load class to register with Kryo, e)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-1813. Add a utility to SparkConf that ma...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/789#issuecomment-57598304 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21185/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327521
--- Diff: core/src/main/java/org/apache/spark/ApplicationId.java ---
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark;
+
+import java.io.Serializable;
+
+/**
+ * This class represents unique application id for identifying each
application
+ */
+public class ApplicationId implements Serializable {
+
+ private String appId;
+
+ public ApplicationId(String appId) {
+this.appId = appId;
+ }
+
+ @Override
+ public boolean equals(Object other) {
+if (!(other instanceof ApplicationId)) {
+ return false;
+} else if (other == this) {
+ return true;
+} else if (appId != null) {
+ return appId.equals(((ApplicationId)other).appId);
+} else {
+ return false;
+}
+ }
+
+ @Override
+ public int hashCode() {
+return appId.hashCode();
+ }
+
+ @Override
+ public String toString() {
+return appId;
+ }
+
+}
--- End diff --
I've removed ApplicationId.java.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327526
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -187,6 +187,15 @@ class SparkContext(config: SparkConf) extends Logging {
val master = conf.get(spark.master)
val appName = conf.get(spark.app.name)
+ val isEventLogEnabled = conf.getBoolean(spark.eventLog.enabled, false)
+ val eventLogDir = {
--- End diff --
I've done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327540
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -294,7 +291,7 @@ class SparkContext(config: SparkConf) extends Logging {
executorEnvs(SPARK_USER) = sparkUser
// Create and start the scheduler
- private[spark] var taskScheduler =
SparkContext.createTaskScheduler(this, master)
+ private[spark] var taskScheduler =
SparkContext.createTaskScheduler(this, master, eventLogDir)
--- End diff --
I've modified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2461] [PySpark] Add a toString method t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2625#issuecomment-57598745 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21186/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327640
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1473,7 +1493,10 @@ object SparkContext extends Logging {
}
/** Creates a task scheduler based on a given master URL. Extracted for
testing. */
- private def createTaskScheduler(sc: SparkContext, master: String):
TaskScheduler = {
+ private def createTaskScheduler(
+sc: SparkContext,
+master: String,
+eventLogDir: Option[String]): TaskScheduler = {
--- End diff --
I've modified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2461] [PySpark] Add a toString method t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2625#issuecomment-57598737 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21186/consoleFull) for PR 2625 at commit [`3544aad`](https://github.com/apache/spark/commit/3544aad2bd5fe691d72db930112252914e63cc5b). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * ` println(sFailed to load main class $childMainClass.)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327621
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1278,8 +1298,8 @@ class SparkContext(config: SparkConf) extends Logging
{
private def postApplicationStart() {
// Note: this code assumes that the task scheduler has been
initialized and has contacted
// the cluster manager to get an application ID (in case the cluster
manager provides one).
-listenerBus.post(SparkListenerApplicationStart(appName,
taskScheduler.applicationId(),
- startTime, sparkUser))
+listenerBus.post(SparkListenerApplicationStart(appName,
+ Some(taskScheduler.applicationId().toString), startTime, sparkUser))
--- End diff --
I think replacing Option[String] with String should be discussed as another
issue so I leave.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327673
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -98,10 +96,23 @@ private[spark] class MetricsSystem private (
sinks.foreach(_.report())
}
+ def buildRegistryName(source: Source) = {
--- End diff --
I've added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327652
--- Diff: core/src/main/scala/org/apache/spark/deploy/DeployMessage.scala
---
@@ -71,11 +72,15 @@ private[deploy] object DeployMessages {
case class RegisterWorkerFailed(message: String) extends DeployMessage
- case class KillExecutor(masterUrl: String, appId: String, execId: Int)
extends DeployMessage
+ case class KillExecutor(
+ masterUrl: String,
+ appId: ApplicationId,
+ execId: Int)
+ extends DeployMessage
--- End diff --
I've modified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327689
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -98,10 +96,23 @@ private[spark] class MetricsSystem private (
sinks.foreach(_.report())
}
+ def buildRegistryName(source: Source) = {
+for {
+ appName - conf.getOption(spark.app.name)
--- End diff --
I've removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327723
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -98,10 +96,23 @@ private[spark] class MetricsSystem private (
sinks.foreach(_.report())
}
+ def buildRegistryName(source: Source) = {
+for {
+ appName - conf.getOption(spark.app.name)
+ appId - conf.getOption(spark.app.id)
+ executorId - conf.getOption(spark.executor.id)
+} yield {
+ MetricRegistry.name(appId, appName, executorId, source.sourceName)
+}
+ }.getOrElse {
+MetricRegistry.name(source.sourceName)
--- End diff --
I've added logging a warning logic.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327737
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -43,24 +43,26 @@ import org.apache.spark.util.{FileLogger, JsonProtocol,
Utils}
* spark.eventLog.buffer.kb - Buffer size to use when writing to output
streams
*/
private[spark] class EventLoggingListener(
-appName: String,
+appId: ApplicationId,
+logBaseDir: String,
sparkConf: SparkConf,
hadoopConf: Configuration)
extends SparkListener with Logging {
import EventLoggingListener._
- def this(appName: String, sparkConf: SparkConf) =
-this(appName, sparkConf,
SparkHadoopUtil.get.newConfiguration(sparkConf))
+ def this(appId: ApplicationId, logBaseDir: String, sparkConf: SparkConf)
=
+this(appId, logBaseDir, sparkConf,
SparkHadoopUtil.get.newConfiguration(sparkConf))
private val shouldCompress =
sparkConf.getBoolean(spark.eventLog.compress, false)
private val shouldOverwrite =
sparkConf.getBoolean(spark.eventLog.overwrite, false)
private val testing = sparkConf.getBoolean(spark.eventLog.testing,
false)
private val outputBufferSize =
sparkConf.getInt(spark.eventLog.buffer.kb, 100) * 1024
- private val logBaseDir = sparkConf.get(spark.eventLog.dir,
DEFAULT_LOG_DIR).stripSuffix(/)
- private val name = appName.replaceAll([ :/],
-).replaceAll([${}'\], _)
-.toLowerCase + - + System.currentTimeMillis
- val logDir = Utils.resolveURI(logBaseDir) + / + name.stripSuffix(/)
+ val logDir = EventLoggingListener.getLogDirName(logBaseDir, appId)
+ private val name = {
--- End diff --
I've modified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2432#discussion_r18327761 --- Diff: core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala --- @@ -115,4 +116,7 @@ private[spark] class LocalBackend(scheduler: TaskSchedulerImpl, val totalCores: localActor ! StatusUpdate(taskId, state, serializedData) } + override def applicationId = appId + + --- End diff -- I've modified. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327771
--- Diff: core/src/test/java/org/apache/spark/ApplicationIdSuite.java ---
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the License); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark;
+
+import static org.hamcrest.CoreMatchers.*;
+import static org.junit.Assert.assertThat;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import org.apache.spark.ApplicationId;
+
+public class ApplicationIdSuite {
+
+ private ApplicationId appId1_1, appId1_2, appId1_3, appId2, appIdNull_1,
appIdNull_2;
+
+ @Before
+ public void setUp() {
+appId1_1 = new ApplicationId(appId1);
+appId1_2 = new ApplicationId(appId1);
+appId1_3 = new ApplicationId(new String(appId1));
+appId2 = new ApplicationId(appId2);
+appIdNull_1 = new ApplicationId(null);
+appIdNull_2 = new ApplicationId(null);
+ }
+
+ @Test
+ public void testEquality() {
+// When ID strings are same object, ApplicationIds are equivalent
+assertThat(appId1_1.toString(), is(sameInstance(appId1_2.toString(;
+assertThat(appId1_1.equals(appId1_2), is(true));
+
+// When ID strings are not same object but equivalent, ApplicationIds
are equivalent
+assertThat(appId1_1.toString(),
is(not(sameInstance(appId1_3.toString();
+assertThat(appId1_1.toString(), is(appId1_3.toString()));
+assertThat(appId1_1.equals(appId1_3), is(true));
+
+// When ID strings are not equivalent, ApplicationIds are not
equivalent
+assertThat(appId1_1.toString(), is(not(appId2.toString(;
+assertThat(appId1_1.equals(appId2), is(false));
+
+// When one of ApplicationIds has null ID, ApplicationIds are not
equivalent
+assertThat(appIdNull_1.toString(), is(nullValue()));
+assertThat(appIdNull_2.toString(), is(nullValue()));
+assertThat(appId1_1.equals(appIdNull_1), is(false));
+assertThat(appIdNull_1.equals(appIdNull_2), is(false));
+ }
+
+ @Test
+ public void testEqualsMethodSpecification() {
+// Test reflexivity rule
+assertThat(appId1_1.equals(appId1_1), is(true));
+
+// Test transitivity rule
+assertThat(appId1_1.equals(appId1_2), is(true));
+assertThat(appId1_2.equals(appId1_3), is(true));
+assertThat(appId1_1.equals(appId1_3), is(true));
+
+// Test symmetric rule
+assertThat(appId1_2.equals(appId1_1), is(true));
+assertThat(appId1_3.equals(appId1_2), is(true));
+assertThat(appId1_3.equals(appId1_1), is(true));
+
+// Test consistency rule
+assertThat(appId1_1.equals(appId1_2), is(appId1_1.equals(appId1_2)));
+assertThat(appId1_3.equals(appId2), is(appId1_3.equals(appId2)));
+
+// Test comparision with null
+assertThat(appId1_1.equals(null), is(false));
+ }
+
+}
--- End diff --
I've removed this file.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/2432#discussion_r18327753 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala --- @@ -76,6 +76,8 @@ private[spark] class CoarseMesosSchedulerBackend( var nextMesosTaskId = 0 + @volatile var appId: FrameworkID = _ --- End diff -- I've modified. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327744
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulerBackend.scala ---
@@ -17,12 +17,16 @@
package org.apache.spark.scheduler
+import org.apache.spark.ApplicationId
+
/**
* A backend interface for scheduling systems that allows plugging in
different ones under
* TaskSchedulerImpl. We assume a Mesos-like model where the application
gets resource offers as
* machines become available and can launch tasks on them.
*/
private[spark] trait SchedulerBackend {
+ private val appId = new ApplicationId(System.currentTimeMillis.toString)
--- End diff --
I've modified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on a diff in the pull request:
https://github.com/apache/spark/pull/2432#discussion_r18327779
--- Diff:
yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala
---
@@ -48,6 +48,17 @@ private[spark] class YarnClusterSchedulerBackend(
totalRegisteredExecutors.get() = totalExpectedExecutors *
minRegisteredRatio
}
- override def applicationId(): Option[String] =
sc.getConf.getOption(spark.yarn.app.id)
+ override def applicationId() =
+/**
+ * In YARN Cluster mode, spark.yarn.app.id is expect to be set
+ * before user application is launched.
+ * So, if spark.yarn.app.id is not set, it is something wrong.
+ */
+ sc.getConf.getOption(spark.yarn.app.id)
+.map(strAppId = new ApplicationId(strAppId))
+.getOrElse {
+ logError(Application ID is not set.)
+ super.applicationId
+ }
--- End diff --
I've modified.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3007][SQL] (This is a debugging PR to t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2626#issuecomment-57599241 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21190/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57599362 test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57599696 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21191/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57599949 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21192/consoleFull) for PR 2432 at commit [`389090d`](https://github.com/apache/spark/commit/389090d33190acaaa72499d81cacce732ec34cf9). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57600075 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21192/consoleFull) for PR 2432 at commit [`389090d`](https://github.com/apache/spark/commit/389090d33190acaaa72499d81cacce732ec34cf9). * This patch **fails** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57600086 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21192/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3763] The example of building with sbt ...
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/2627 [SPARK-3763] The example of building with sbt should be sbt assembly instead of sbt compile In building-spark.md, there are some examples for making assembled package with maven but the example for building with sbt is only about for compiling. You can merge this pull request into a Git repository by running: $ git pull https://github.com/sarutak/spark SPARK-3763 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2627 commit fadb990366fe9ee674dd7c2e9e732df1a34155b7 Author: Kousuke Saruta saru...@oss.nttdata.co.jp Date: 2014-10-02T08:46:10Z Modified the example to build with sbt in building-spark.md --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3763] The example of building with sbt ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2627#issuecomment-57600468 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21193/consoleFull) for PR 2627 at commit [`fadb990`](https://github.com/apache/spark/commit/fadb990366fe9ee674dd7c2e9e732df1a34155b7). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DEPLOY] SPARK-3759: Return the exit code of t...
GitHub user ericeijkelenboom opened a pull request: https://github.com/apache/spark/pull/2628 [DEPLOY] SPARK-3759: Return the exit code of the driver process SparkSubmitDriverBootstrapper.scala now returns the exit code of the driver process, instead of always returning 0. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ericeijkelenboom/spark master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2628.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2628 commit cc4a571c7df7260b134166486166fb8bcf98b7f4 Author: Eric Eijkelenboom e...@userreport.com Date: 2014-10-01T14:39:58Z Return the exit code of the driver process --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [DEPLOY] SPARK-3759: Return the exit code of t...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2628#issuecomment-57600708 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57601907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21194/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [WIP][SPARK-1405][MLLIB] topic modeling on Gra...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2388#issuecomment-57602181 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21195/consoleFull) for PR 2388 at commit [`4e17606`](https://github.com/apache/spark/commit/4e17606b3508a9208f42a6304c19ca970e07dbea). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: SPARK-3223 runAsSparkUser cannot change HDFS w...
Github user jongyoul commented on the pull request: https://github.com/apache/spark/pull/2126#issuecomment-57602773 +1 @tnachen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57602748 test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57603370 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21196/consoleFull) for PR 2432 at commit [`f9b6fb3`](https://github.com/apache/spark/commit/f9b6fb36a6e0b385a1fe5a769295c133e519f8ce). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3007][SQL] (This is a debugging PR to t...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2626#issuecomment-57603751 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/254/consoleFull) for PR 2626 at commit [`e69ce88`](https://github.com/apache/spark/commit/e69ce883ee9d337a81d4aae3a63943937f771e84). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3758] [Windows] Wrong EOL character in ...
Github user tsudukim commented on the pull request: https://github.com/apache/spark/pull/2612#issuecomment-57606210 Generally, using LF as EOL character of *.cmd files may cause some troubles. For example, when the *.cmd file includes LF and multibyte character, some characters of head of line might be removed internally for some reason. The problem @sarutak mentioned seems to be same as this. In another case, goto may go to wrong place. This problem occured once in ruby-lang project. Their build script for Windows happened to be LF, and they faced strange behaviour of cmd.exe. https://bugs.ruby-lang.org/issues/10145 I know *.cmd with LF runs seemingly well in many cases, but sometimes cause inexplicable trouble because LF is defenetely not the proper EOL character of Windows *.cmd files. So if possible, I think we should use CRLF as EOL to avoid such trouble. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3755][Core] Do not bind port 1 - 1024 t...
Github user scwf commented on a diff in the pull request:
https://github.com/apache/spark/pull/2623#discussion_r18330540
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1437,7 +1437,13 @@ private[spark] object Utils extends Logging {
val serviceString = if (serviceName.isEmpty) else s '$serviceName'
for (offset - 0 to maxRetries) {
// Do not increment port if startPort is 0, which is treated as a
special port
- val tryPort = if (startPort == 0) startPort else (startPort +
offset) % 65536
+ val _tryPort = if (startPort == 0) startPort else (startPort +
offset) % 65536
--- End diff --
I think it is ok. By this PR even root user can not use port 1- 1024(leave
them to system) in spark. An important point here is we can not change the port
before, for example the old app web ui is 4040, we can not change it. But if
use ```... else ((startPort + offset - 1024) % (65536 - 1024)) + 1024``` , the
new web ui port will be 4040+1024
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3763] The example of building with sbt ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2627#issuecomment-57607461 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21193/consoleFull) for PR 2627 at commit [`fadb990`](https://github.com/apache/spark/commit/fadb990366fe9ee674dd7c2e9e732df1a34155b7). * This patch **passes** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3763] The example of building with sbt ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2627#issuecomment-57607465 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21193/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57608532 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21196/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-3377] [SPARK-3610] Metrics can be accid...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2432#issuecomment-57608529 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21196/consoleFull) for PR 2432 at commit [`f9b6fb3`](https://github.com/apache/spark/commit/f9b6fb36a6e0b385a1fe5a769295c133e519f8ce). * This patch **fails** unit tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
