[GitHub] spark issue #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryProviders...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15490 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryProviders...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15490 **[Test build #67148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67148/consoleFull)** for PR 15490 at commit [`62e0f65`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15218: [SPARK-17637][Scheduler]Packed scheduling for Spark task...

2016-10-18 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/15218 @wangmiao1981 Thanks for reviewing this. I will open another PR solving these comments soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9 **[Test build #67156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67156/consoleFull)** for PR 9 at commit [`e529972`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrainedSche...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15481 **[Test build #67155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67155/consoleFull)** for PR 15481 at commit [`af6072a`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrainedSche...

2016-10-18 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/15481 ok, i will revert to the initial commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #15525: [SPARK-17985][CORE] Bump commons-lang3 version to 3.5.

2016-10-18 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15525 @rxin I think the failure is not related to this PR. The build for hadoop-2.2 is failing after reverted. https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.2/1948/consol

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15476 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15476 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67149/ Test PASSed. ---

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15476 **[Test build #67149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67149/consoleFull)** for PR 15476 at commit [`41e7e58`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15538 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67150/ Test FAILed. ---

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-18 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r83981625 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1486,10 +1485,10 @@ private[spark] object Utils extends Logging { val gzInput

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15538 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrainedSche...

2016-10-18 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/15481 Seems it could be changed to `send` instead. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15538 **[Test build #67150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67150/consoleFull)** for PR 15538 at commit [`6101b83`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15398 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15398 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67147/ Test PASSed. ---

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15398 **[Test build #67147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67147/consoleFull)** for PR 15398 at commit [`85a3a38`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15471: [SPARK-17919] Make timeout to RBackend configurable in S...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15471 **[Test build #67153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67153/consoleFull)** for PR 15471 at commit [`fcc3376`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15434: [SPARK-17873][SQL] ALTER TABLE RENAME TO should allow us...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15434 **[Test build #67154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67154/consoleFull)** for PR 15434 at commit [`65c1885`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15471: [SPARK-17919] Make timeout to RBackend configurable in S...

2016-10-18 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15471 @shivaram it was indeed my fault. I did not run local tests after I added the heartbeat. I am now using +1 for heartbeat. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark issue #15528: [SPARK-17899][SQL][follow-up] debug mode should work for...

2016-10-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15528 @rxin this depends on how often we will debug for branch 2.0 in the future --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark issue #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67146/ Test PASSed. ---

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-18 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/15539#discussion_r83980885 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1486,10 +1485,10 @@ private[spark] object Utils extends Logging { val gzInp

[GitHub] spark issue #15434: [SPARK-17873][SQL] ALTER TABLE RENAME TO should allow us...

2016-10-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15434 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15537 **[Test build #67146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67146/consoleFull)** for PR 15537 at commit [`fd0b076`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67145/ Test PASSed. ---

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67145/consoleFull)** for PR 15530 at commit [`1679077`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactore...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15530 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-18 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15421 Thanks @wangmiao1981. I thin it is best to file a separate JIRA for this issue. Thanks a lot for catching it. --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15530 I am merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15535 I am merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67144/ Test PASSed. ---

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67144/consoleFull)** for PR 15535 at commit [`4cac044`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #3361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3361/consoleFull)** for PR 15535 at commit [`4cac044`](https://github.com/apache/spark/commit

[GitHub] spark issue #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to run in th...

2016-10-18 Thread jodersky
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/15338 no objections from me ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #15539: [SPARK-17994] [SQL] Add back a file status cache for cat...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15539 **[Test build #67152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67152/consoleFull)** for PR 15539 at commit [`947c32d`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #15539: [SPARK-17994] [SQL] Add back a file status cache ...

2016-10-18 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15539 [SPARK-17994] [SQL] Add back a file status cache for catalog tables ## What changes were proposed in this pull request? In SPARK-16980, we removed the full in-memory cache of table partitions

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-18 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r83972911 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala --- @@ -75,9 +75,11 @@ class KryoSerializerSuite extends SparkFunSuite wit

[GitHub] spark issue #12913: [SPARK-928][CORE] Add support for Unsafe-based serialize...

2016-10-18 Thread mateiz
Github user mateiz commented on the issue: https://github.com/apache/spark/pull/12913 Looks pretty good overall! I made two small comments but it seems worthwhile to add in and it's not a huge change. --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-18 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r83971706 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala --- @@ -75,9 +75,11 @@ class KryoSerializerSuite extends SparkFunSuite wit

[GitHub] spark pull request #12913: [SPARK-928][CORE] Add support for Unsafe-based se...

2016-10-18 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/12913#discussion_r83971396 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -78,8 +79,14 @@ class KryoSerializer(conf: SparkConf) .filter(!

[GitHub] spark issue #15538: [SPARK-17993][SQL] Fix Parquet log output redirection

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15538 **[Test build #67150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67150/consoleFull)** for PR 15538 at commit [`6101b83`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15536 **[Test build #67151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67151/consoleFull)** for PR 15536 at commit [`bf0117b`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #15538: [SPARK-17993][SQL] Fix Parquet log output redirec...

2016-10-18 Thread mallman
GitHub user mallman opened a pull request: https://github.com/apache/spark/pull/15538 [SPARK-17993][SQL] Fix Parquet log output redirection (Link to Jira issue: https://issues.apache.org/jira/browse/SPARK-17993) ## What changes were proposed in this pull request? PR

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15536 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15536 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67142/ Test FAILed. ---

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15536 **[Test build #67142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67142/consoleFull)** for PR 15536 at commit [`bf0117b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15536 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-18 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15421 Master Branch: 16/10/18 16:10:15 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1) java.lang.NegativeArraySizeException at org.apache.spark.api.r.SerDe$.readStringBy

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15476 **[Test build #67149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67149/consoleFull)** for PR 15476 at commit [`41e7e58`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #15396: [SPARK-14804][Spark][Graphx] Fix checkpointing of...

2016-10-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15396#discussion_r83968123 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1589,7 +1589,8 @@ abstract class RDD[T: ClassTag]( * This is introduced as an alias

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67139/ Test PASSed. ---

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15530 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryProviders...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15490 **[Test build #67148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67148/consoleFull)** for PR 15490 at commit [`62e0f65`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67139/consoleFull)** for PR 15530 at commit [`0257b9e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15532: [SPARK-17989][SQL] Check ascendingOrder type in sort_arr...

2016-10-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15532 Thank you both. Will adress the comments soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15490: [SPARK-10541] [Web UI] Allow ApplicationHistoryProviders...

2016-10-18 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15490 The dir is actually already displayed so I just added a few words to clarify that the current setting is displayed above. ![screen shot 2016-10-18 at 3 48 39 pm](https://cloud.githubuserc

[GitHub] spark issue #15398: [SPARK-17647][SQL] Fix backslash escaping in 'LIKE' patt...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15398 **[Test build #67147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67147/consoleFull)** for PR 15398 at commit [`85a3a38`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source...

2016-10-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/14553#discussion_r83965312 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -336,17 +342,27 @@ class StreamExecution( }

[GitHub] spark pull request #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source...

2016-10-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/14553#discussion_r83965244 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala --- @@ -336,17 +342,27 @@ class StreamExecution( }

[GitHub] spark pull request #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source...

2016-10-18 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/14553#discussion_r83953473 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/Source.scala --- @@ -30,16 +30,30 @@ trait Source { /** Returns the schem

[GitHub] spark issue #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation...

2016-10-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15537 LGTM pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2016-10-18 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15496 @itg-abby I see what you're going for, but I don't think it's a great idea in general. For many operations on sparse vectors, we should not materialize the sparse vector as dense. I'm not sure if it'

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-18 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83963294 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -250,24 +263,26 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15537 **[Test build #67146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67146/consoleFull)** for PR 15537 at commit [`fd0b076`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-18 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83962588 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,151 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark pull request #15537: [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 comp...

2016-10-18 Thread loneknightpy
GitHub user loneknightpy opened a pull request: https://github.com/apache/spark/pull/15537 [SPARK-17711] [test-hadoop2.2] Fix hadoop2.2 compilation error ## What changes were proposed in this pull request? Fix hadoop2.2 compilation error. ## How was this patch teste

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-18 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15421 @wangmiao1981 would you please also test the master branch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark issue #15530: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15530 **[Test build #67145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67145/consoleFull)** for PR 15530 at commit [`1679077`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #15218: [SPARK-17637][Scheduler]Packed scheduling for Spa...

2016-10-18 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/15218#discussion_r83961053 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskAssigner.scala --- @@ -0,0 +1,154 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] spark issue #15475: [SPARK-17991] [SQL] Enable metastore partition pruning b...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15475 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67138/ Test PASSed. ---

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #3361 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3361/consoleFull)** for PR 15535 at commit [`4cac044`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15475: [SPARK-17991] [SQL] Enable metastore partition pruning b...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15475 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15475: [SPARK-17991] [SQL] Enable metastore partition pruning b...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15475 **[Test build #67138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67138/consoleFull)** for PR 15475 at commit [`2581380`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15535 I cancelled previous jenkins tests, hence the failures. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67141/ Test FAILed. ---

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67143/consoleFull)** for PR 15535 at commit [`bb1b234`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67143/ Test FAILed. ---

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15535 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67144/consoleFull)** for PR 15535 at commit [`4cac044`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15285 This one breaks hadoop-2.2 builds: https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.2/1951/console --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark issue #10154: [SPARK-12149] [Web UI] Executor UI improvement suggestio...

2016-10-18 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/10154 @rxin The reds only show up on failed tasks or a high gc time, what would you expect from each color? Also this was a while ago (and one of my first prs) so I'm willing to revisit how this was don

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67143/consoleFull)** for PR 15535 at commit [`bb1b234`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #10154: [SPARK-12149] [Web UI] Executor UI improvement suggestio...

2016-10-18 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/10154 So I looked at the UI today and I have to say the color code is *extremely confusing*. On a normal cluster I saw a bunch of reds, which is usually reserved for errors. (I'm going to leave

[GitHub] spark pull request #15445: [SPARK-17817][PySpark][FOLLOWUP] PySpark RDD Repa...

2016-10-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15445 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15445: [SPARK-17817][PySpark][FOLLOWUP] PySpark RDD Repartition...

2016-10-18 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15445 @viirya That's good point, javaToPython can only used for known types that could deserialized in Java (for example, some types in sql and ml), this PR make better sense. Merging this into ma

[GitHub] spark issue #15398: [SPARK-17647][SQL][WIP] Fix backslash escaping in 'LIKE'...

2016-10-18 Thread jodersky
Github user jodersky commented on the issue: https://github.com/apache/spark/pull/15398 Agreed, I'll update the regex parsing to copy Hive's behaviour --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark issue #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrainedSche...

2016-10-18 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15481 @zsxwing Ah, then simply making it send() instead of askWithRetry() should do, no ? That was actually in the initial PR - I was not sure if we want to change the behavior from askWithRetry to se

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15536 This is a purely aesthetic update as the lines already show the exact time, but given jobs are shown as ranges the executor bubbles center alignment on the line was confusing visually for some.

[GitHub] spark issue #15536: [SPARK-13275] [Web UI] Visually clarified executors star...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15536 **[Test build #67142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67142/consoleFull)** for PR 15536 at commit [`bf0117b`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15274: [SPARK-17699] Support for parsing JSON string columns

2016-10-18 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15274 @DanielMe oh, I see. `get_json_object` will not parse json array. You need to have a UDF to do that for Spark 1.6. --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark issue #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrainedSche...

2016-10-18 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15481 > reset is called in YarnSchedulerEndpoint which ideally should not be a blocking action. I'm wondering if we can also fix this. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request #15536: [SPARK-13275] [Web UI] Visually clarified executo...

2016-10-18 Thread ajbozarth
GitHub user ajbozarth opened a pull request: https://github.com/apache/spark/pull/15536 [SPARK-13275] [Web UI] Visually clarified executors start time in timeline ## What changes were proposed in this pull request? Updated the Executors added/removed bubble in the time line

[GitHub] spark issue #15481: [SPARK-17929] [CORE] Fix deadlock when CoarseGrainedSche...

2016-10-18 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/15481 @scwf I think the initial fix with a small change might be sufficient. What I meant was something like this : ``` protected def reset(): Unit = { val executors = synchroniz

[GitHub] spark issue #15535: [SPARK-17731][SQL][STREAMING][FOLLOWUP] Refactored Strea...

2016-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15535 **[Test build #67140 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67140/consoleFull)** for PR 15535 at commit [`0fe56d2`](https://github.com/apache/spark/commit/

<    1   2   3   4   5   6   >