[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150690783 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150690438 Heh, I think this was just an oversight from the long-ago rename from `spark.*` to `org.apache.spark.*` as it exists in the initial revision: https://github.com/apache/sp

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-150689641 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1537: pure service API and test service

2015-10-23 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9182#issuecomment-150689362 -Will do. Then I can combine everything nicely. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-11020] [core] Wait for HDFS to leave sa...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9043#issuecomment-150689561 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-11265 hive tokens

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9232#issuecomment-150688788 **[Test build #44257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44257/consoleFull)** for PR 9232 at commit [`9713cf5`](https://gith

[GitHub] spark pull request: SPARK-11265 hive tokens

2015-10-23 Thread steveloughran
Github user steveloughran commented on the pull request: https://github.com/apache/spark/pull/9232#issuecomment-150688417 Updated patch 1. Moves all arg validation up before any hive class reflection 1. gets all hive class methods before attempting to call any 1. calls `Hive

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150685994 Yes, it was missing "org.apache" and when running the TestClient, the master continuously tries and fails to start an executor. --- If your project is set up for it

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150684436 Is the issue that it's missing "org.apache" or something -- does it cause a problem? --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42912068 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -277,10 +311,20 @@ public long cleanUpAllAllocatedMemory() {

[GitHub] spark pull request: SPARK-11265 hive tokens

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9232#issuecomment-150684298 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-11265 hive tokens

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9232#issuecomment-150684229 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10342] [SQL] [WIP] Cooperative memory m...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9241#issuecomment-150684162 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10342] [SQL] [WIP] Cooperative memory m...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9241#issuecomment-150684105 **[Test build #44242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44242/consoleFull)** for PR 9241 at commit [`0c77c94`](https://git

[GitHub] spark pull request: [SPARK-10342] [SQL] [WIP] Cooperative memory m...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9241#issuecomment-150684163 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10847] [SQL] [PySpark] Pyspark - DataFr...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8969#issuecomment-150682844 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10847] [SQL] [PySpark] Pyspark - DataFr...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8969#issuecomment-150682712 **[Test build #44246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44246/consoleFull)** for PR 8969 at commit [`5524a92`](https://git

[GitHub] spark pull request: [SPARK-10847] [SQL] [PySpark] Pyspark - DataFr...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8969#issuecomment-150682842 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10266][Documentation, ML] Fixed @Since ...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8729#issuecomment-150680245 **[Test build #44256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44256/consoleFull)** for PR 8729 at commit [`4cbb69f`](https://gith

[GitHub] spark pull request: [SPARK-10266][Documentation, ML] Fixed @Since ...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8729#issuecomment-150679296 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10266][Documentation, ML] Fixed @Since ...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8729#issuecomment-150679315 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150677936 **[Test build #44255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44255/consoleFull)** for PR 9127 at commit [`e56d039`](https://gith

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150677184 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150677135 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150676901 **[Test build #44253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44253/consoleFull)** for PR 9127 at commit [`04ec429`](https://git

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150676906 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150676909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11274][SQL] Text data source support fo...

2015-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9240 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150676756 **[Test build #44254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44254/consoleFull)** for PR 9255 at commit [`1fab574`](https://gith

[GitHub] spark pull request: [SPARK-11274][SQL] Text data source support fo...

2015-10-23 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9240#issuecomment-150676331 LGTM. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150675826 **[Test build #44253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44253/consoleFull)** for PR 9127 at commit [`04ec429`](https://gith

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150675162 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150675184 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11274][SQL] Text data source support fo...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9240#issuecomment-150674484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11274][SQL] Text data source support fo...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9240#issuecomment-150674483 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11274][SQL] Text data source support fo...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9240#issuecomment-150674341 **[Test build #44240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44240/consoleFull)** for PR 9240 at commit [`740c263`](https://git

[GitHub] spark pull request: Fix typos

2015-10-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9250#issuecomment-150674296 Yeah that's fine, I think Scala files may be too much to bear. I just use the spell-checker in the IDE as I find it a little easier to manage. Docs are indeed higher valu

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150674186 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150674234 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150674196 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42906959 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -42,8 +60,10 @@ private[spark] abstract class MemoryManager extends Loggin

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42906919 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -115,6 +205,35 @@ private[spark] abstract class MemoryManager extends Logg

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42906899 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -155,4 +274,44 @@ private[spark] abstract class MemoryManager extends Logg

[GitHub] spark pull request: [SPARK-10266][Documentation, ML] Fixed @Since ...

2015-10-23 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/8729#issuecomment-150673626 You don't need to include it. You should fix style errors which are pointed out by `dev/lint-scala`. According to the Jenkins result, there are some `White space at end

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42906908 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -102,9 +113,88 @@ private[spark] abstract class MemoryManager extends Logg

[GitHub] spark pull request: [SPARK-11253][SQL] reset all accumulators in p...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9215#issuecomment-15067 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11253][SQL] reset all accumulators in p...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9215#issuecomment-150673263 **[Test build #44237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44237/consoleFull)** for PR 9215 at commit [`4ff8912`](https://git

[GitHub] spark pull request: [SPARK-11253][SQL] reset all accumulators in p...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9215#issuecomment-150673330 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42906605 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -277,10 +311,20 @@ public long cleanUpAllAllocatedMemory() {

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150672412 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150672339 **[Test build #44239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44239/consoleFull)** for PR 9255 at commit [`4d0a593`](https://git

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150672415 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42906184 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -316,9 +313,13 @@ private long freeMemory() { long mem

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150671967 **[Test build #44252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44252/consoleFull)** for PR 9220 at commit [`dc75828`](https://gith

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42905961 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala --- @@ -87,6 +88,7 @@ import org.apache.spark.storage.{BlockId, DiskBlo

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42905857 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala --- @@ -87,6 +88,7 @@ import org.apache.spark.storage.{BlockId, DiskBlo

[GitHub] spark pull request: [SPARK-10266][Documentation, ML] Fixed @Since ...

2015-10-23 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/8729#discussion_r42905702 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala --- @@ -36,30 +36,31 @@ import org.apache.spark.sql.DataFrame

[GitHub] spark pull request: [SPARK-10266][Documentation, ML] Fixed @Since ...

2015-10-23 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/8729#discussion_r42905531 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -23,9 +23,13 @@ import breeze.linalg.{DenseVector => BDV} imp

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150670422 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150670449 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9220#discussion_r42904851 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -266,6 +266,7 @@ object SparkSubmit { } } +

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-23 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150669670 LGTM, will merge once tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5966][WIP]

2015-10-23 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9220#issuecomment-150669570 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10582] If a new AM restarts, the total ...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8737#issuecomment-150669518 @KaiXinXiaoLei were you able to address the comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150669424 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150669271 **[Test build #44235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44235/consoleFull)** for PR 9214 at commit [`4145651`](https://git

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150669422 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150668921 **[Test build #44251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44251/consoleFull)** for PR 9214 at commit [`2089e12`](https://gith

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150668648 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150668613 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r42904139 --- Diff: core/src/test/scala/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriterSuite.scala --- @@ -121,12 +140,16 @@ class BypassMergeSortShuffleWrit

[GitHub] spark pull request: [SPARK-11271][SPARK-11016][Core] Use Spark Bit...

2015-10-23 Thread lemire
Github user lemire commented on the pull request: https://github.com/apache/spark/pull/9243#issuecomment-150668521 @rxin There are definitively cases where attempting to use compressed bitmaps is wasteful. For example, if you have a small universe size. E.g., your bitmaps re

[GitHub] spark pull request: [SPARK-8029][core][wip] first successful shuff...

2015-10-23 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/9214#issuecomment-150667835 @JoshRosen @rxin I'm going to see if I can do a little cleanup still, but the functionality is there now. However, I did realize there are some open questions in the log

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9127#issuecomment-150667335 The high level refactoring looks good. Once you make it pass tests and address the remaining comments (all of which are minor) I will merge this. Thanks for all your

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150667131 **[Test build #44250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44250/consoleFull)** for PR 9255 at commit [`1fab574`](https://gith

[GitHub] spark pull request: [SPARK-11194] [SQL] [BRANCH-1.5] [WIP] Use Mut...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9171#issuecomment-150667081 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11194] [SQL] [BRANCH-1.5] [WIP] Use Mut...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9171#issuecomment-150666902 [Test build #44233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44233/console) for PR 9171 at commit [`7951df1`](https://github.

[GitHub] spark pull request: [SPARK-11194] [SQL] [BRANCH-1.5] [WIP] Use Mut...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9171#issuecomment-150667080 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42902990 --- Diff: core/src/test/scala/org/apache/spark/shuffle/ShuffleMemoryManagerSuite.scala --- @@ -1,326 +0,0 @@ -/* - * Licensed to the Apache Softwar

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42902844 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala --- @@ -87,6 +88,7 @@ import org.apache.spark.storage.{BlockId, DiskBl

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42902688 --- Diff: core/src/main/scala/org/apache/spark/SparkEnv.scala --- @@ -70,10 +69,7 @@ class SparkEnv ( val httpFileServer: HttpFileServer,

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150665518 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11287] Fixed class name to properly sta...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9255#issuecomment-150665545 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42902567 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -316,9 +313,13 @@ private long freeMemory() { long me

[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-150665299 **[Test build #44248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44248/consoleFull)** for PR 9186 at commit [`0ac3df6`](https://gith

[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-150665419 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-150665416 **[Test build #44248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44248/consoleFull)** for PR 9186 at commit [`0ac3df6`](https://git

[GitHub] spark pull request: [SPARK-10984] [WIP] Simplify *MemoryManager cl...

2015-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9127#discussion_r42902440 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -277,10 +311,20 @@ public long cleanUpAllAllocatedMemory() {

[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-150665420 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-8992] [SQL] Add pivot to dataframe api

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7841#issuecomment-150665042 **[Test build #44249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44249/consoleFull)** for PR 7841 at commit [`f2827ea`](https://gith

[GitHub] spark pull request: [SPARK-8992] [SQL] Add pivot to dataframe api

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7841#issuecomment-150664411 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8992] [SQL] Add pivot to dataframe api

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7841#issuecomment-150664375 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10641][WIP][SQL] Add Skewness and Kurto...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9003#issuecomment-150662563 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-8992] [SQL] Add pivot to dataframe api

2015-10-23 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/7841#issuecomment-150663288 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-150663329 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6328] [Python] Python API for Streaming...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9186#issuecomment-150663302 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11271][SPARK-11016][Core] Use Spark Bit...

2015-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9243#issuecomment-150662823 **[Test build #44247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44247/consoleFull)** for PR 9243 at commit [`d30ec97`](https://gith

[GitHub] spark pull request: SPARK-11265 hive tokens

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9232#issuecomment-150662642 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: SPARK-11265 hive tokens

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9232#issuecomment-150662640 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10641][WIP][SQL] Add Skewness and Kurto...

2015-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9003#issuecomment-150662564 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

<    1   2   3   4   5   6   7   8   >