[GitHub] spark pull request: [SPARK-12583][Mesos] Mesos shuffle service: Do...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11272#discussion_r53674006 --- Diff: network/shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/BlockTransferMessage.java --- @@ -40,7 +41,8 @@ /**

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2016-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8872 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-13013][Docs] Replace example code in ml...

2016-02-22 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/6#issuecomment-187359474 @keypointt Most of the codes look good. However, there are still 3 major issues to solve: 1. According to PEP8 code style of Python, the imports should be

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53684304 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13329] [SQL] considering output for sta...

2016-02-22 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/11210#issuecomment-187363949 @andrewor14 I had update the description to explain more on change of defaultSize --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53685356 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53685204 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53687324 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53687313 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53688765 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -995,21 +1003,10 @@ private[spark] class BlockManager( data: ()

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53688782 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -805,13 +814,12 @@ private[spark] class BlockManager( }

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53681407 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -90,22 +90,22 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53682420 --- Diff: core/src/main/scala/org/apache/spark/network/BlockDataManager.scala --- @@ -31,6 +31,14 @@ trait BlockDataManager { /** *

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53682337 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -90,22 +90,22 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53684085 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13329] [SQL] considering output for sta...

2016-02-22 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11210#discussion_r53684255 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -63,36 +63,40 @@ class JoinSuite extends QueryTest with SharedSQLContext {

[GitHub] spark pull request: [SPARK-12941][SQL][MASTER] Spark-SQL JDBC Orac...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11306#issuecomment-187395005 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13435] [MLlib] Add Weighted Cohen's kap...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11303#issuecomment-187395032 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13266][PYSPARK] None read/writer option...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11305#issuecomment-187395020 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Branch 1.4

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11302#issuecomment-187395048 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12583][Mesos] Mesos shuffle service: Do...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11272#issuecomment-187320271 I would like to backport a variant of this patch into 1.6.1. This was clearly working in 1.5 but then became unusable in 1.6. This means people who upgrade to 1.6

[GitHub] spark pull request: [SPARK-12583][Mesos] Mesos shuffle service: Do...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11272#discussion_r53674382 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/MesosExternalShuffleService.scala --- @@ -17,69 +17,88 @@ package

[GitHub] spark pull request: [SPARK-11624][SPARK-11972][SQL]fix commands th...

2016-02-22 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9589#discussion_r53677494 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala --- @@ -288,8 +288,11 @@ private[hive] class

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53683536 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12546][SQL] Change default number of op...

2016-02-22 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/11308 [SPARK-12546][SQL] Change default number of open parquet files A common problem that users encounter with Spark 1.6.0 is that writing to a partitioned parquet table OOMs. The root cause is that

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53683602 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12546][SQL] Change default number of op...

2016-02-22 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/11308#issuecomment-187361458 /cc @nongli @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53683591 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -136,7 +147,13 @@ private[spark] class CacheManager(blockManager: BlockManager)

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53686068 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53686131 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12546][SQL] Change default number of op...

2016-02-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11308#issuecomment-187367277 Do we have any off by one error? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12546][SQL] Change default number of op...

2016-02-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11308#issuecomment-187367376 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53696758 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12941][SQL][MASTER] Spark-SQL JDBC Orac...

2016-02-22 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11306#issuecomment-187323211 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11229#issuecomment-187323397 LGTM, unfortunately this no longer merges. Can you resolve the conflicts? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-13437][SQL] Introduce InternalColumn an...

2016-02-22 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/11307 [SPARK-13437][SQL] Introduce InternalColumn and MutableColumn ## What changes were proposed in this pull request? This PR introduce ``InternalColumn`` and ``MutableColumn`` to abstract a

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53683409 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13013][Docs] Replace example code in ml...

2016-02-22 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r53683493 --- Diff: examples/src/main/python/mllib/streaming_k_means_example.py --- @@ -0,0 +1,53 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53684788 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13422][SQL] Use HashedRelation instead ...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11291#issuecomment-187367833 **[Test build #2566 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2566/consoleFull)** for PR 11291 at commit

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53687486 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13013][Docs] Replace example code in ml...

2016-02-22 Thread keypointt
Github user keypointt commented on the pull request: https://github.com/apache/spark/pull/6#issuecomment-187385620 Sure I'll fix all these, thanks a lot --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53695813 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13139][SQL] Create native DDL commands

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11048#issuecomment-187396766 **[Test build #51681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51681/consoleFull)** for PR 11048 at commit

[GitHub] spark pull request: [SPARK-13028] [ML] Add MaxAbsScaler to ML.feat...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10939#issuecomment-187396076 **[Test build #51683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51683/consoleFull)** for PR 10939 at commit

[GitHub] spark pull request: [SPARK-13376] [SQL] improve column pruning

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11256#issuecomment-187396600 **[Test build #51674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51674/consoleFull)** for PR 11256 at commit

[GitHub] spark pull request: [SPARK-13236] SQL Generation for Set Operation...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11195#issuecomment-187396282 **[Test build #51678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51678/consoleFull)** for PR 11195 at commit

[GitHub] spark pull request: [SPARK-13263] [SQL] SQL Generation Support for...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11148#issuecomment-187396329 **[Test build #51679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51679/consoleFull)** for PR 11148 at commit

[GitHub] spark pull request: [SPARK-13383][SQL] Keep broadcast hint after c...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11260#issuecomment-187396775 **[Test build #51672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51672/consoleFull)** for PR 11260 at commit

[GitHub] spark pull request: [SPARK-12617][PySpark]Move Py4jCallbackConnect...

2016-02-22 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10621#issuecomment-187328406 @sarathj if you want to upgrade py4j to 0.9.1, you can just check-pick this patch: https://github.com/zsxwing/spark/commit/a3e3e1755897d21247399a3bc40336bde8e1d8b3

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53681599 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -46,7 +48,14 @@ private[spark] class CacheManager(blockManager: BlockManager)

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r5368 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53694095 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53694268 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13080] [SQL] Implement new Catalog API ...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11293#discussion_r53671961 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveCatalog.scala --- @@ -0,0 +1,293 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12583][Mesos] Mesos shuffle service: Do...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11272#discussion_r53673482 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/MesosExternalShuffleService.scala --- @@ -17,69 +17,88 @@ package

[GitHub] spark pull request: [SPARK-12583][Mesos] Mesos shuffle service: Do...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11272#issuecomment-187329496 (Update: this will have to go into 1.6.2 since the 1.6.1 RC is being cut today already) --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12941][SQL][MASTER] Spark-SQL JDBC Orac...

2016-02-22 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11306#issuecomment-187329489 @thomastechs Thank you for the PR. Since the jar is not in maven repo and you have tested it locally, how about we ignore the test? Let's keep the test but just ignore

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53683331 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -46,7 +48,14 @@ private[spark] class CacheManager(blockManager: BlockManager)

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53685793 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53696226 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53681851 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -90,22 +90,22 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53683999 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13298][Core][UI]Escape "label" to avoid...

2016-02-22 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/11309#issuecomment-187368418 cc @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53688589 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -805,13 +814,12 @@ private[spark] class BlockManager( }

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53696069 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-13403: Pass hadoopConfiguration to HiveC...

2016-02-22 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/11273#issuecomment-187323694 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-13403: Pass hadoopConfiguration to HiveC...

2016-02-22 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/11273#issuecomment-187323664 Seems reasonable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53682666 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10780][ML][WIP] Add initial model to km...

2016-02-22 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/9#issuecomment-187365420 Ping @dbtsai Coming back? :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53684951 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-13298][Core][UI]Escape "label" to avoid...

2016-02-22 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/11309 [SPARK-13298][Core][UI]Escape "label" to avoid DAG being broken by some special character ## What changes were proposed in this pull request? When there are some special characters (e.g.,

[GitHub] spark pull request: [SPARK-13298][Core][UI]Escape "label" to avoid...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11309#issuecomment-187383283 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13327][SPARKR] Added parameter validati...

2016-02-22 Thread olarayej
Github user olarayej commented on the pull request: https://github.com/apache/spark/pull/11220#issuecomment-187318737 Jenkins, retest please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13328][Core]: Poor read performance for...

2016-02-22 Thread nezihyigitbasi
Github user nezihyigitbasi commented on the pull request: https://github.com/apache/spark/pull/11241#issuecomment-187319511 @andrewor14 addressed your comments, can you please take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-13266][PYSPARK] None read/writer option...

2016-02-22 Thread mathieulongtin
Github user mathieulongtin commented on the pull request: https://github.com/apache/spark/pull/11305#issuecomment-187323930 Sorry but I already spent enough time tracking and fixing this bug --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-10749][MESOS] Support multiple roles wi...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8872#issuecomment-187324256 No worries, thanks for addressing them. I'm merging this into master. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-187327281 Sorry this conflicts with master again. Once you rebase I'll go ahead and merge it. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53685607 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala --- @@ -0,0 +1,356 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-11327] [MESOS] Dispatcher does not resp...

2016-02-22 Thread dragos
Github user dragos commented on the pull request: https://github.com/apache/spark/pull/10370#issuecomment-187137035 OK, I retract that. We need to use `shell=true`, so @jayv if you have that character escape version, let's go that way. The reason why it won't work is

[GitHub] spark pull request: [SPARK-13028] [ML] Add MaxAbsScaler to ML.feat...

2016-02-22 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10939#issuecomment-187141222 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13028] [ML] Add MaxAbsScaler to ML.feat...

2016-02-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10939#issuecomment-187135021 Jenkins, add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13422][SQL] Use HashedRelation instead ...

2016-02-22 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/11291#issuecomment-187109425 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13422][SQL] Use HashedRelation instead ...

2016-02-22 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/11291#issuecomment-187109363 LGTM - pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12632][PYSPARK][DOC] PySpark fpm and al...

2016-02-22 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/11186#issuecomment-187121452 LGTM, merged into master. Thanks @BryanCutler! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12632][PYSPARK][DOC] PySpark fpm and al...

2016-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11186 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12153][SPARK-7617][MLlib]add support of...

2016-02-22 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10152#issuecomment-187119717 @srowen thanks for merging it. @ygcao thanks for the PR! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-13028] [ML] Add MaxAbsScaler to ML.feat...

2016-02-22 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/10939#issuecomment-187131170 @srowen can you add me to Jenkins whitelist? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13334] [ML] ML KMeansModel / BisectingK...

2016-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11214 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-13334] [ML] ML KMeansModel / BisectingK...

2016-02-22 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/11214#issuecomment-187124904 LGTM, merged to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11229#issuecomment-187583445 **[Test build #51733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51733/consoleFull)** for PR 11229 at commit

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2016-02-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-187584849 **[Test build #51734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51734/consoleFull)** for PR 10058 at commit

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-187585349 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13429] [MLlib] Unify Logistic Regressio...

2016-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11299 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-187585346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13383][SQL] Keep broadcast hint after c...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11260#issuecomment-187585367 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13383][SQL] Keep broadcast hint after c...

2016-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11260#issuecomment-187585365 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53744546 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -318,7 +316,11 @@ private[spark] class BlockManager( * may not

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53744601 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerManagedBuffer.scala --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-12757] Add block-level read/write locks...

2016-02-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10705#discussion_r53744637 --- Diff: core/src/main/scala/org/apache/spark/network/BlockDataManager.scala --- @@ -31,6 +31,14 @@ trait BlockDataManager { /** *

  1   2   3   4   5   6   7   8   >