[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16135 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16220 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69981/ Test FAILed. ---

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16248 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16248: [SPARK-18810][SPARKR] SparkR install.spark does n...

2016-12-11 Thread felixcheung
GitHub user felixcheung reopened a pull request: https://github.com/apache/spark/pull/16248 [SPARK-18810][SPARKR] SparkR install.spark does not work for RCs, snapshots ## What changes were proposed in this pull request? Support overriding the download url (include version

[GitHub] spark pull request #16248: [SPARK-18810][SPARKR] SparkR install.spark does n...

2016-12-11 Thread felixcheung
Github user felixcheung closed the pull request at: https://github.com/apache/spark/pull/16248 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16248 **[Test build #69984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69984/consoleFull)** for PR 16248 at commit

[GitHub] spark pull request #16244: [SQL][minor] simplify a test to fix the maven tes...

2016-12-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16244 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16196: [SPARK-18231] Optimise SizeEstimator implementation

2016-12-11 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/16196 Agreed, I'll be back working on this and answering the queries after the 2.1.0 release vote passes, that's my current priority as we're nearing the Christmas break period --- If your project is

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread Mironor
Github user Mironor commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91856040 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -956,24 +976,24 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16220 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16248 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69982/ Test FAILed. ---

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16135 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69979/ Test FAILed. ---

[GitHub] spark issue #16236: SPARK-18809: KCL version to 1.6.2 on master

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16236 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #13944: [SPARK-16297][SQL] Fix mapping Microsoft SQLServer diale...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13944 @meknio if you'll update as Marcelo says, I'll merge this. Change the description here to contain your comment at https://github.com/apache/spark/pull/13944#issuecomment-229395810 --- If your

[GitHub] spark pull request #16179: [SPARK-18752][hive] "isSrcLocal" value should be ...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16179#discussion_r91852909 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala --- @@ -168,14 +200,50 @@ class HiveCommandSuite extends

[GitHub] spark issue #16137: [SPARK-18708][CORE] Improvement/improve docs in spark co...

2016-12-11 Thread Mironor
Github user Mironor commented on the issue: https://github.com/apache/spark/pull/16137 @srowen I see that my rebase didn't go well, sorry for that. Is there a way to revert the last push or it would be better to create another pull request and close this one? --- If your project is

[GitHub] spark pull request #16242: [SPARK-18628][ML] Update Scala param and Python p...

2016-12-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16242 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16086 Unless someone vigorously objects, yes let's close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15915 @uncleGen can you follow up on this so we can finish this out? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #16179: [SPARK-18752][hive] "isSrcLocal" value should be ...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16179#discussion_r91853357 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala --- @@ -190,87 +258,39 @@ class HiveCommandSuite extends

[GitHub] spark issue #16137: [SPARK-18708][CORE] Improvement/improve docs in spark co...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16137 Up to you but it's fine to just start from scratch in a new PR. Sometimes that's simpler, esp. because it will probably be fairly different anyway. --- If your project is set up for it, you can

[GitHub] spark issue #16245: [SQL][WIP] Add optimizer rule to reorder Filter predicat...

2016-12-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16245 I will create Jira and add test for this later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16179: [SPARK-18752][hive] "isSrcLocal" value should be set fro...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16179 LGTM except 2 minor comments: https://github.com/apache/spark/pull/16179#discussion_r91852909 and https://github.com/apache/spark/pull/16179#discussion_r91846277 --- If your project is set up

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69983/ Test FAILed. ---

[GitHub] spark pull request #16104: [SPARK-18675][SQL] CTAS for hive serde table shou...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16104#discussion_r91853527 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -86,14 +85,15 @@ case class InsertIntoHiveTable(

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #69983 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69983/consoleFull)** for PR 16249 at commit

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16249 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16104 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16137: [SPARK-18708][CORE] Improvement/improve docs in spark co...

2016-12-11 Thread Mironor
Github user Mironor commented on the issue: https://github.com/apache/spark/pull/16137 @srowen I fixed everything. All comments are squashed, please review the current one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15995 **[Test build #69986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69986/consoleFull)** for PR 15995 at commit

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16104 **[Test build #69985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69985/consoleFull)** for PR 16104 at commit

[GitHub] spark issue #15730: [SPARK-18218][ML][MLLib] Optimize BlockMatrix multiplica...

2016-12-11 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15730 @brkyvz All right, I'll update code ASAP. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16248 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16248 **[Test build #69984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69984/consoleFull)** for PR 16248 at commit

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16248 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69984/ Test PASSed. ---

[GitHub] spark issue #15713: [SPARK-18196] [CORE] Optimise CompactBuffer implementati...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15713 Although I'd love to find we can get rid of a custom collections class, it seems like we can't do that and that the optimization here doesn't result in a win. Let's close this for now. --- If your

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16104 **[Test build #69985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69985/consoleFull)** for PR 16104 at commit

[GitHub] spark pull request #16134: [SPARK-18703] [SQL] Drop Staging Directories and ...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16134#discussion_r91853572 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -166,6 +166,30 @@ class InsertIntoHiveTableSuite

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91853950 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -55,6 +58,15 @@ public

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855221 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -816,6 +833,10 @@ class SparkContext(config: SparkConf) extends Logging { /**

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855239 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -816,6 +833,10 @@ class SparkContext(config: SparkConf) extends Logging { /**

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855254 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -956,24 +976,24 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark pull request #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15995#discussion_r91855259 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -192,19 +200,13 @@ case class

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855172 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -486,9 +491,11 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855194 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -721,16 +732,15 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855158 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -276,6 +276,9 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855251 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -956,24 +976,24 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r91855189 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -705,13 +712,17 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/16249 [SPARKR] Refactor scripts for R ## What changes were proposed in this pull request? Refactored script to remove duplications and clearer purpose for each script ## How was

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16249 **[Test build #69983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69983/consoleFull)** for PR 16249 at commit

[GitHub] spark issue #16244: [SQL][minor] simplify a test to fix the maven tests

2016-12-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16244 @cloud-fan Ok. I see. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16145: [MINOR][CORE][SQL] Remove explicit RDD and Partit...

2016-12-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16145#discussion_r91851737 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ShuffledRowRDD.scala --- @@ -117,7 +113,7 @@ class CoalescedPartitioner(val parent:

[GitHub] spark issue #15736: [SPARK-18224] [CORE] Optimise PartitionedPairBuffer impl...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15736 @a-roberts let's either finish the thought and merge this as mostly a code cleanup and maybe marginal win, or just close it. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #16137: [SPARK-18708][CORE] Improvement/improve docs in spark co...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16137 Ping @Mironor are you able to pare this down further? I want to figure out whether this will proceed or not. It needs a rebase anyhow. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14638 I am really on the fence about it. I can see the argument that it's useful to support, and not hard to support. But also that this is just one of many Hive features that one could support, and we

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15717 Why can't we remove `alterTableComment`? It seems that the implementations are same, so we can make it a util function to map alter column comment action to `alterTable` call. --- If your

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91854780 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -55,6 +58,15 @@ public BufferHolder(UnsafeRow

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91854826 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -55,6 +58,15 @@ public BufferHolder(UnsafeRow

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-11 Thread lins05
Github user lins05 commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91855053 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +458,78 @@ private[spark] class Executor( } /** +

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69985/ Test PASSed. ---

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16104 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91855403 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java --- @@ -55,6 +58,15 @@ public

[GitHub] spark issue #16248: [SPARK-18810][SPARKR] SparkR install.spark does not work...

2016-12-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16248 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16244: [SQL][minor] simplify a test to fix the maven tests

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16244 Merged to master and 2.1, in order to see if this resolves the test failures for 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #16236: SPARK-18809: KCL version to 1.6.2 on master

2016-12-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16236 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16196: [SPARK-18231] Optimise SizeEstimator implementation

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16196 Ping @a-roberts -- I think some sections of this are clearly a win, like near https://github.com/apache/spark/pull/16196#discussion_r91270660 but maybe best to back out anything controversial. And

[GitHub] spark issue #16194: [SPARK-18767][ML] Unify Models' toString methods

2016-12-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16194 Yes, it would be nicer to just have a default like `ClassName(uid=...)` at least. That however would not include the class-specific details like number of layers. I am neutral on whether that's so

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-11 Thread lins05
Github user lins05 commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91855204 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +458,78 @@ private[spark] class Executor( } /** +

[GitHub] spark pull request #16189: [SPARK-18761][CORE] Introduce "task reaper" to ov...

2016-12-11 Thread lins05
Github user lins05 commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91856289 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +458,78 @@ private[spark] class Executor( } /** +

[GitHub] spark issue #11045: [SPARK-8321][SQL][WIP] Authorization Support(on all oper...

2016-12-11 Thread winningsix
Github user winningsix commented on the issue: https://github.com/apache/spark/pull/11045 @yaooqinn yes, the validation is working on server side. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16086: [SPARK-18653][SQL] Fix incorrect space padding for unico...

2016-12-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/16086 I am thinking about an simpler approach. However, it is fine to close for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16245: [SPARK-18824][SQL] Add optimizer rule to reorder ...

2016-12-11 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16245#discussion_r91881557 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -514,6 +514,25 @@ case class OptimizeCodegen(conf:

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16135 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69998/ Test PASSed. ---

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16135 **[Test build #69998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69998/consoleFull)** for PR 16135 at commit

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add StripedLock for each table's rela...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16135 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16245: [SPARK-18824][SQL] Add optimizer rule to reorder ...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16245#discussion_r91886133 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -514,6 +514,25 @@ case class OptimizeCodegen(conf:

[GitHub] spark pull request #16252: [SPARK-18827][Core] Fix cannot read broadcast on ...

2016-12-11 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/16252 [SPARK-18827][Core] Fix cannot read broadcast on disk ## What changes were proposed in this pull request? Fix cannot read broadcast on disk ## How was this patch tested? Add

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15915 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70001/ Test FAILed. ---

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15915 **[Test build #70001 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70001/consoleFull)** for PR 15915 at commit

[GitHub] spark issue #15915: [SPARK-18485][CORE] Underlying integer overflow when cre...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15915 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16104 **[Test build #70006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70006/consoleFull)** for PR 16104 at commit

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Do you mean to replace the current whole `TableReader.scala` which was introduced in SPARK-1251 ? I guessed Spark chose this direct access approach for the performance issue at that time.

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r9169 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -78,6 +80,7 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #16142: [SPARK-18716][CORE] Restrict the disk usage of sp...

2016-12-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/16142#discussion_r91889826 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -90,6 +91,10 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-11 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16030 the new behavior LGTM, but I'm not sure if we still need to keep the old behavior --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16220 **[Test build #70003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70003/consoleFull)** for PR 16220 at commit

[GitHub] spark pull request #16135: [SPARK-18700][SQL] Add StripedLock for each table...

2016-12-11 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/16135#discussion_r91879183 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -105,6 +111,7 @@ object HiveCatalogMetrics extends Source {

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16214 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16220 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16219: [SPARK-18790][SS] Keep a general offset history of strea...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16219 **[Test build #3493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3493/consoleFull)** for PR 16219 at commit

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70003/ Test PASSed. ---

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r91881279 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -56,33 +58,93 @@ case class

[GitHub] spark issue #16249: [SPARKR] Refactor scripts for R

2016-12-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16249 Can we open a JIRA for this ? Its good to track this change --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16220: [SPARK-18796][SS]StreamingQueryManager should not block ...

2016-12-11 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16220 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16104 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r91893984 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -331,7 +332,7 @@ private[spark] class MemoryStore( var

[GitHub] spark issue #16251: [SPARK-18826][SS]Add 'newestFirst' option to FileStreamS...

2016-12-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16251 **[Test build #70002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70002/consoleFull)** for PR 16251 at commit

[GitHub] spark pull request #16219: [SPARK-18790][SS] Keep a general offset history o...

2016-12-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16219 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15297: [SPARK-9862]Handling data skew

2016-12-11 Thread YuhuWang2002
Github user YuhuWang2002 commented on the issue: https://github.com/apache/spark/pull/15297 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15915: [SPARK-18485][CORE] Underlying integer overflow w...

2016-12-11 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/15915#discussion_r91880781 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -223,8 +222,10 @@ private[spark] abstract class MemoryManager(

  1   2   3   >