[GitHub] spark pull request #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext...

2016-09-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15201 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15187: [SPARK-17616][SQL] Support a single distinct aggregate c...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15187 **[Test build #65784 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65784/consoleFull)** for PR 15187 at commit

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #65791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65791/consoleFull)** for PR 15172 at commit

[GitHub] spark issue #15202: Backport SPARK-17599 and SPARK-17569 to Spark 2.0 branch

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15202 **[Test build #65790 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65790/consoleFull)** for PR 15202 at commit

[GitHub] spark issue #15202: Backport SPARK-17599 and SPARK-17569 to Spark 2.0 branch

2016-09-22 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15202 @zsxwing and @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15172 That's a weird error... complaining about classes that really should exist. retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #15202: Backport SPARK-17599 and SPARK-17569 to Spark 2.0...

2016-09-22 Thread brkyvz
GitHub user brkyvz opened a pull request: https://github.com/apache/spark/pull/15202 Backport SPARK-17599 and SPARK-17569 to Spark 2.0 branch ## What changes were proposed in this pull request? This Backports PR #15153 and PR #15122 to Spark 2.0 branch for Structured

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15201 Merging to master / 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/10212#discussion_r80137783 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -759,7 +761,15 @@ private[spark] object JsonProtocol { return metrics

[GitHub] spark pull request #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/10212#discussion_r80137728 --- Diff: core/src/test/resources/HistoryServerExpectations/complete_stage_list_json_expectation.json --- @@ -6,6 +6,7 @@ "numCompleteTasks" : 8,

[GitHub] spark pull request #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/10212#discussion_r80137957 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -1097,7 +1100,9 @@ private[spark] object JsonProtocolSuite extends

[GitHub] spark pull request #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated S...

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14971#discussion_r80137168 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -134,7 +134,7 @@ case class CatalogTable(

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65786/ Test PASSed. ---

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15201 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15201 **[Test build #65786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65786/consoleFull)** for PR 15201 at commit

[GitHub] spark issue #15034: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15034 **[Test build #65789 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65789/consoleFull)** for PR 15034 at commit

[GitHub] spark issue #15034: [SPARK-16240][ML] ML persistence backward compatibility ...

2016-09-22 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/15034 @JoshRosen That is weird; MiMa passes for me locally, but I see that it shouldn't. I added a MiMaException; this should not be a problem for users. --- If your project is set up for it, you can

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15199 **[Test build #65788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65788/consoleFull)** for PR 15199 at commit

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15199 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15089: [SPARK-15621] [SQL] Support spilling for Python U...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15089#discussion_r80128271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala --- @@ -17,18 +17,270 @@ package

[GitHub] spark pull request #15089: [SPARK-15621] [SQL] Support spilling for Python U...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15089#discussion_r80129158 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/BatchEvalPythonExec.scala --- @@ -17,18 +17,270 @@ package

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15190 @dilipbiswal Based on your tests and the source codes, I think your fix does not break anything. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15089: [SPARK-15621] [SQL] Support spilling for Python U...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15089#discussion_r80130429 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/RowQueue.scala --- @@ -0,0 +1,278 @@ +/* +* Licensed to the Apache

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15199 **[Test build #3287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3287/consoleFull)** for PR 15199 at commit

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10212 **[Test build #65787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65787/consoleFull)** for PR 10212 at commit

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-09-22 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/15190 @gatorsmile Thanks.. didn't realize we wanted to find out the CTAS behaviour. Here is the result.. When covertCTAS is set to true, we create a data source table with parquet format.

[GitHub] spark issue #15197: [SPARK-17631] [SQL] Add HttpStreamSink for structured st...

2016-09-22 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15197 Thanks for working on this, it does seem like it could be useful. I'm not sure if this should go into Spark or into a separate package. It really depends on how many people want this feature.

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15199 **[Test build #3287 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3287/consoleFull)** for PR 15199 at commit

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15199 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15199: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAgg...

2016-09-22 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15199 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15169: [SPARK-17613] S3A base paths with no '/' at the e...

2016-09-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15169 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15201 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15190 @dilipbiswal Are they data source tables? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15169: [SPARK-17613] S3A base paths with no '/' at the end retu...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15169 LGTM. I walked through the code in IntelliJ and have verified that the added unit test fails without this change. It's weird that `s3a` would return a non-absolute path when producing the

[GitHub] spark issue #15179: [SPARK-10835] [ML] Change Output of NGram to Array(Strin...

2016-09-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15179 Sorry, what's a UT here? user defined type? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-09-22 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/14803 Mostly looks good, I've also asked @tdas to take a look since he wrote this initially. A few more cases came to mind while while I was rephrasing your documentation. Specifically, -

[GitHub] spark pull request #14803: [SPARK-17153][SQL] Should read partition data whe...

2016-09-22 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/14803#discussion_r80120376 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -512,6 +512,10 @@ csvDF = spark \ These examples generate streaming DataFrames

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15201 **[Test build #65786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65786/consoleFull)** for PR 15201 at commit

[GitHub] spark issue #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext when t...

2016-09-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15201 /cc @rxin @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15201: [SPARK-17638][Streaming]Stop JVM StreamingContext...

2016-09-22 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15201 [SPARK-17638][Streaming]Stop JVM StreamingContext when the Python process is dead ## What changes were proposed in this pull request? When the Python process is dead, the JVM

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15102 **[Test build #65785 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65785/consoleFull)** for PR 15102 at commit

[GitHub] spark issue #15187: [SPARK-17616][SQL] Support a single distinct aggregate c...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15187 **[Test build #65784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65784/consoleFull)** for PR 15187 at commit

[GitHub] spark pull request #15187: [SPARK-17616][SQL] Support a single distinct aggr...

2016-09-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15187#discussion_r80115663 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala --- @@ -0,0 +1,74 @@ +/* + *

[GitHub] spark pull request #15187: [SPARK-17616][SQL] Support a single distinct aggr...

2016-09-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15187#discussion_r80115646 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala --- @@ -0,0 +1,74 @@ +/* + *

[GitHub] spark pull request #15187: [SPARK-17616][SQL] Support a single distinct aggr...

2016-09-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15187#discussion_r80115529 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala --- @@ -0,0 +1,74 @@ +/* + *

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65782/ Test FAILed. ---

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65782/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14079: [SPARK-8425][CORE] New Blacklist Mechanism

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14079 **[Test build #65783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65783/consoleFull)** for PR 14079 at commit

[GitHub] spark issue #15179: [SPARK-10835] [ML] Change Output of NGram to Array(Strin...

2016-09-22 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/15179 LGTM. How about adding an UT for the input type new ArrayType(StringType, false)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65782/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65781/ Test FAILed. ---

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65781/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix Multipl...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15174 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65777/ Test PASSed. ---

[GitHub] spark issue #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix Multipl...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15174 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65781/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix Multipl...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15174 **[Test build #65777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65777/consoleFull)** for PR 15174 at commit

[GitHub] spark pull request #15187: [SPARK-17616][SQL] Support a single distinct aggr...

2016-09-22 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/15187#discussion_r80112793 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala --- @@ -0,0 +1,74 @@ +/*

[GitHub] spark pull request #15187: [SPARK-17616][SQL] Support a single distinct aggr...

2016-09-22 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/15187#discussion_r80112666 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala --- @@ -0,0 +1,74 @@ +/*

[GitHub] spark pull request #15187: [SPARK-17616][SQL] Support a single distinct aggr...

2016-09-22 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/15187#discussion_r80109506 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregatesSuite.scala --- @@ -0,0 +1,74 @@ +/*

[GitHub] spark pull request #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15200 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #9766: [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Jav...

2016-09-22 Thread GregBowyer
Github user GregBowyer commented on the issue: https://github.com/apache/spark/pull/9766 Where do we stand on this, I just reapplied this patch to a spark 2.1-xxx build to get the same behaviour. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15200 I'm going to merge this in master/branch-2.0 because I need this to cut an rc. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-09-22 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/15190 @yhuai Hi Yin, create table ... as select ... would respect the setting of hive.default.fileformat. ``` SQL scala> spark.sql("SET hive.default.fileformat=parquet")

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65780/ Test FAILed. ---

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65780/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15041: [SPARK-17488][CORE] TakeAndOrder will OOM when the data ...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15041 **[Test build #65780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65780/consoleFull)** for PR 15041 at commit

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15200 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65778/ Test PASSed. ---

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15200 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15200 **[Test build #65778 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65778/consoleFull)** for PR 15200 at commit

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15172 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65779/ Test FAILed. ---

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #65779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65779/consoleFull)** for PR 15172 at commit

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15172 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15172 **[Test build #65779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65779/consoleFull)** for PR 15172 at commit

[GitHub] spark issue #15172: [SPARK-13331] AES support for over-the-wire encryption

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15172 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15186: [SPARK-17485] Prevent failed remote reads of cach...

2016-09-22 Thread JoshRosen
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/15186 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15186: [SPARK-17485] Prevent failed remote reads of cached bloc...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15186 I'm going to merge this into branch-1.6. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15196: [SPARK-17614] [SQL] sparkSession.read() .jdbc(***...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15196#discussion_r80100267 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala --- @@ -58,11 +58,11 @@ object JDBCRDD extends Logging {

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-22 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15090 @gatorsmile OK, I'll rebase and add some tests to handle negative cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15090: [SPARK-17073] [SQL] generate column-level statist...

2016-09-22 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/15090#discussion_r80097979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeColumnCommand.scala --- @@ -0,0 +1,189 @@ +/* + * Licensed to the

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15200 **[Test build #65778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65778/consoleFull)** for PR 15200 at commit

[GitHub] spark issue #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15200 The only downside of this is that we won't be uploading our vignette to the website. I think this is fine -- Any thoughts @junyangq @felixcheung cc @rxin --- If your project is set up

[GitHub] spark pull request #15200: Skip building R vignettes if Spark is not built

2016-09-22 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/15200 Skip building R vignettes if Spark is not built ## What changes were proposed in this pull request? When we build the docs separately we don't have the JAR files from the Spark build in

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/15102 This is pretty much the fundamental issue. Kafka offsets alone aren't capable of meeting the SQL Offset interface as defined. I think that means the Offset interface needs to be

[GitHub] spark issue #15186: [SPARK-17485] Prevent failed remote reads of cached bloc...

2016-09-22 Thread srinathshankar
Github user srinathshankar commented on the issue: https://github.com/apache/spark/pull/15186 Thanks for confirming, LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15186: [SPARK-17485] Prevent failed remote reads of cached bloc...

2016-09-22 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15186 Yep, 1.6 has fewer BlockManager tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-09-22 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15102 > PR with failing test indicating at least one reason why it's wrong from an end-user perspective: @koeninger Thanks for writing the test. Yes, we are aware of this issue. However, it's

[GitHub] spark issue #15113: [SPARK-17508][PYSPARK][ML] PySpark treat Param values No...

2016-09-22 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/15113 Hi @yanboliang , thanks for taking a look! I'm more inclined to think that most Python users would think that `weightCol=None` does not cause a problem, as in this corresponding JIRA. Most

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14959 LGTM. Let's see if others have any comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15152: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15152 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15149 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65776/ Test PASSed. ---

[GitHub] spark issue #15152: [SPARK-17365][Core] Remove/Kill multiple executors toget...

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/15152 LGTM. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15149 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15149: [SPARK-17057] [ML] ProbabilisticClassifierModels' thresh...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15149 **[Test build #65776 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65776/consoleFull)** for PR 15149 at commit

[GitHub] spark issue #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix Multipl...

2016-09-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15174 **[Test build #65777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65777/consoleFull)** for PR 15174 at commit

[GitHub] spark pull request #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix ...

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15174#discussion_r80088798 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala --- @@ -342,6 +342,7 @@ trait CheckAnalysis extends

[GitHub] spark pull request #15174: [SPARK-17502] [17609] [SQL] [Backport] [2.0] Fix ...

2016-09-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15174#discussion_r80088127 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -200,8 +200,9 @@ case class

[GitHub] spark issue #10212: [SPARK-12221] add cpu time to metrics

2016-09-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/10212 @jisookim0513 unfortunately there are conflicts once more. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

<    1   2   3   4   5   >