[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968974 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -578,4 +578,66 @@ class

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15998 I suspect this is a spurious, unrelated test failure. Can we get a rebuild, please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968652 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -578,4 +578,66 @@ class

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread xwu0226
Github user xwu0226 commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968718 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -578,4 +578,66 @@ class

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread xwu0226
Github user xwu0226 commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968659 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -578,4 +578,66 @@ class

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968590 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -578,4 +578,66 @@ class

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread xwu0226
Github user xwu0226 commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968586 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java --- @@ -107,7 +107,16 @@ public

[GitHub] spark pull request #16157: [SPARK-18723][DOC] Expanded programming guide inf...

2016-12-05 Thread michalsenkyr
GitHub user michalsenkyr opened a pull request: https://github.com/apache/spark/pull/16157 [SPARK-18723][DOC] Expanded programming guide information on wholeTex… ## What changes were proposed in this pull request? Add additional information to wholeTextFiles in the

[GitHub] spark issue #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColu...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16152 cc @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16149#discussion_r90968163 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -479,7 +479,12 @@ object GeneralizedLinearRegression

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/16156#discussion_r90968113 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java --- @@ -107,7 +107,16 @@ public

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16151 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69681/ Test PASSed. ---

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16151 **[Test build #69681 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69681/consoleFull)** for PR 16151 at commit

[GitHub] spark issue #16090: [SPARK-18661] [SQL] Creating a partitioned datasource ta...

2016-12-05 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/16090 Filed https://issues.apache.org/jira/browse/SPARK-18725 https://issues.apache.org/jira/browse/SPARK-18726 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15998 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69683/ Test FAILed. ---

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15998 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15998 **[Test build #69683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69683/consoleFull)** for PR 15998 at commit

[GitHub] spark pull request #16131: [SPARK-18701][ML] Fix Poisson GLM failure due to ...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16131#discussion_r90966317 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -505,7 +505,7 @@ object GeneralizedLinearRegression

[GitHub] spark issue #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColu...

2016-12-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16152 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15722 **[Test build #69689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69689/consoleFull)** for PR 15722 at commit

[GitHub] spark pull request #16131: [SPARK-18701][ML] Fix Poisson GLM failure due to ...

2016-12-05 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16131#discussion_r90965352 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -505,7 +505,7 @@ object

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-12-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15722 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16156: [SPARK-18539][SQL]: tolerate pushed-down filter on non-e...

2016-12-05 Thread xwu0226
Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/16156 @gatorsmile @liancheng Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16156: [SPARK-18539][SQL]: tolerate pushed-down filter on non-e...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16156 **[Test build #69688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69688/consoleFull)** for PR 16156 at commit

[GitHub] spark pull request #16156: [SPARK-18539][SQL]: tolerate pushed-down filter o...

2016-12-05 Thread xwu0226
GitHub user xwu0226 opened a pull request: https://github.com/apache/spark/pull/16156 [SPARK-18539][SQL]: tolerate pushed-down filter on non-existing parquet columns ## What changes were proposed in this pull request? When `spark.sql.parquet.filterPushdown` is on,

[GitHub] spark issue #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColu...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16152 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69680/ Test PASSed. ---

[GitHub] spark issue #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColu...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16152 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColu...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16152 **[Test build #69680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69680/consoleFull)** for PR 16152 at commit

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16068 **[Test build #69687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69687/consoleFull)** for PR 16068 at commit

[GitHub] spark issue #16155: [SPARK-18722][SS]Move no data rate limit from StreamExec...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16155 **[Test build #69686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69686/consoleFull)** for PR 16155 at commit

[GitHub] spark pull request #16155: [SPARK-18722][SS]Move no data rate limit from Str...

2016-12-05 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16155 [SPARK-18722][SS]Move no data rate limit from StreamExecution to ProgressReporter ## What changes were proposed in this pull request? Move no data rate limit from StreamExecution to

[GitHub] spark pull request #16131: [SPARK-18701][ML] Fix Poisson GLM failure due to ...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16131#discussion_r90956362 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -505,7 +505,7 @@ object GeneralizedLinearRegression

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16053 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69685/ Test FAILed. ---

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16053 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16053 **[Test build #69685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69685/consoleFull)** for PR 16053 at commit

[GitHub] spark issue #16137: [SPARK-18708][CORE] Improvement/improve docs in spark co...

2016-12-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16137 It is good to start small, but not all small changes are worthwhile. It would be better to focus on fixing actual problems like a typo or missing doc. You can then expand to improve / fix docs

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16053 **[Test build #69685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69685/consoleFull)** for PR 16053 at commit

[GitHub] spark pull request #16131: [SPARK-18701][ML] Fix Poisson GLM failure due to ...

2016-12-05 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16131#discussion_r90955908 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -505,7 +505,7 @@ object

[GitHub] spark issue #16137: [SPARK-18708][CORE] Improvement/improve docs in spark co...

2016-12-05 Thread Mironor
Github user Mironor commented on the issue: https://github.com/apache/spark/pull/16137 Hi @srowen , it's quite hard to directly jump on something like "refactoring core scheduling logic", I thought fixing small things would be a good start. To make it simple I limited all the changes

[GitHub] spark issue #16154: [SPARK-17822] Make JVMObjectTracker a member variable of...

2016-12-05 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/16154 @shivaram Let's wait Jenkins first:) I'm not sure it will pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16138: [WIP][SPARK-16609] Add to_date/to_timestamp with format ...

2016-12-05 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/16138 This will be very convenient! Looking forward to the whole patch. For SQL I think you should look at

[GitHub] spark pull request #16130: Update location of Spark YARN shuffle jar

2016-12-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16130 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16068 LGTM except one comment on test cases. cc @hvanhovell @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16130 Didn't merge to 2.0 (forgot I changed these docs in 2.1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16068: [SPARK-18637][SQL]Stateful UDF should be consider...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16068#discussion_r90953581 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala --- @@ -487,6 +488,43 @@ class HiveUDFSuite extends QueryTest

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16130 And 2.0 too, since it's relevant there. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16130 Merging to master / 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16154: [SPARK-17822] Make JVMObjectTracker a member variable of...

2016-12-05 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16154 Thanks @mengxr for the change. Taking a look now cc @felixcheung @falaki --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16143: [SPARK-18711][SQL] should disable subexpression eliminat...

2016-12-05 Thread koertkuipers
Github user koertkuipers commented on the issue: https://github.com/apache/spark/pull/16143 thanks for getting this fixed so fast --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16154: [SPARK-17822] Make JVMObjectTracker a member variable of...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16154 **[Test build #69684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69684/consoleFull)** for PR 16154 at commit

[GitHub] spark pull request #16154: [SPARK-17822] Make JVMObjectTracker a member vari...

2016-12-05 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/16154 [SPARK-17822] Make JVMObjectTracker a member variable of RBackend ## What changes were proposed in this pull request? * This PR changes `JVMObjectTracker` from `object` to `class` and let

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16068 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69676/ Test PASSed. ---

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16068 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16068 **[Test build #69676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69676/consoleFull)** for PR 16068 at commit

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r90948903 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -276,13 +279,17 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r90948767 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -224,8 +224,8 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r90948858 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -276,13 +279,17 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r90949013 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -461,11 +478,12 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r90948705 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -119,22 +119,22 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16137: [SPARK-18708][CORE] Improvement/improve docs in s...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16137#discussion_r90948641 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -119,22 +119,22 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #16120: [SPARK-18634][PySpark][SQL] Corruption and Correc...

2016-12-05 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16120#discussion_r90948431 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -93,13 +93,13 @@ case class

[GitHub] spark pull request #16120: [SPARK-18634][PySpark][SQL] Corruption and Correc...

2016-12-05 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16120#discussion_r90947415 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -93,13 +93,13 @@ case class

[GitHub] spark issue #16103: [SPARK-18374][ML]Incorrect words in StopWords/english.tx...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69677/ Test PASSed. ---

[GitHub] spark issue #16103: [SPARK-18374][ML]Incorrect words in StopWords/english.tx...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16103 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16103: [SPARK-18374][ML]Incorrect words in StopWords/english.tx...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16103 **[Test build #69677 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69677/consoleFull)** for PR 16103 at commit

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15998 **[Test build #69683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69683/consoleFull)** for PR 15998 at commit

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15998 @gatorsmile I've applied your patch and reverted the change I made in the previous commit to workaround that defect. The failed test now passes for me. Let's see what Jenkins says. --- If your

[GitHub] spark issue #16153: [SPARK-18694][SS] Add StreamingQuery.explain and excepti...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16153 **[Test build #69682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69682/consoleFull)** for PR 16153 at commit

[GitHub] spark pull request #16153: [SPARK-18694][SS] Add StreamingQuery.explain and ...

2016-12-05 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16153 [SPARK-18694][SS] Add StreamingQuery.explain and exception to Python and fix StreamingQueryException (branch 2.1) ## What changes were proposed in this pull request? Backport #16125 to

[GitHub] spark issue #16140: [SPARK-18714][SQL] Add a simple time function to SparkSe...

2016-12-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16140 _shrug_ It seems pretty unrelated to SparkSession. I get that it's a convenience method but something Spark really needs to expose as an API method? --- If your project is set up for it, you can

[GitHub] spark issue #16125: [SPARK-18694][SS]Add StreamingQuery.explain and exceptio...

2016-12-05 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16125 Merged to master, but 2.1 failed due to mima exclude conflict. can you make another PR for 2.1 @zsxwing thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #16131: [SPARK-18701][ML] Fix Poisson GLM failure due to ...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16131#discussion_r90944278 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -505,7 +505,7 @@ object GeneralizedLinearRegression

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15998 > #15998 (comment) found a bug. If this PR will not be merged to Spark 2.1 branch, I think we need to submit a separate PR for resolving the bug. I would like to get this patch into Spark

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/15998 @gatorsmile I can't find your commit: ``` [msa@ip-10-0-8-34 spark-master]$ git fetch origin remote: Counting objects: 114, done. remote: Compressing objects: 100% (53/53), done.

[GitHub] spark issue #16150: [SPARK-18349][SparkR]:Update R API documentation on ml m...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16150 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16150: [SPARK-18349][SparkR]:Update R API documentation on ml m...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69678/ Test PASSed. ---

[GitHub] spark issue #16150: [SPARK-18349][SparkR]:Update R API documentation on ml m...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16150 **[Test build #69678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69678/consoleFull)** for PR 16150 at commit

[GitHub] spark pull request #16144: [MINOR][CORE][SQL][DOCS] Typo fixes

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16144#discussion_r90939466 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala --- @@ -21,7 +21,7 @@ import

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15998 @mallman , https://github.com/apache/spark/commit/f4c48e1d61dc4d1fc581b78dd9e5c948965acad5 might be what @cloud-fan wants. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #16143: [SPARK-18711][SQL] should disable subexpression e...

2016-12-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16143 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16125: [SPARK-18694][SS]Add StreamingQuery.explain and e...

2016-12-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16125 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16151 **[Test build #69681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69681/consoleFull)** for PR 16151 at commit

[GitHub] spark pull request #16144: [MINOR][CORE][SQL][DOCS] Typo fixes

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16144#discussion_r90939139 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -292,7 +292,7 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread nchammas
Github user nchammas commented on the issue: https://github.com/apache/spark/pull/16151 @srowen - OK, I elaborated a bit based on the snippet you posted. Feel free to nitpick on the wording. Would be happy to tweak further. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16151 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16125: [SPARK-18694][SS]Add StreamingQuery.explain and exceptio...

2016-12-05 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16125 Merging this to master and 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16147: [SPARK-18718][TESTS] Skip some test failures due to path...

2016-12-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16147 It sounds like some of the failures aren't just due to tests, but due to some functionality that doesn't work on Windows? or do I misundesrtand it? It still may be the right thing to do to

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69679/ Test PASSed. ---

[GitHub] spark pull request #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of w...

2016-12-05 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/16152 [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColumn ### What changes were proposed in this pull request? Our existing withColumn for adding metadata can simply use the existing

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16151 **[Test build #69679 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69679/consoleFull)** for PR 16151 at commit

[GitHub] spark issue #16151: [SPARK-18719] Add spark.ui.showConsoleProgress to config...

2016-12-05 Thread nchammas
Github user nchammas commented on the issue: https://github.com/apache/spark/pull/16151 @srowen - Good call. Will elaborate a bit based on what you posted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #15992: [SPARK-18560][CORE][STREAMING][WIP] Receiver data can no...

2016-12-05 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15992 @uncleGen could you close this one, please? The current workaround in #15992 looks enough. You can create a new one if you have a better solution for Java APIs. --- If your project is set up for

[GitHub] spark issue #16143: [SPARK-18711][SQL] should disable subexpression eliminat...

2016-12-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16143 LGTM - merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-12-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15998 https://github.com/apache/spark/pull/15998#issuecomment-264761754 found a bug. If this PR will not be merged to Spark 2.1 branch, I think we need to submit a separate PR for resolving the bug.

[GitHub] spark pull request #16145: [MINOR][CORE][SQL] Remove explicit RDD and Partit...

2016-12-05 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16145#discussion_r90936511 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ShuffledRowRDD.scala --- @@ -117,7 +113,7 @@ class CoalescedPartitioner(val parent:

[GitHub] spark issue #16152: [SPARK-18720] [SQL] [MINOR] Code Refactoring of withColu...

2016-12-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16152 **[Test build #69680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69680/consoleFull)** for PR 16152 at commit

[GitHub] spark pull request #16128: [WIP][SPARK-18671][SS] Added tests to ensure stab...

2016-12-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16128#discussion_r90934943 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLogSuite.scala --- @@ -60,4 +60,39 @@ class OffsetSeqLogSuite extends

[GitHub] spark pull request #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers exam...

2016-12-05 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/16148#discussion_r90934508 --- Diff: examples/src/main/r/ml/ml.R --- @@ -0,0 +1,64 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers exam...

2016-12-05 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/16148#discussion_r90934363 --- Diff: examples/src/main/r/ml/lda.R --- @@ -0,0 +1,46 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

<    1   2   3   4   5   6   >