[GitHub] spark issue #16011: [SPARK-18587][ML] Remove handleInvalid from QuantileDisc...

2016-11-27 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/16011 As far as I recall, the idea is that the `Bucketizer` can be used standalone, and because the `QuantileDiscretizer` itself produced the same thing as a bucketizer, it was used as the model rather

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15995 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15995 **[Test build #69220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69220/consoleFull)** for PR 15995 at commit

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15995 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69220/ Test FAILed. ---

[GitHub] spark pull request #16008: [SPARK-18585][SQL] Use `ev.isNull = "false"` if p...

2016-11-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16008 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16008: [SPARK-18585][SQL] Use `ev.isNull = "false"` if possible...

2016-11-27 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16008 Thanks - merging in master/branch-2.1. We should look into the Janino change in the future. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16013: [SPARK-3359][DOCS] Make javadoc8 working for unidoc/genj...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16013 I think it is ready to be reviewed - @srowen. Thank you for your close look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16008: [SPARK-18585][SQL] Use `ev.isNull = "false"` if possible...

2016-11-27 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16008 Hmm, it is a great idea, but I think it would be very hard to submit such a patch (at least for me now). --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89727420 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -40,14 +40,9 @@ case class JdbcType(databaseTypeDefinition :

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-27 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r89727312 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -437,4 +444,38 @@ final class DataFrameNaFunctions

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89727297 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala --- @@ -41,7 +41,7 @@ private[ml] trait VectorIndexerParams extends

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89727185 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDDCheckpointData.scala --- @@ -24,9 +24,7 @@ import org.apache.spark.Partition /** *

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89727091 --- Diff: core/src/main/scala/org/apache/spark/ui/UIUtils.scala --- @@ -422,13 +422,8 @@ private[spark] object UIUtils extends Logging { * the

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89727059 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/SQLTransformer.scala --- @@ -33,9 +33,9 @@ import org.apache.spark.sql.types.StructType

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89727120 --- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala --- @@ -41,9 +41,9 @@ private[spark] class JdbcPartition(idx: Int, val lower: Long, val

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16013 **[Test build #69228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69228/consoleFull)** for PR 16013 at commit

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15994 **[Test build #69229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69229/consoleFull)** for PR 15994 at commit

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89726626 --- Diff: core/src/main/scala/org/apache/spark/rdd/DoubleRDDFunctions.scala --- @@ -153,11 +153,9 @@ class DoubleRDDFunctions(self: RDD[Double]) extends

[GitHub] spark issue #16029: [MINOR][ML] Remove duplicate import in GLR

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16029 **[Test build #69227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69227/consoleFull)** for PR 16029 at commit

[GitHub] spark pull request #16029: [MINOR][ML] Remove duplicate import in GLR

2016-11-27 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/16029 [MINOR][ML] Remove duplicate import in GLR ## What changes were proposed in this pull request? there were two `import GeneralizedLinearRegression._` in trait

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #69226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69226/consoleFull)** for PR 16028 at commit

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69226/ Test FAILed. ---

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #69226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69226/consoleFull)** for PR 16028 at commit

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15986 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15986 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69218/ Test PASSed. ---

[GitHub] spark pull request #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-27 Thread zhengruifeng
GitHub user zhengruifeng opened a pull request: https://github.com/apache/spark/pull/16028 [SPARK-18518][ML] HasSolver supports override ## What changes were proposed in this pull request? 1, make param support non-final with `finalFields` option 2, generate `HasSolver` with

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15986 **[Test build #69218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69218/consoleFull)** for PR 15986 at commit

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89724422 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDDCheckpointData.scala --- @@ -23,7 +23,8 @@ import org.apache.spark.Partition /**

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16013 **[Test build #69225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69225/consoleFull)** for PR 16013 at commit

[GitHub] spark issue #15983: [SPARK-18544] [SQL] Append with df.saveAsTable writes da...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15983 **[Test build #69223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69223/consoleFull)** for PR 15983 at commit

[GitHub] spark issue #15976: [SPARK-18403][SQL] Fix unsafe data false sharing issue i...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15976 **[Test build #69224 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69224/consoleFull)** for PR 15976 at commit

[GitHub] spark issue #15979: [SPARK-18251][SQL] the type of Dataset can't be Option o...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15979 retest it please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15983: [SPARK-18544] [SQL] Append with df.saveAsTable writes da...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15983 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

2016-11-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15837 @kiszk Do you mean to avoid subexpression elimination? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15976: [SPARK-18403][SQL] Fix unsafe data false sharing issue i...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15976 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching Using ...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15975 LGTM except https://github.com/apache/spark/pull/15975/files#r89722356, what's the status of it? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15975#discussion_r89722356 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -404,6 +425,7 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

2016-11-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15837 @cloud-fan Sure, no problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15837: [SPARK-18395][SQL] Evaluate common subexpression like la...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15837 Sorry for the delay, but I may not have time to review it before the 2.1 release, can you hold it off until 2.1 release? thanks! --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16013 **[Test build #69222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69222/consoleFull)** for PR 16013 at commit

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89721863 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -405,13 +406,14 @@ case class WrapOption(child:

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15780 **[Test build #69221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69221/consoleFull)** for PR 15780 at commit

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89721435 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r89721405 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -437,4 +444,38 @@ final class DataFrameNaFunctions

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r89721142 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -437,4 +444,38 @@ final class DataFrameNaFunctions

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r89721112 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -153,19 +168,20 @@ final class DataFrameNaFunctions

[GitHub] spark pull request #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss ...

2016-11-27 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/15994#discussion_r89721087 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala --- @@ -128,66 +128,49 @@ final class DataFrameNaFunctions

[GitHub] spark pull request #16003: [SPARK-18482][SQL] make sure Spark can access the...

2016-11-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16003 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15995 **[Test build #69220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69220/consoleFull)** for PR 15995 at commit

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-11-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15995 @ericl you are right, I pushed a new commit to do `convertStaticPartitions` right before we convert `InsertIntoTable` to `InsertIntoHadoopFsRelation`, so the partitioning information won't be

[GitHub] spark issue #16003: [SPARK-18482][SQL] make sure Spark can access the table ...

2016-11-27 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16003 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16003: [SPARK-18482][SQL] make sure Spark can access the table ...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16003 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16003: [SPARK-18482][SQL] make sure Spark can access the table ...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16003 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69217/ Test PASSed. ---

[GitHub] spark issue #16003: [SPARK-18482][SQL] make sure Spark can access the table ...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16003 **[Test build #69217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69217/consoleFull)** for PR 16003 at commit

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89719801 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/NNLS.scala --- @@ -53,8 +53,13 @@ private[spark] object NNLS { * projected

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15986 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89719600 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/SQLTransformer.scala --- @@ -32,9 +32,11 @@ import org.apache.spark.sql.types.StructType

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15986 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69216/ Test FAILed. ---

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15986 **[Test build #69216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69216/consoleFull)** for PR 15986 at commit

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89719290 --- Diff: core/src/main/scala/org/apache/spark/rdd/JdbcRDD.scala --- @@ -41,7 +41,10 @@ private[spark] class JdbcPartition(idx: Int, val lower: Long,

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89719223 --- Diff: core/src/main/scala/org/apache/spark/rdd/DoubleRDDFunctions.scala --- @@ -152,10 +152,10 @@ class DoubleRDDFunctions(self: RDD[Double])

[GitHub] spark issue #10942: [SPARK-12850] [SQL] Support Bucket Pruning (Predicate Pu...

2016-11-27 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/10942 @gatorsmile, seems like getBuckets() is removed by below PR, which makes this feture not work any more, could you kindly help check? [SPARK-14535][SQL] Remove buildInternalScan from FileFormat

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16013 **[Test build #69219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69219/consoleFull)** for PR 16013 at commit

[GitHub] spark issue #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for unidoc...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16013 Let me leave some images that I changed and some comments to double check. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #16013: [WIP][SPARK-3359][DOCS] Make javadoc8 working for...

2016-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16013#discussion_r89718720 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -57,9 +57,17 @@ import

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89718517 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89718401 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89718250 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89718035 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark issue #16011: [SPARK-18587][ML] Remove handleInvalid from QuantileDisc...

2016-11-27 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16011 @MLnick Yeah, I think this is the most common case that copying Params from estimators to models. However, I also found some algorithms do not comply this rule, such as ```ALS``` which has

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89716676 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69214/ Test PASSed. ---

[GitHub] spark pull request #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskS...

2016-11-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15986#discussion_r89716522 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -350,20 +350,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15994 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15994 **[Test build #69214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69214/consoleFull)** for PR 15994 at commit

[GitHub] spark issue #16027: Make sure CollapseWindow returns the attributes in the s...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16027 Could you add `[SPARK-18604][SQL]` before merging? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskSetManag...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15986 **[Test build #69218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69218/consoleFull)** for PR 15986 at commit

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15994 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16027: Make sure CollapseWindow returns the attributes i...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16027#discussion_r89716395 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/CollapseWindowSuite.scala --- @@ -46,12 +46,15 @@ class

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69213/ Test PASSed. ---

[GitHub] spark issue #16027: Make sure CollapseWindow returns the attributes in the s...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16027 LGTM. Thank you for correct this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16027: Make sure CollapseWindow returns the attributes i...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16027#discussion_r89716308 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -545,7 +545,7 @@ object CollapseRepartition

[GitHub] spark issue #15994: [SPARK-18555][SQL]DataFrameNaFunctions.fill miss up orig...

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15994 **[Test build #69213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69213/consoleFull)** for PR 15994 at commit

[GitHub] spark pull request #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskS...

2016-11-27 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15986#discussion_r89716251 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -350,20 +350,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15874 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69215/ Test PASSed. ---

[GitHub] spark issue #15874: [Spark-18408][ML] API Improvements for LSH

2016-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15874 **[Test build #69215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69215/consoleFull)** for PR 15874 at commit

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89715587 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -590,7 +591,11 @@ object ScalaReflection extends

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89715546 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -590,7 +591,11 @@ object ScalaReflection extends

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89712321 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89713304 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -177,9 +177,10 @@ case class Invoke(

[GitHub] spark pull request #15780: [SPARK-18284][SQL] Make ExpressionEncoder.seriali...

2016-11-27 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/15780#discussion_r89712401 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -405,13 +406,14 @@ case class WrapOption(child:

[GitHub] spark issue #16027: Make sure CollapseWindow returns the attributes in the s...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16027 Oh, I missed this. Yep. I'll take a look at this, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16026: [SPARK-18597][SQL] Do push-down predicates to rig...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16026#discussion_r89715298 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -514,6 +514,39 @@ class

[GitHub] spark issue #16026: [SPARK-18597][SQL] Do push-down predicates to right side...

2016-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16026 LGTM, @hvanhovell . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16026: [SPARK-18597][SQL] Do push-down predicates to right side...

2016-11-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16026 LGTM except a minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskS...

2016-11-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15986#discussion_r89715118 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -350,20 +350,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #15986: [SPARK-18553][CORE][branch-2.0] Fix leak of TaskS...

2016-11-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/15986#discussion_r89715211 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -274,4 +276,70 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter met...

2016-11-27 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16017#discussion_r89715218 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -52,33 +52,49 @@ class DecisionTreeClassifier

  1   2   3   >