[GitHub] spark issue #15935: [SPARK-18188] add checksum for blocks of broadcast

2016-11-29 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15935 LGTM. Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15935: [SPARK-18188] add checksum for blocks of broadcas...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15935 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15975#discussion_r89956606 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -76,9 +76,6 @@ class JDBCOptions(

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15975#discussion_r89956695 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -209,6 +209,14 @@ class JDBCSuite extends SparkFunSuite

[GitHub] spark pull request #15975: [SPARK-18538] [SQL] Fix Concurrent Table Fetching...

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15975#discussion_r89956633 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -667,13 +667,13 @@ object JdbcUtils extends

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16052 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter methods to...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16017 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69306/ Test FAILed. ---

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16048 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69305/ Test FAILed. ---

[GitHub] spark issue #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter methods to...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16017 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69309/ Test FAILed. ---

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69307/ Test FAILed. ---

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16045 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69308/ Test FAILed. ---

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16052: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" fe...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16052 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69303/ Test FAILed. ---

[GitHub] spark issue #15954: [WIP][SPARK-18516][SQL] Split state and progress in stre...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69295/ Test FAILed. ---

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16045 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69304/ Test FAILed. ---

[GitHub] spark pull request #16055: [SPARK-17897] [SQL] Attribute is not NullIntolera...

2016-11-29 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/16055 [SPARK-17897] [SQL] Attribute is not NullIntolerant ### What changes were proposed in this pull request? `Attribute` is not `NullIntolerant`. This PR is to fix it. Without the fix,

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-11-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16030 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16055: [SPARK-17897] [SQL] Attribute is not NullIntolerant

2016-11-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16055 Can you explain how did nullintolerant impact the case? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16055: [SPARK-17897] [SQL] Attribute is not NullIntolerant

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16055 **[Test build #69310 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69310/consoleFull)** for PR 16055 at commit

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16030 **[Test build #69312 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69312/consoleFull)** for PR 16030 at commit

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16048 **[Test build #69311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69311/consoleFull)** for PR 16048 at commit

[GitHub] spark issue #16055: [SPARK-17897] [SQL] Attribute is not NullIntolerant

2016-11-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16055 Sure, will update the PR description tomorrow. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter methods to...

2016-11-29 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16017 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter methods to...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16017 **[Test build #69313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69313/consoleFull)** for PR 16017 at commit

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16014 @shivaram Cool, I did know about release-build but I didn't know it's running on Jenkins. I *think* we should be ok but might want to check Jenkins has "e1071" and "survival" which are optional

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #69314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69314/consoleFull)** for PR 16028 at commit

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2016-11-29 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16028 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16014: [SPARK-18590][SPARKR] build R source package when making...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16014 **[Test build #69315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69315/consoleFull)** for PR 16014 at commit

[GitHub] spark issue #16061: [SPARK-18278] [Scheduler] Support native submission of s...

2016-11-29 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/16061 @erikerlandson For the RAT failure, you may either add Apache license header to newly added files or add the file to `dev/.rat-excludes`. --- If your project is set up for it, you can reply to

[GitHub] spark issue #15972: [SPARK-18319][ML][QA2.1] 2.1 QA: API: Experimental, Deve...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15972 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69338/ Test PASSed. ---

[GitHub] spark pull request #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter met...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16017 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16064: [SPARK-18633][ML][Example]: Add multiclass logist...

2016-11-29 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/16064 [SPARK-18633][ML][Example]: Add multiclass logistic regression summary python example and document ## What changes were proposed in this pull request? Logistic Regression summary is added

[GitHub] spark issue #16064: [SPARK-18633][ML][Example]: Add multiclass logistic regr...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16064 **[Test build #69344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69344/consoleFull)** for PR 16064 at commit

[GitHub] spark pull request #15924: [SPARK-18498] [SQL] Revise HDFSMetadataLog API fo...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15924 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16065: [SPARK-18631][SQL] Changed ExchangeCoordinator re-partit...

2016-11-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16065 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15998: [SPARK-18572][SQL] Add a method `listPartitionNames` to ...

2016-11-29 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15998 * looks good once InMemoryCatalog is fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15954: [SPARK-18516][SQL] Split state and progress in st...

2016-11-29 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/15954#discussion_r90130320 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQuery.scala --- @@ -64,23 +68,26 @@ trait StreamingQuery { /**

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16048 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KM...

2016-11-29 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/13557#discussion_r90132591 --- Diff: python/pyspark/ml/clustering.py --- @@ -316,6 +316,36 @@ def computeCost(self, dataset): """ return

[GitHub] spark issue #16065: [SPARK-18631][SQL] Changed ExchangeCoordinator re-partit...

2016-11-29 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/16065 Thanks @markhamstra Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69346/ Test PASSed. ---

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16068: stateful udf should be nondeterministic

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16068 **[Test build #69361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69361/consoleFull)** for PR 16068 at commit

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16066 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16066 **[Test build #69349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69349/consoleFull)** for PR 16066 at commit

[GitHub] spark issue #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint Inference...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16067 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint Inference...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16067 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69350/ Test PASSed. ---

[GitHub] spark issue #16045: [SPARK-18553][CORE] Fix leak of TaskSetManager following...

2016-11-29 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16045 Cool, I'm going to merge this into master and branch-2.1 in that case. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15954 **[Test build #69355 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69355/consoleFull)** for PR 15954 at commit

[GitHub] spark issue #15255: [SPARK-17680] [SQL] [TEST] Added a Testcase for Verifyin...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15255 **[Test build #69357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69357/consoleFull)** for PR 15255 at commit

[GitHub] spark issue #16065: [SPARK-18631][SQL] Changed ExchangeCoordinator re-partit...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16065 **[Test build #69347 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69347/consoleFull)** for PR 16065 at commit

[GitHub] spark issue #16048: [DO_NOT_MERGE]Test kafka deletion

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16048 **[Test build #69359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69359/consoleFull)** for PR 16048 at commit

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15982 **[Test build #69360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69360/consoleFull)** for PR 15982 at commit

[GitHub] spark pull request #16065: [SPARK-18631][SQL] Changed ExchangeCoordinator re...

2016-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16065 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #14164: [SPARK-16629] Allow comparisons between UDTs and ...

2016-11-29 Thread damnMeddlingKid
Github user damnMeddlingKid closed the pull request at: https://github.com/apache/spark/pull/14164 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13557: [SPARK-15819][PYSPARK][ML] Add KMeanSummary in KMeans of...

2016-11-29 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13557 Thanks for updating this @zjffdu, it looks good to me now that @sethah's comments have been addressed maybe we can get @davies or @MLnick to take a final pass? --- If your project is set up for

[GitHub] spark pull request #15496: [SPARK-17950] [Python] Match SparseVector behavio...

2016-11-29 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/15496#discussion_r90139924 --- Diff: python/pyspark/ml/linalg/__init__.py --- @@ -705,6 +705,22 @@ def __eq__(self, other): return Vectors._equals(self.indices,

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15982 **[Test build #69348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69348/consoleFull)** for PR 15982 at commit

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14638 **[Test build #69362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69362/consoleFull)** for PR 14638 at commit

[GitHub] spark pull request #15946: [SPARK-18513][Structured Streaming] Record and re...

2016-11-29 Thread lw-lin
Github user lw-lin closed the pull request at: https://github.com/apache/spark/pull/15946 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15946: [SPARK-18513][Structured Streaming] Record and recover w...

2016-11-29 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/15946 sure! closing this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #15954: [SPARK-18516][SQL] Split state and progress in st...

2016-11-29 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15954#discussion_r90147041 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQuery.scala --- @@ -33,25 +35,27 @@ trait StreamingQuery { * Returns the name

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15780 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15780: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nul...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15780 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69342/ Test PASSed. ---

[GitHub] spark issue #16065: [SPARK-18631][SQL] Changed ExchangeCoordinator re-partit...

2016-11-29 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/16065 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r88075283 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapRedCommitProtocol.scala --- @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r90122251 --- Diff: core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala --- @@ -0,0 +1,408 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r90121527 --- Diff: core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala --- @@ -0,0 +1,408 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r90120144 --- Diff: core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala --- @@ -0,0 +1,408 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r90127987 --- Diff: core/src/main/scala/org/apache/spark/internal/io/SparkHadoopWriter.scala --- @@ -0,0 +1,408 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint Inference...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16067 **[Test build #69354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69354/consoleFull)** for PR 16067 at commit

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r90129155 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -1089,66 +1064,10 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark pull request #15861: [SPARK-18294][CORE] Implement commit protocol to ...

2016-11-29 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/15861#discussion_r90116879 --- Diff: core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala --- @@ -1016,11 +1013,6 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)])

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15954 **[Test build #69353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69353/consoleFull)** for PR 15954 at commit

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15954 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69353/ Test FAILed. ---

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16066 **[Test build #69356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69356/consoleFull)** for PR 16066 at commit

[GitHub] spark pull request #15982: [SPARK-18546][core] Fix merging shuffle spills wh...

2016-11-29 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/15982#discussion_r90133223 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java --- @@ -337,42 +340,47 @@ void forceSorterToSpill() throws IOException {

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16066 **[Test build #69358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69358/consoleFull)** for PR 16066 at commit

[GitHub] spark pull request #15954: [SPARK-18516][SQL] Split state and progress in st...

2016-11-29 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/15954#discussion_r90133572 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -59,13 +62,20 @@ class StreamingQueryManager

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16044 LGTM - merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15954 **[Test build #69346 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69346/consoleFull)** for PR 15954 at commit

[GitHub] spark pull request #16068: stateful udf should be nondeterministic

2016-11-29 Thread zhzhan
GitHub user zhzhan opened a pull request: https://github.com/apache/spark/pull/16068 stateful udf should be nondeterministic ## What changes were proposed in this pull request? Make stateful udf as nondeterministic ## How was this patch tested? Mainly

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15982 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15982: [SPARK-18546][core] Fix merging shuffle spills when usin...

2016-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69348/ Test PASSed. ---

[GitHub] spark issue #16067: [SPARK-17897] [SQL] Fixed IsNotNull Constraint Inference...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16067 **[Test build #69350 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69350/consoleFull)** for PR 16067 at commit

[GitHub] spark issue #16063: [SPARK-18622][SQL] Remove TypeCoercion rules for Average...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16063 **[Test build #69363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69363/consoleFull)** for PR 16063 at commit

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16066 **[Test build #69358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69358/consoleFull)** for PR 16066 at commit

[GitHub] spark issue #15954: [SPARK-18516][SQL] Split state and progress in streaming

2016-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15954 **[Test build #69353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69353/consoleFull)** for PR 15954 at commit

[GitHub] spark pull request #15982: [SPARK-18546][core] Fix merging shuffle spills wh...

2016-11-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15982#discussion_r90127514 --- Diff: core/src/test/java/org/apache/spark/unsafe/map/AbstractBytesToBytesMapSuite.java --- @@ -75,13 +75,6 @@ @Mock(answer = RETURNS_SMART_NULLS)

[GitHub] spark pull request #15982: [SPARK-18546][core] Fix merging shuffle spills wh...

2016-11-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15982#discussion_r90121766 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java --- @@ -337,42 +340,47 @@ void forceSorterToSpill() throws IOException {

[GitHub] spark pull request #15982: [SPARK-18546][core] Fix merging shuffle spills wh...

2016-11-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15982#discussion_r90127615 --- Diff: core/src/test/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorterSuite.java --- @@ -86,14 +88,7 @@ public int compare(

[GitHub] spark pull request #15982: [SPARK-18546][core] Fix merging shuffle spills wh...

2016-11-29 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15982#discussion_r90126726 --- Diff: core/src/test/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriterSuite.java --- @@ -40,9 +41,11 @@ import org.mockito.stubbing.Answer;

  1   2   3   4   5   6   7   >