[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16782 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73713/ Test PASSed. ---

[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16782 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17129 **[Test build #73717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73717/testReport)** for PR 17129 at commit

[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/16782 I think this is ready for a final review @jkbradley @davies - thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17110: [SPARK-19635][ML] DataFrame-based API for chi squ...

2017-03-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17110#discussion_r103813679 --- Diff: mllib/src/test/scala/org/apache/spark/ml/stat/ChiSquareSuite.scala --- @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #14299: [SPARK-16440][MLlib] Ensure broadcasted variables are de...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14299 **[Test build #3588 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3588/testReport)** for PR 14299 at commit

[GitHub] spark issue #17106: [SPARK-19775][SQL] Remove an obsolete `partitionBy().ins...

2017-03-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17106 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16954: [SPARK-18874][SQL] First phase: Deferring the correlated...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16954 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16954: [SPARK-18874][SQL] First phase: Deferring the correlated...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16954 **[Test build #73712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73712/testReport)** for PR 16954 at commit

[GitHub] spark issue #16954: [SPARK-18874][SQL] First phase: Deferring the correlated...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73712/ Test PASSed. ---

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73720/testReport)** for PR 16944 at commit

[GitHub] spark issue #17067: [SPARK-19602][SQL][TESTS] Add tests for qualified column...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17067 **[Test build #73722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73722/testReport)** for PR 17067 at commit

[GitHub] spark issue #17067: [SPARK-19602][SQL][TESTS] Add tests for qualified column...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17067 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73722/ Test PASSed. ---

[GitHub] spark issue #17067: [SPARK-19602][SQL][TESTS] Add tests for qualified column...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17067 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16782 **[Test build #73713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73713/testReport)** for PR 16782 at commit

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/17081 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17081 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17081 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73716/ Test FAILed. ---

[GitHub] spark issue #17100: [SPARK-13947][PYTHON][SQL] PySpark DataFrames: The error...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17100 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17100: [SPARK-13947][PYTHON][SQL] PySpark DataFrames: The error...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73714/ Test PASSed. ---

[GitHub] spark issue #17100: [SPARK-13947][PYTHON][SQL] PySpark DataFrames: The error...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17100 **[Test build #73714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73714/testReport)** for PR 17100 at commit

[GitHub] spark pull request #17106: [SPARK-19775][SQL] Remove an obsolete `partitionB...

2017-03-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17106 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14299: [SPARK-16440][MLlib] Ensure broadcasted variables are de...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14299 **[Test build #3588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3588/testReport)** for PR 14299 at commit

[GitHub] spark issue #17124: [SPARK-19779][SS]Delete needless tmp file after restart ...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17124 **[Test build #3589 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3589/testReport)** for PR 17124 at commit

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17130 **[Test build #73719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73719/testReport)** for PR 17130 at commit

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/17081 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73715/testReport)** for PR 17081 at commit

[GitHub] spark issue #17031: [SPARK-19702][MESOS] Increase default refuse_seconds tim...

2017-03-01 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/17031 Your understanding is correct. You must set refuse_seconds for all your frameworks to some value N, such that N >= #frameworks. So for this change, if some operator is running >120 frameworks,

[GitHub] spark issue #17124: [SPARK-19779][SS]Delete needless tmp file after restart ...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17124 **[Test build #3589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3589/testReport)** for PR 17124 at commit

[GitHub] spark pull request #17122: [SPARK-19786][SQL] Facilitate loop optimizations ...

2017-03-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17122#discussion_r103838737 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -77,6 +77,10 @@ trait CodegenSupport extends SparkPlan

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17129 @mgummelt I merged this to 2.1 (and so you will have to close this PR manually) but the cherry-pick to 2.0.x doesn't succeed either, and it's non-trivial. If you're willing to evaluate the conflict

[GitHub] spark pull request #17130: [SPARK-19791] [ML] Add doc and example for fpgrow...

2017-03-01 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17130#discussion_r103822448 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -240,12 +240,13 @@ class FPGrowthModel private[ml] ( val predictUDF =

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17059 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17059 **[Test build #73718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73718/testReport)** for PR 17059 at commit

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73718/ Test PASSed. ---

[GitHub] spark issue #17067: [SPARK-19602][SQL][TESTS] Add tests for qualified column...

2017-03-01 Thread skambha
Github user skambha commented on the issue: https://github.com/apache/spark/pull/17067 - Changes to the SQLQueryTestSuite framework to mask the exprId so I can add the -ve cases as well using this framework. - Added -ve test cases to the SQLQueryTestSuite framework and so removed

[GitHub] spark issue #17067: [SPARK-19602][SQL][TESTS] Add tests for qualified column...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17067 **[Test build #73722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73722/testReport)** for PR 17067 at commit

[GitHub] spark pull request #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegi...

2017-03-01 Thread mgummelt
Github user mgummelt closed the pull request at: https://github.com/apache/spark/pull/17129 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-01 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/17120 @steveloughran thanks for the comments. @marmbrus @zsxwing it'd be great if you could share some thoughts! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17059 **[Test build #73718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73718/testReport)** for PR 17059 at commit

[GitHub] spark issue #17106: [SPARK-19775][SQL] Remove an obsolete `partitionBy().ins...

2017-03-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17106 Thank you for merging, @srowen . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73716/testReport)** for PR 17081 at commit

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #73720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73720/testReport)** for PR 16944 at commit

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17130 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17130 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73719/ Test FAILed. ---

[GitHub] spark issue #17130: [SPARK-19791] [ML] Add doc and example for fpgrowth

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17130 **[Test build #73719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73719/testReport)** for PR 17130 at commit

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16909 **[Test build #73723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73723/testReport)** for PR 16909 at commit

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73720/ Test FAILed. ---

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73724/testReport)** for PR 17081 at commit

[GitHub] spark pull request #17110: [SPARK-19635][ML] DataFrame-based API for chi squ...

2017-03-01 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17110#discussion_r103813169 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/ChiSquare.scala --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/17129 2.1 is sufficient. Thanks for the merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17123: [SPARK-19781][ML] Handle NULLs as well as NaNs in Bucket...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17123 **[Test build #3590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3590/testReport)** for PR 17123 at commit

[GitHub] spark pull request #17122: [SPARK-19786][SQL] Facilitate loop optimizations ...

2017-03-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17122#discussion_r103837174 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -434,6 +434,17 @@ case class RangeExec(range:

[GitHub] spark issue #17100: [SPARK-13947][PYTHON][SQL] PySpark DataFrames: The error...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17100 **[Test build #73714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73714/testReport)** for PR 17100 at commit

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73716/testReport)** for PR 17081 at commit

[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16782 **[Test build #73709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73709/testReport)** for PR 16782 at commit

[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16782 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73709/ Test PASSed. ---

[GitHub] spark issue #16782: [SPARK-19348][PYTHON] PySpark keyword_only decorator is ...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16782 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #73721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73721/testReport)** for PR 15505 at commit

[GitHub] spark issue #17125: [SPARK-19211][SQL] Explicitly prevent Insert into View o...

2017-03-01 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/17125 cc @gatorsmile @cloud-fan Please have a look at this when you have time, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #17099: [SPARK-19766][SQL] Constant alias columns in INNER JOIN ...

2017-03-01 Thread stanzhai
Github user stanzhai commented on the issue: https://github.com/apache/spark/pull/17099 ok --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegi...

2017-03-01 Thread mgummelt
GitHub user mgummelt opened a pull request: https://github.com/apache/spark/pull/17129 [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredResourceRatio … …on registered cores rather than accepted cores See JIRA Unit tests, Mesos/Spark integration tests

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/17129 @srowen As discussed here https://github.com/apache/spark/pull/17045#issuecomment-283192230, this is the backport of SPARK-19373 into branch-2.1 --- If your project is set up for it, you can

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73717/ Test PASSed. ---

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17129 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17129: [SPARK-19373][MESOS] Base spark.scheduler.minRegisteredR...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17129 **[Test build #73717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73717/testReport)** for PR 17129 at commit

[GitHub] spark issue #17123: [SPARK-19781][ML] Handle NULLs as well as NaNs in Bucket...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17123 **[Test build #3590 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3590/testReport)** for PR 17123 at commit

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17081 **[Test build #73715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73715/testReport)** for PR 17081 at commit

[GitHub] spark pull request #17130: [SPARK-19791] [ML] Add doc and example for fpgrow...

2017-03-01 Thread hhbyyh
GitHub user hhbyyh opened a pull request: https://github.com/apache/spark/pull/17130 [SPARK-19791] [ML] Add doc and example for fpgrowth ## What changes were proposed in this pull request? Add a new section for fpm Add Example for FPGrowth in scala and Java ##

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17081 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73715/ Test FAILed. ---

[GitHub] spark issue #17081: [SPARK-18726][SQL][FOLLOW-UP]resolveRelation for FileFor...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17081 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17113: [SPARK-13669][Core] Improve the blacklist mechanism to h...

2017-03-01 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17113 @tgravescs , the main scenario is external shuffle service unavailable scenario, this could be happened in working preserving + NM failure situation. Also like Mesos + external standalone shuffle

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-03-01 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17052 \cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17080: [SPARK-19739][CORE] propagate S3 session token to cluser

2017-03-01 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/17080 \cc @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #17099: [SPARK-19766][SQL] Constant alias columns in INNE...

2017-03-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17099 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17099: [SPARK-19766][SQL] Constant alias columns in INNER JOIN ...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17099 Thanks! Merging to master/2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17099: [SPARK-19766][SQL] Constant alias columns in INNER JOIN ...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17099 @stanzhai Could you submit another PR to backport it to Spark 2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17093: [SPARK-19761][SQL]create InMemoryFileIndex with an empty...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17093 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17121: [SPARK-19787][ML] Changing the default parameter of regP...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17121 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17123: [SPARK-19781][ML] Handle NULLs as well as NaNs in Bucket...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17123 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17122: [SPARK-19786][SQL] Facilitate loop optimizations in a JI...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17122 **[Test build #73689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73689/testReport)** for PR 17122 at commit

[GitHub] spark issue #17119: [SPARK-19784][SQL][WIP]refresh table after alter the loc...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17119 **[Test build #73690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73690/testReport)** for PR 17119 at commit

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17120 **[Test build #73691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73691/testReport)** for PR 17120 at commit

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9571 **[Test build #73695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73695/testReport)** for PR 9571 at commit

[GitHub] spark issue #17059: [SPARK-19733][ML]Removed unnecessary castings and refact...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17059 **[Test build #73693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73693/testReport)** for PR 17059 at commit

[GitHub] spark issue #17116: [SPARK-18890][CORE](try 2) Move task serialization from ...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17116 **[Test build #73692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73692/testReport)** for PR 17116 at commit

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #73694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73694/testReport)** for PR 15505 at commit

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9571 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73695/ Test FAILed. ---

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9571 **[Test build #73695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73695/testReport)** for PR 9571 at commit

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-03-01 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/9571 Style police. FWIW I think the lines that failed were already >100 chars, it was just they got indented slightly more. ``` Scalastyle checks failed at following occurrences: [error]

[GitHub] spark pull request #17093: [SPARK-19761][SQL]create InMemoryFileIndex with a...

2017-03-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17093 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17120: [SPARK-19715][Structured Streaming] Option to Strip Path...

2017-03-01 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/17120 -1, non binding I understand the rationale for this, to aid migration from s3/s3n to s3a, but given the need is schema independence, you should be using the full path name from

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2017-03-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9571 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17093: [SPARK-19761][SQL]create InMemoryFileIndex with an empty...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17093 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-03-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16909 **[Test build #73723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73723/testReport)** for PR 16909 at commit

[GitHub] spark pull request #17099: [SPARK-19766][SQL] Constant alias columns in INNE...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17099#discussion_r103844650 --- Diff: sql/core/src/test/resources/sql-tests/results/inner-join.sql.out --- @@ -0,0 +1,68 @@ +-- Automatically generated by SQLQueryTestSuite

[GitHub] spark issue #15505: [SPARK-18890][CORE] Move task serialization from the Tas...

2017-03-01 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/15505 @kayousterhout It takes some time to update the test report. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103850385 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -365,17 +385,66 @@ object TypeCoercion {

[GitHub] spark pull request #16954: [SPARK-18874][SQL] First phase: Deferring the cor...

2017-03-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16954#discussion_r103852025 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -83,29 +95,150 @@ object RewritePredicateSubquery

  1   2   3   4   5   6   >