[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75327/testReport)** for PR 17436 at commit

[GitHub] spark issue #17407: [SPARK-20043][ML] DecisionTreeModel: ImpurityCalculator ...

2017-03-28 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17407 LGTM pending tests Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17407: [SPARK-20043][ML] DecisionTreeModel: ImpurityCalculator ...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17407 **[Test build #3618 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3618/testReport)** for PR 17407 at commit

[GitHub] spark issue #17416: [SPARK-20075][CORE][WIP] Support classifier, packaging i...

2017-03-28 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17416 Thanks @BryanCutler , here's the current full output after trying to fix per https://github.com/apache/spark/pull/17416#discussion_r108508229 . Looks like progress I think but still missed something

[GitHub] spark issue #17052: [SPARK-19690][SS] Join a streaming DataFrame with a batc...

2017-03-28 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17052 This is marked as "Critical" for 2.1.1, but I'm not clear it's a regression or that urgent? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17436 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75325/ Test FAILed. ---

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75325/testReport)** for PR 17436 at commit

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17460 @sureshthalamati Could you please close this PR, since it has been merged? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread sureshthalamati
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/17460 Thank you @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle nu...

2017-03-28 Thread sureshthalamati
Github user sureshthalamati closed the pull request at: https://github.com/apache/spark/pull/17460 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17460 Thanks! Merging to 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17421: [SPARK-20040][ML][python] pyspark wrapper for Chi...

2017-03-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17421#discussion_r108535695 --- Diff: dev/sparktestsupport/modules.py --- @@ -431,6 +431,7 @@ def __hash__(self): "pyspark.ml.linalg.__init__",

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75324/ Test PASSed. ---

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17460 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17460 **[Test build #75324 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75324/testReport)** for PR 17460 at commit

[GitHub] spark pull request #17355: [SPARK-19955][PySpark] Jenkins Python Conda based...

2017-03-28 Thread shaneknapp
Github user shaneknapp commented on a diff in the pull request: https://github.com/apache/spark/pull/17355#discussion_r108534754 --- Diff: dev/run-pip-tests --- @@ -35,32 +35,38 @@ function delete_virtualenv() { } trap delete_virtualenv EXIT +PYTHON_EXECS=()

[GitHub] spark issue #17370: [SPARK-20042] [Web UI] Fix log page buttons for reverse ...

2017-03-28 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17370 @srowen can you ok to test this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-03-28 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/16985 @hvanhovell @cloud-fan @gatorsmile can anyone please review this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17379: [SPARK-20048][SQL] Cloning SessionState does not ...

2017-03-28 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/17379#discussion_r108532483 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -38,10 +38,7 @@ import

[GitHub] spark pull request #17379: [SPARK-20048][SQL] Cloning SessionState does not ...

2017-03-28 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/17379#discussion_r108532136 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -134,17 +131,20 @@ private[sql] class SessionState(

[GitHub] spark pull request #17379: [SPARK-20048][SQL] Cloning SessionState does not ...

2017-03-28 Thread kunalkhamar
Github user kunalkhamar commented on a diff in the pull request: https://github.com/apache/spark/pull/17379#discussion_r108531631 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -32,19 +32,9 @@ import

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75323/ Test PASSed. ---

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17295 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75326/ Test FAILed. ---

[GitHub] spark issue #17379: [SPARK-20048][SQL] Cloning SessionState does not clone q...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17379 **[Test build #75328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75328/testReport)** for PR 17379 at commit

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17394 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17295 **[Test build #75323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75323/testReport)** for PR 17295 at commit

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17394 **[Test build #75326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75326/testReport)** for PR 17394 at commit

[GitHub] spark pull request #17450: [SPARK-20121][SQL] simplify NullPropagation with ...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17450#discussion_r108530204 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1122,7 +1119,7 @@ case class

[GitHub] spark issue #17450: [SPARK-20121][SQL] simplify NullPropagation with NullInt...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17450 Great! LGTM except a minor comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17450: [SPARK-20121][SQL] simplify NullPropagation with ...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17450#discussion_r108529414 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -384,33 +379,13 @@ case class

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread dbolshak
Github user dbolshak commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108529238 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagesTab.scala --- @@ -35,7 +35,7 @@ private[ui] class StagesTab(parent: SparkUI) extends

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108528285 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -103,7 +103,7 @@ private[ui] class StagePage(parent: StagesTab) extends

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread dbolshak
Github user dbolshak commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108528243 --- Diff: core/src/main/scala/org/apache/spark/ui/storage/RDDPage.scala --- @@ -69,7 +69,7 @@ private[ui] class RDDPage(parent: StorageTab) extends

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread dbolshak
Github user dbolshak commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108527691 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -103,7 +103,7 @@ private[ui] class StagePage(parent: StagesTab) extends

[GitHub] spark pull request #17416: [SPARK-20075][CORE][WIP] Support classifier, pack...

2017-03-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17416#discussion_r108526688 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -829,33 +830,56 @@ private[spark] object SparkSubmitUtils { var

[GitHub] spark issue #17461: [SPARK-20082][ml][WIP] LDA incremental model learning

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17461 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17421: [SPARK-20040][ML][python] pyspark wrapper for ChiSquareT...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17421 **[Test build #3617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3617/testReport)** for PR 17421 at commit

[GitHub] spark pull request #17461: [SPARK-20082][ml][WIP] LDA incremental model lear...

2017-03-28 Thread mdespriee
GitHub user mdespriee opened a pull request: https://github.com/apache/spark/pull/17461 [SPARK-20082][ml][WIP] LDA incremental model learning ## What changes were proposed in this pull request? This PR implements an initialModel parameter for LDA. (Right now, only

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108521117 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala --- @@ -34,9 +34,9 @@ private[ui] class AllStagesPage(parent: StagesTab) extends

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108521234 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -103,7 +103,7 @@ private[ui] class StagePage(parent: StagesTab) extends

[GitHub] spark issue #17401: [SPARK-18364][YARN] Expose metrics for YarnShuffleServic...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17401 **[Test build #75322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75322/testReport)** for PR 17401 at commit

[GitHub] spark issue #17401: [SPARK-18364][YARN] Expose metrics for YarnShuffleServic...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17401 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75322/ Test PASSed. ---

[GitHub] spark issue #17401: [SPARK-18364][YARN] Expose metrics for YarnShuffleServic...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17401 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17458: [SPARK-20127][CORE] few warning have been fixed w...

2017-03-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17458#discussion_r108521452 --- Diff: core/src/main/scala/org/apache/spark/ui/storage/RDDPage.scala --- @@ -69,7 +69,7 @@ private[ui] class RDDPage(parent: StorageTab) extends

[GitHub] spark pull request #17442: [SPARK-20107][SQL] Speed up HadoopMapReduceCommit...

2017-03-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17442#discussion_r108519339 --- Diff: docs/configuration.md --- @@ -1137,6 +1137,17 @@ Apart from these, the following properties are also available, and may be useful mapping

[GitHub] spark pull request #17450: [SPARK-20121][SQL] simplify NullPropagation with ...

2017-03-28 Thread nsyca
Github user nsyca commented on a diff in the pull request: https://github.com/apache/spark/pull/17450#discussion_r108519047 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1122,7 +1119,7 @@ case class

[GitHub] spark issue #17458: [SPARK-20127][CORE] few warning have been fixed which In...

2017-03-28 Thread dbolshak
Github user dbolshak commented on the issue: https://github.com/apache/spark/pull/17458 @HyukjinKwon could you please explain how to request to merge the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17446: [SPARK-17075][SQL][followup] Add Estimation of Constant ...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17446 @wzhfy `BooleanSimplification ` is unable to get rid of `null`. Boolean operations on null is complex. We need more investigation. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75327/testReport)** for PR 17436 at commit

[GitHub] spark issue #17454: [SPARK-20125][SQL] Dataset of type option of map does no...

2017-03-28 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/17454 OK, resolved the conflict manually and merged to branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17394 **[Test build #75326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75326/testReport)** for PR 17394 at commit

[GitHub] spark issue #17406: [SPARK-20009][SQL] Use DDL strings for defining schema i...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17406 @maropu Could you update the PR title? Then, I can merge it once it is done. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #17208: [SPARK-19868] conflict TasksetManager lead to spa...

2017-03-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17208 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17208 LGTM merged this to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17421: [SPARK-20040][ML][python] pyspark wrapper for Chi...

2017-03-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17421#discussion_r108511847 --- Diff: dev/sparktestsupport/modules.py --- @@ -431,6 +431,7 @@ def __hash__(self): "pyspark.ml.linalg.__init__",

[GitHub] spark issue #17454: [SPARK-20125][SQL] Dataset of type option of map does no...

2017-03-28 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/17454 This PR conflicts with branch-2.1, trying to resolve manually while merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17436 **[Test build #75325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75325/testReport)** for PR 17436 at commit

[GitHub] spark pull request #17416: [SPARK-20075][CORE][WIP] Support classifier, pack...

2017-03-28 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17416#discussion_r108508229 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -941,6 +965,12 @@ private[spark] object SparkSubmitUtils { val ri =

[GitHub] spark pull request #17416: [SPARK-20075][CORE][WIP] Support classifier, pack...

2017-03-28 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17416#discussion_r108507118 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -829,33 +830,56 @@ private[spark] object SparkSubmitUtils { var

[GitHub] spark pull request #17454: [SPARK-20125][SQL] Dataset of type option of map ...

2017-03-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17454 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17454: [SPARK-20125][SQL] Dataset of type option of map does no...

2017-03-28 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/17454 LGTM, merging to master and branch-2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17416: [SPARK-20075][CORE][WIP] Support classifier, pack...

2017-03-28 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/17416#discussion_r108502945 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -941,6 +965,12 @@ private[spark] object SparkSubmitUtils { val

[GitHub] spark issue #15192: [SPARK-14536] [SQL] fix to handle null value in array ty...

2017-03-28 Thread sureshthalamati
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/15192 Created PR to back port this fix to 2.1: https://github.com/apache/spark/pull/17460 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-28 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17297 Agree sounds good! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null valu...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17460 **[Test build #75324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75324/testReport)** for PR 17460 at commit

[GitHub] spark pull request #17460: [SPARK-14536][SQL][BACKPORT-2.1] fix to handle nu...

2017-03-28 Thread sureshthalamati
GitHub user sureshthalamati opened a pull request: https://github.com/apache/spark/pull/17460 [SPARK-14536][SQL][BACKPORT-2.1] fix to handle null value in array type column for postgres. ## What changes were proposed in this pull request? JDBC read is failing with NPE due to

[GitHub] spark issue #17297: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2017-03-28 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/17297 Sounds good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17394: [SPARK-20067] [SQL] Unify and Clean Up Desc Commands Usi...

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17394 Yes. We do not need to keep the difference between `DESC EXTENDED` and `DESC FORMATTED` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #13932: [SPARK-15354] [CORE] Topology aware block replica...

2017-03-28 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/13932#discussion_r108495070 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockReplicationPolicy.scala --- @@ -53,6 +53,46 @@ trait BlockReplicationPolicy {

[GitHub] spark pull request #17402: [SPARK-7200] Add cleanup for memory leak tests to...

2017-03-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17402#discussion_r108491146 --- Diff: core/src/test/java/org/apache/spark/memory/TaskMemoryManagerSuite.java --- @@ -25,27 +26,37 @@ public class TaskMemoryManagerSuite {

[GitHub] spark pull request #17402: [SPARK-7200] Add cleanup for memory leak tests to...

2017-03-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17402#discussion_r108491343 --- Diff: core/src/test/java/org/apache/spark/memory/TaskMemoryManagerSuite.java --- @@ -25,27 +26,37 @@ public class TaskMemoryManagerSuite {

[GitHub] spark pull request #17402: [SPARK-7200] Add cleanup for memory leak tests to...

2017-03-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17402#discussion_r108492013 --- Diff: core/src/test/java/org/apache/spark/memory/TaskMemoryManagerSuite.java --- @@ -25,27 +26,37 @@ public class TaskMemoryManagerSuite {

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75316/ Test PASSed. ---

[GitHub] spark issue #17388: [SPARK-20059][YARN] Use the correct classloader for HBas...

2017-03-28 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17388 Minor comment on a comment, otherwise LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17388: [SPARK-20059][YARN] Use the correct classloader f...

2017-03-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17388#discussion_r108490519 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -485,12 +485,17 @@ object SparkSubmit extends CommandLineUtils {

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #75316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75316/testReport)** for PR 15009 at commit

[GitHub] spark issue #17295: [SPARK-19556][core] Do not encrypt block manager data in...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17295 **[Test build #75323 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75323/testReport)** for PR 17295 at commit

[GitHub] spark pull request #17335: [SPARK-19995][YARN] Register tokens to current UG...

2017-03-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17335: [SPARK-19995][YARN] Register tokens to current UGI to av...

2017-03-28 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17335 LGTM, merging to master / 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13932: [SPARK-15354] [CORE] Topology aware block replication st...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13932 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16985 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75319/ Test PASSed. ---

[GitHub] spark issue #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added if joi...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16985 **[Test build #75319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75319/testReport)** for PR 16985 at commit

[GitHub] spark issue #13932: [SPARK-15354] [CORE] Topology aware block replication st...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13932 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75315/ Test PASSed. ---

[GitHub] spark issue #17421: [SPARK-20040][ML][python] pyspark wrapper for ChiSquareT...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17421 **[Test build #3617 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3617/testReport)** for PR 17421 at commit

[GitHub] spark issue #13932: [SPARK-15354] [CORE] Topology aware block replication st...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13932 **[Test build #75315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75315/testReport)** for PR 13932 at commit

[GitHub] spark pull request #17421: [SPARK-20040][ML][python] pyspark wrapper for Chi...

2017-03-28 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17421#discussion_r108485003 --- Diff: dev/sparktestsupport/modules.py --- @@ -431,6 +431,7 @@ def __hash__(self): "pyspark.ml.linalg.__init__",

[GitHub] spark issue #17450: [SPARK-20121][SQL] simplify NullPropagation with NullInt...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17450 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17450: [SPARK-20121][SQL] simplify NullPropagation with NullInt...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17450 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75318/ Test PASSed. ---

[GitHub] spark issue #17450: [SPARK-20121][SQL] simplify NullPropagation with NullInt...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17450 **[Test build #75318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75318/testReport)** for PR 17450 at commit

[GitHub] spark pull request #17457: [SPARK-20126][SQL] Remove HiveSessionState

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17457#discussion_r108483107 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala --- @@ -379,8 +379,7 @@ class MetastoreDataSourcesSuite

[GitHub] spark issue #17457: [SPARK-20126][SQL] Remove HiveSessionState

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17457 Maybe it is time to clean up `HiveMetastoreCatalog`? Are you working on it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17457: [SPARK-20126][SQL] Remove HiveSessionState

2017-03-28 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17457 A late LGTM. : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17398: [SPARK-19716][SQL] support by-name resolution for struct...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17398 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75317/ Test PASSed. ---

[GitHub] spark issue #17398: [SPARK-19716][SQL] support by-name resolution for struct...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17398 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17398: [SPARK-19716][SQL] support by-name resolution for struct...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17398 **[Test build #75317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75317/testReport)** for PR 17398 at commit

<    1   2   3   4   >