[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75582/ Test PASSed. ---

[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17553 **[Test build #75582 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75582/testReport)** for PR 17553 at commit

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17540 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75581/ Test FAILed. ---

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17540 **[Test build #75581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75581/testReport)** for PR 17540 at commit

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17540 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17553 **[Test build #75582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75582/testReport)** for PR 17553 at commit

[GitHub] spark issue #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR in pro...

2017-04-06 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/17553 @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #17553: [SPARK-20026][Doc] Add Tweedie example for SparkR...

2017-04-06 Thread actuaryzhang
GitHub user actuaryzhang opened a pull request: https://github.com/apache/spark/pull/17553 [SPARK-20026][Doc] Add Tweedie example for SparkR in programming guide ## What changes were proposed in this pull request? Add Tweedie example for SparkR in programming guide. The doc

[GitHub] spark issue #16354: [SPARK-18886][Scheduler][WIP] Adjust Delay scheduling to...

2017-04-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/16354 closing this since on the jira we decided that a different approach is needed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #16354: [SPARK-18886][Scheduler][WIP] Adjust Delay schedu...

2017-04-06 Thread squito
Github user squito closed the pull request at: https://github.com/apache/spark/pull/16354 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17551 > Our use case involves jobs running in a remote cluster without a Spark master. It's still running your code, right? Why can't you add a configuration to your own code that tells it to

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17552 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75578/ Test FAILed. ---

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/17540 The new test failures are caused by a check I inserted. Moving where `withNewExecutionId` gets called could result in missing SQL queries in the UI, so anywhere I'm replacing `withNewExecutionId`

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17552 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17540 **[Test build #75581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75581/testReport)** for PR 17540 at commit

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17552 **[Test build #75578 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75578/testReport)** for PR 17552 at commit

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14617 btw, anybody interested in looking at getting the memory to show up in the history server as well? this issue we were discussing earlier: >> AFAIK we don't record block update events in

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17540 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17540 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75579/ Test FAILed. ---

[GitHub] spark pull request #14617: [SPARK-17019][Core] Expose on-heap and off-heap m...

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14617 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17540 **[Test build #75579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75579/testReport)** for PR 17540 at commit

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2017-04-06 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/14617 merged to master. Thanks @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17500: [SPARK-20181] [CORE] tries to bind the port to avoid jet...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17500 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17500: [SPARK-20181] [CORE] tries to bind the port to avoid jet...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75576/ Test PASSed. ---

[GitHub] spark issue #17500: [SPARK-20181] [CORE] tries to bind the port to avoid jet...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17500 **[Test build #75576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75576/testReport)** for PR 17500 at commit

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17541 **[Test build #75580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75580/testReport)** for PR 17541 at commit

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-04-06 Thread dgingrich
Github user dgingrich commented on the issue: https://github.com/apache/spark/pull/16845 Great, thanks Holden! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110223590 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -372,49 +410,19 @@ abstract class QueryPlan[PlanType

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110221774 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,426 @@ +/*

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17540 **[Test build #75579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75579/testReport)** for PR 17540 at commit

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17494 LGTM if others are ok too --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17552 **[Test build #75578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75578/testReport)** for PR 17552 at commit

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110218239 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/LogicalRelation.scala --- @@ -51,27 +51,11 @@ case class LogicalRelation(

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17541 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75577/ Test FAILed. ---

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17541 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16788: [SPARK-16742] Kerberos impersonation support

2017-04-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16788 Hello all, if you're not going to update this PR then it should be closed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17541 **[Test build #75577 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75577/testReport)** for PR 17541 at commit

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17552#discussion_r110217881 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/PathOptionSuite.scala --- @@ -75,13 +75,13 @@ class PathOptionSuite extends DataSourceTest

[GitHub] spark issue #17552: [SPARK-20245][SQL][minor] pass output to LogicalRelation...

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17552 cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #17552: [SPARK-20245][SQL][minor] pass output to LogicalR...

2017-04-06 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17552 [SPARK-20245][SQL][minor] pass output to LogicalRelation directly ## What changes were proposed in this pull request? Currently `LogicalRelation` has a `expectedOutputAttributes`

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110213152 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,426 @@ +/*

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110212976 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala --- @@ -0,0 +1,426 @@ +/*

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110212520 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -359,9 +359,47 @@ abstract class QueryPlan[PlanType

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread barnardb
Github user barnardb commented on the issue: https://github.com/apache/spark/pull/17551 Our use case involves jobs running in a remote cluster without a Spark master. I agree that the history server is the better to solve this, but we'd like to get a solution that doesn't depend on a

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110211908 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110211689 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110211394 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark pull request #17546: [SPARK-20233] [SQL] Apply star-join filter heuris...

2017-04-06 Thread ioana-delaney
Github user ioana-delaney commented on a diff in the pull request: https://github.com/apache/spark/pull/17546#discussion_r110210819 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/CostBasedJoinReorder.scala --- @@ -327,3 +345,104 @@ object

[GitHub] spark issue #17540: [SPARK-20213][SQL][UI] Fix DataFrameWriter operations in...

2017-04-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/17540 @cloud-fan, can you look at this? What do you think about the question above: when should `withNewExecutionId` get called? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110208947 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -372,49 +410,19 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110208623 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -372,49 +410,19 @@ abstract class QueryPlan[PlanType <:

[GitHub] spark pull request #17511: [SPARK-20195][SPARKR][SQL] add createTable catalo...

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17511 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17511: [SPARK-20195][SPARKR][SQL] add createTable catalog API a...

2017-04-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17511 merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog functi...

2017-04-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17512 merged to master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog...

2017-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17512 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17500: [SPARK-20181] [CORE] tries to bind the port to avoid jet...

2017-04-06 Thread d2r
Github user d2r commented on the issue: https://github.com/apache/spark/pull/17500 @srowen > Hm, why does this help? isn't it somehow trying to bind/close a port that's already in use? If the port is in use, then it will throw a BindException here, which will be

[GitHub] spark issue #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17541 **[Test build #75577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75577/testReport)** for PR 17541 at commit

[GitHub] spark pull request #17541: [SPARK-20229][SQL] add semanticHash to QueryPlan

2017-04-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17541#discussion_r110199279 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -372,49 +410,19 @@ abstract class QueryPlan[PlanType

[GitHub] spark pull request #17534: [SPARK-20218]'/applications/[app-id]/stages' in R...

2017-04-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17534#discussion_r110198657 --- Diff: docs/monitoring.md --- @@ -299,12 +299,12 @@ can be identified by their `[attempt-id]`. In the API listed below, when running

[GitHub] spark issue #17500: [SPARK-20181] [CORE] tries to bind the port to avoid jet...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17500 **[Test build #75576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75576/testReport)** for PR 17500 at commit

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/17551 I agree with @srowen and @jerryshao this is what the history server is for --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17500: [SPARK-20181] [CORE] tries to bind the port to avoid jet...

2017-04-06 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/17500 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method on Ind...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75575/ Test PASSed. ---

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method on Ind...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17459 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method on Ind...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17459 **[Test build #75575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75575/testReport)** for PR 17459 at commit

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17494 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75574/ Test PASSed. ---

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17494 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17494 **[Test build #75574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75574/testReport)** for PR 17494 at commit

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method on Ind...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17459 **[Test build #75575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75575/testReport)** for PR 17459 at commit

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method on Ind...

2017-04-06 Thread johnc1231
Github user johnc1231 commented on the issue: https://github.com/apache/spark/pull/17459 @viirya Added appropriate tests, back to you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17459: [SPARK-20109][MLlib] Rewrote toBlockMatrix method...

2017-04-06 Thread johnc1231
Github user johnc1231 commented on a diff in the pull request: https://github.com/apache/spark/pull/17459#discussion_r110163778 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrixSuite.scala --- @@ -89,11 +89,19 @@ class IndexedRowMatrixSuite

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/17551 Agree with @srowen , the proposed solution overlaps the key functionality of history server. Usually we should stop the app and release the resources as soon as application finished. This

[GitHub] spark issue #17494: [SPARK-20076][ML][PySpark] Add Python interface for ml.s...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17494 **[Test build #75574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75574/testReport)** for PR 17494 at commit

[GitHub] spark pull request #17494: [SPARK-20076][ML][PySpark] Add Python interface f...

2017-04-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17494#discussion_r110161378 --- Diff: python/pyspark/ml/stat.py --- @@ -71,6 +71,67 @@ def test(dataset, featuresCol, labelCol): return _java2py(sc,

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17222 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75573/ Test PASSed. ---

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17222 **[Test build #75573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75573/testReport)** for PR 17222 at commit

[GitHub] spark pull request #17494: [SPARK-20076][ML][PySpark] Add Python interface f...

2017-04-06 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/17494#discussion_r110156124 --- Diff: python/pyspark/ml/stat.py --- @@ -71,6 +71,67 @@ def test(dataset, featuresCol, labelCol): return _java2py(sc,

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17551 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17551 I would oppose this change. It is what the history server is for. Or also profiling tools, or simple debugging mechanisms you allude to. It doesn't belong as yet another flag in Spark --- If your

[GitHub] spark pull request #17551: [SPARK-20242][Web UI] Add spark.ui.stopDelay

2017-04-06 Thread barnardb
GitHub user barnardb opened a pull request: https://github.com/apache/spark/pull/17551 [SPARK-20242][Web UI] Add spark.ui.stopDelay ## What changes were proposed in this pull request? Adds a spark.ui.stopDelay configuration property that can be used to keep the UI running

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75572/ Test PASSed. ---

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17527 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17527: [SPARK-20156][CORE][SQL][STREAMING][MLLIB] Java String t...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17527 **[Test build #75572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75572/testReport)** for PR 17527 at commit

[GitHub] spark issue #17506: [SPARK-20189][DStream] Fix spark kinesis testcases to re...

2017-04-06 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/17506 Thanks @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17550: [SPARK-20240][SQL] SparkSQL support limitations of max d...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17550 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #17550: [SPARK-20240][SQL] SparkSQL support limitations o...

2017-04-06 Thread zenglinxi0615
GitHub user zenglinxi0615 opened a pull request: https://github.com/apache/spark/pull/17550 [SPARK-20240][SQL] SparkSQL support limitations of max dynamic partit… …ions when inserting hive table ## What changes were proposed in this pull request? support limitations

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17222 **[Test build #75573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75573/testReport)** for PR 17222 at commit

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-04-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16845 Let's make sure everything is still fine since we've made some changes in the testing infrastructure though - if it passes I'll merge this tonight (european time). Jenkins retest this

[GitHub] spark issue #16845: [SPARK-19505][Python] AttributeError on Exception.messag...

2017-04-06 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16845 LGTM - sorry for the slowness. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75571/ Test FAILed. ---

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17222 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17222 **[Test build #75571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75571/testReport)** for PR 17222 at commit

[GitHub] spark issue #17505: [SPARK-20187][SQL] Replace loadTable with moveFile to sp...

2017-04-06 Thread yuananf
Github user yuananf commented on the issue: https://github.com/apache/spark/pull/17505 @wangyum @rxin @gatorsmile Is it possible to upgrade the built-in hive-exec to resolve this problem? I believe the built-in hive-exec is this

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17222 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17222 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75570/ Test FAILed. ---

[GitHub] spark issue #17222: [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFuncti...

2017-04-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17222 **[Test build #75570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75570/testReport)** for PR 17222 at commit

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2017-04-06 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/16158 Sorry this slipped! I'd like to revisit soon after 2.2. settles down. I think we may need to consider how this integrates with training / evaluation summaries to create a holistic solution

[GitHub] spark pull request #17494: [SPARK-20076][ML][PySpark] Add Python interface f...

2017-04-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/17494#discussion_r110127639 --- Diff: python/pyspark/ml/stat.py --- @@ -71,6 +71,67 @@ def test(dataset, featuresCol, labelCol): return _java2py(sc,

[GitHub] spark issue #15726: [SPARK-18107][SQL][FOLLOW-UP] Insert overwrite statement...

2017-04-06 Thread yuananf
Github user yuananf commented on the issue: https://github.com/apache/spark/pull/15726 @viirya Is it possible to upgrade the built-in hive-exec to resolve this problem? We are facing the same problem, insert overwrite dynamic partition is extremely slow, the data written is over in 5

<    1   2   3   4   >