[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/17521 @gatorsmile @cloud-fan @ueshin Sorry .. i was on transit from work. Sure, i will make a try. However , i wanted to understand this a bit more. In my understanding, the current problem we are

[GitHub] spark issue #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog functi...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17512 **[Test build #75534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75534/testReport)** for PR 17512 at commit

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17532 I don't think is directly related, but, I tried to make a similar change to the Spark Scala implementation of this type of conversion and couldn't make it work, just because it changed a bunch of

[GitHub] spark pull request #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog...

2017-04-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17512#discussion_r109833836 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -456,7 +460,8 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark issue #17258: [SPARK-19807][Web UI]Add reason for cancellation when a ...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17258 **[Test build #3638 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3638/testReport)** for PR 17258 at commit

[GitHub] spark issue #17535: [SPARK-20222][SQL] Bring back the Spark SQL UI when exec...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17535 **[Test build #75533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75533/testReport)** for PR 17535 at commit

[GitHub] spark pull request #17526: [SPARKR][DOC] update doc for fpgrowth

2017-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17526 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17535: [SPARK-20222][SQL] Bring back the Spark SQL UI wh...

2017-04-04 Thread carsonwang
GitHub user carsonwang opened a pull request: https://github.com/apache/spark/pull/17535 [SPARK-20222][SQL] Bring back the Spark SQL UI when executing queries in Spark SQL CLI ## What changes were proposed in this pull request? There is no Spark SQL UI when executing queries in

[GitHub] spark issue #15899: [SPARK-18466] added withFilter method to RDD

2017-04-04 Thread danielyli
Github user danielyli commented on the issue: https://github.com/apache/spark/pull/15899 Hey, Checking in again on this PR. Can we please support `withFilter` for pair RDDs? For-expressions are a central sugar in Scala syntax, and without them developers are hampered in

[GitHub] spark issue #17534: [SPARK-20218]'/applications/[app-id]/stages' in REST API...

2017-04-04 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/17534 Added one more 2.'/applications/[app-id]/stages/[stage-id]' in REST API,remove redundant description ‘?status=[active|complete|pending|failed] list only stages in the state.’.

[GitHub] spark pull request #17511: [SPARK-20195][SPARKR][SQL] add createTable catalo...

2017-04-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17511#discussion_r109833091 --- Diff: R/pkg/R/DataFrame.R --- @@ -557,7 +557,7 @@ setMethod("insertInto", jmode <- convertToJSaveMode(ifelse(overwrite,

[GitHub] spark issue #17534: [SPARK-20218]'/applications/[app-id]/stages' in REST API...

2017-04-04 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/17534 Because I just use this API today to develop, only to find the problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75527/ Test PASSed. ---

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17525 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17525 **[Test build #75527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75527/testReport)** for PR 17525 at commit

[GitHub] spark pull request #14617: [SPARK-17019][Core] Expose on-heap and off-heap m...

2017-04-04 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14617#discussion_r109830934 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -111,7 +115,11 @@ class RDDDataDistribution private[spark]( val

[GitHub] spark pull request #14617: [SPARK-17019][Core] Expose on-heap and off-heap m...

2017-04-04 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14617#discussion_r109832240 --- Diff: core/src/main/scala/org/apache/spark/storage/StorageUtils.scala --- @@ -176,26 +178,51 @@ class StorageStatus(val blockManagerId: BlockManagerId,

[GitHub] spark pull request #14617: [SPARK-17019][Core] Expose on-heap and off-heap m...

2017-04-04 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14617#discussion_r109830395 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala --- @@ -115,8 +115,9 @@ private[spark] object ExecutorsPage { val

[GitHub] spark pull request #14617: [SPARK-17019][Core] Expose on-heap and off-heap m...

2017-04-04 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14617#discussion_r109830705 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala --- @@ -81,6 +115,11 @@ private[spark] object ExecutorsPage { val

[GitHub] spark pull request #14617: [SPARK-17019][Core] Expose on-heap and off-heap m...

2017-04-04 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/14617#discussion_r109831142 --- Diff: core/src/main/scala/org/apache/spark/storage/StorageUtils.scala --- @@ -35,7 +35,13 @@ import org.apache.spark.internal.Logging * class

[GitHub] spark issue #17534: [SPARK-20218]'/applications/[app-id]/stages' in REST API...

2017-04-04 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17534 Why didn't you add this to your last PR? What other related changes do you want to make? Making several small closely related PRs is frowned on --- If your project is set up for it, you can reply

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17521 @dilipbiswal Can you please try it based on what @cloud-fan and @ueshin suggested? Does it resolve the issue you report? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #17149: [SPARK-19257][SQL]location for table/partition/da...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17149#discussion_r109827813 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -386,7 +386,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

2017-04-04 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17471 cc @cloud-fan / @ueshin / @sameeragarwal can you review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #17149: [SPARK-19257][SQL]location for table/partition/da...

2017-04-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17149#discussion_r109827145 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -386,7 +386,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #17149: [SPARK-19257][SQL]location for table/partition/da...

2017-04-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17149#discussion_r109827042 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -386,7 +386,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #17149: [SPARK-19257][SQL]location for table/partition/da...

2017-04-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17149#discussion_r109827045 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -386,7 +386,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #17149: [SPARK-19257][SQL]location for table/partition/da...

2017-04-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17149#discussion_r109826890 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -386,7 +386,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark pull request #17149: [SPARK-19257][SQL]location for table/partition/da...

2017-04-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17149#discussion_r109826847 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -386,7 +386,7 @@ class SparkSqlAstBuilder(conf: SQLConf)

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17532 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17532 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75530/ Test PASSed. ---

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17532 **[Test build #75530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75530/testReport)** for PR 17532 at commit

[GitHub] spark issue #17533: [WIP][SPARK-20219] Schedule tasks based on size of input...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17533 **[Test build #75532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75532/testReport)** for PR 17533 at commit

[GitHub] spark issue #17534: [SPARK-20218]'/applications/[app-id]/stages' in REST API...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17534 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17532 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75528/ Test PASSed. ---

[GitHub] spark pull request #17534: [SPARK-20218]'/applications/[app-id]/stages' in R...

2017-04-04 Thread guoxiaolongzte
GitHub user guoxiaolongzte opened a pull request: https://github.com/apache/spark/pull/17534 [SPARK-20218]'/applications/[app-id]/stages' in REST API,add description. ## What changes were proposed in this pull request? '/applications/[app-id]/stages' in rest api.status

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17532 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17532 **[Test build #75528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75528/testReport)** for PR 17532 at commit

[GitHub] spark issue #17533: [WIP][SPARK-20219] Schedule tasks based on size of input...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17533 **[Test build #75531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75531/testReport)** for PR 17533 at commit

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17521 I agree with @cloud-fan's suggestion, `ConfigEntryWithDefaultFunction` approach. Hopefully the default value should be fixed when `SQLConf` instance is created with the timezone in the JVM at that

[GitHub] spark issue #17533: [SPARK-20219] Schedule tasks based on size of input from...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17533 **[Test build #75529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75529/testReport)** for PR 17533 at commit

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17532 **[Test build #75530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75530/testReport)** for PR 17532 at commit

[GitHub] spark pull request #17533: [SPARK-20219] Schedule tasks based on size of inp...

2017-04-04 Thread jinxing64
GitHub user jinxing64 opened a pull request: https://github.com/apache/spark/pull/17533 [SPARK-20219] Schedule tasks based on size of input from ScheduledRDD ## What changes were proposed in this pull request? When data is highly skewed on `ShuffledRDD`, it make sense to

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17532 cc @jkbradley @MLnick @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17532: [SPARK-20214][ML] Make sure converted csc matrix has sor...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17532 **[Test build #75528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75528/testReport)** for PR 17532 at commit

[GitHub] spark pull request #17532: [SPARK-20214][ML] Make sure converted csc matrix ...

2017-04-04 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/17532 [SPARK-20214][ML] Make sure converted csc matrix has sorted indices ## What changes were proposed in this pull request? `_convert_to_vector` converts a scipy sparse matrix to csc matrix for

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17520 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75526/ Test PASSed. ---

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17525 **[Test build #75527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75527/testReport)** for PR 17525 at commit

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17520 **[Test build #75526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75526/testReport)** for PR 17520 at commit

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/17525 Thanks for the change. It is easier to understand when things are being triggered now. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17521 These test cases are timezone sensitive, right? If so, the changes made by @dilipbiswal are reasonable to me. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17521 also cc @ueshin , I think the default value of `SESSION_LOCAL_TIMEZONE` should always be the current timezone in the JVM. We may need something like `ConfigEntryWithDefaultFunction`, so that the

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/17525 left one minor comment, otherwise LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17525: [SPARK-20209][SS] Execute next trigger immediatel...

2017-04-04 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/17525#discussion_r109821142 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ProcessingTimeExecutorSuite.scala --- @@ -35,6 +42,56 @@ class

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17521 `SESSION_LOCAL_TIMEZONE` sounds a static SQLConf. Do we allow users to change it at runtime? It sounds like our codes do not allow users to change/refresh it. --- If your project is set up

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/17521 @cloud-fan @nsyca A quick update.. I ran the problematic tests and they pass with a change to move the time zone setting code to PlanTest.scala just before we create the SQLConf like following

[GitHub] spark issue #17531: [SPARK-20217][core] Executor should not fail stage if ki...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17531 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17531: [SPARK-20217][core] Executor should not fail stage if ki...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17531 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75525/ Test FAILed. ---

[GitHub] spark issue #17531: [SPARK-20217][core] Executor should not fail stage if ki...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17531 **[Test build #75525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75525/testReport)** for PR 17531 at commit

[GitHub] spark issue #16165: [SPARK-8617] [WEBUI] HistoryServer: Include in-progress ...

2017-04-04 Thread merlintang
Github user merlintang commented on the issue: https://github.com/apache/spark/pull/16165 @vanzin sorry, I mean the 2.1.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75524/ Test PASSed. ---

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17525 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17525 **[Test build #75524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75524/testReport)** for PR 17525 at commit

[GitHub] spark issue #17459: [SPARK-20109][MLlib] Added toBlockMatrixDense to Indexed...

2017-04-04 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17459 Please modify the PR title also. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17459: [SPARK-20109][MLlib] Added toBlockMatrixDense to ...

2017-04-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17459#discussion_r109817633 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrix.scala --- @@ -108,8 +108,64 @@ class IndexedRowMatrix

[GitHub] spark pull request #17459: [SPARK-20109][MLlib] Added toBlockMatrixDense to ...

2017-04-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/17459#discussion_r109817581 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrixSuite.scala --- @@ -89,11 +89,19 @@ class IndexedRowMatrixSuite

[GitHub] spark pull request #17511: [SPARK-20195][SPARKR][SQL] add createTable catalo...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17511#discussion_r109816897 --- Diff: R/pkg/R/DataFrame.R --- @@ -557,7 +557,7 @@ setMethod("insertInto", jmode <- convertToJSaveMode(ifelse(overwrite, "overwrite",

[GitHub] spark pull request #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17512#discussion_r109815601 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -367,6 +367,7 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark issue #17251: [SPARK-19910][SQL] `stack` should not reject NULL values...

2017-04-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17251 Hi, @cloud-fan and @gatorsmile . If there is something to do more, please let me know. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

2017-04-04 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 cc - @rxin, @kayousterhout, @squito --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16165: [SPARK-8617] [WEBUI] HistoryServer: Include in-progress ...

2017-04-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/16165 2.1 has already shipped and this is not a regression. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/17521 @cloud-fan @nsyca Changing to make it lazy works for the test cases i have tried. I am running the full tests now. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/17521 @dilipbiswal has narrowed down that this PR is changing the behaviour. He will continue to investigate and will post an update in the next hour or so before he calls it a day. --- If your project

[GitHub] spark issue #17521: [SPARK-20204][SQL] remove SimpleCatalystConf and Catalys...

2017-04-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17521 @nsyca can you try marking `SQLConf.SESSION_LOCAL_TIMEZONE` as `lazy val`? I think the issue is that, once `object SQLConf` is instantiated, the default value for `SESSION_LOCAL_TIMEZONE` is

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17520 **[Test build #75526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75526/testReport)** for PR 17520 at commit

[GitHub] spark pull request #17336: [SPARK-20003] [ML] FPGrowthModel setMinConfidence...

2017-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17336 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17524: [SPARK-19235] [SQL] [TEST] [FOLLOW-UP] Enable Test Cases...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17524 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75523/ Test FAILed. ---

[GitHub] spark issue #17524: [SPARK-19235] [SQL] [TEST] [FOLLOW-UP] Enable Test Cases...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17524 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17524: [SPARK-19235] [SQL] [TEST] [FOLLOW-UP] Enable Test Cases...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17524 **[Test build #75523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75523/testReport)** for PR 17524 at commit

[GitHub] spark issue #17336: [SPARK-20003] [ML] FPGrowthModel setMinConfidence should...

2017-04-04 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17336 Thanks a lot for the second update! This LGTM Merging with master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #17336: [SPARK-20003] [ML] FPGrowthModel setMinConfidence...

2017-04-04 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17336#discussion_r109810945 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -85,38 +85,58 @@ class FPGrowthSuite extends SparkFunSuite with

[GitHub] spark issue #17531: [SPARK-20217][core] Executor should not fail stage if ki...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17531 **[Test build #75525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75525/testReport)** for PR 17531 at commit

[GitHub] spark pull request #17531: [SPARK-20217][core] Executor should not fail stag...

2017-04-04 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/17531 [SPARK-20217][core] Executor should not fail stage if killed task throws non-interrupted exception ## What changes were proposed in this pull request? If tasks throw non-interrupted

[GitHub] spark pull request #17501: [SPARK-20183][ML] Added outlierRatio arg to MLTes...

2017-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17501: [SPARK-20183][ML] Added outlierRatio arg to MLTestingUti...

2017-04-04 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17501 Thanks! As you can tell, interrupts happen for me too : P Merging with master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #17525: [SPARK-20209][SS] Execute next trigger immediately if pr...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17525 **[Test build #75524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75524/testReport)** for PR 17525 at commit

[GitHub] spark pull request #17525: [SPARK-20209][SS] Execute next trigger immediatel...

2017-04-04 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17525#discussion_r109805354 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/TriggerExecutor.scala --- @@ -83,6 +84,6 @@ case class

[GitHub] spark pull request #17525: [SPARK-20209][SS] Execute next trigger immediatel...

2017-04-04 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/17525#discussion_r109805306 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala --- @@ -207,80 +207,91 @@ class StreamingQuerySuite extends

[GitHub] spark pull request #17398: [SPARK-19716][SQL] support by-name resolution for...

2017-04-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17398 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17512#discussion_r109803579 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -456,7 +460,8 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark issue #17398: [SPARK-19716][SQL] support by-name resolution for struct...

2017-04-04 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/17398 LGTM. Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75522/ Test PASSed. ---

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17520 **[Test build #75522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75522/testReport)** for PR 17520 at commit

[GitHub] spark issue #17520: [WIP][SPARK-19712][SQL] Move PullupCorrelatedPredicates ...

2017-04-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17520 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17524: [SPARK-19235] [SQL] [TEST] [FOLLOW-UP] Enable Test Cases...

2017-04-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17524 **[Test build #75523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75523/testReport)** for PR 17524 at commit

[GitHub] spark issue #17524: [SPARK-19235] [SQL] [TEST] [FOLLOW-UP] Enable Test Cases...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17524 retest this pleaes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17512#discussion_r109798818 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala --- @@ -447,6 +447,7 @@ abstract class Catalog { /** *

[GitHub] spark pull request #17512: [SPARK-20196][PYTHON][SQL] update doc for catalog...

2017-04-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17512#discussion_r109798528 --- Diff: python/pyspark/sql/catalog.py --- @@ -276,14 +277,24 @@ def clearCache(self): @since(2.0) def refreshTable(self,

[GitHub] spark issue #16165: [SPARK-8617] [WEBUI] HistoryServer: Include in-progress ...

2017-04-04 Thread merlintang
Github user merlintang commented on the issue: https://github.com/apache/spark/pull/16165 should we backport this into 2.1? @vanzin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

  1   2   3   >