[GitHub] spark pull request #13929: [SPARK-16220][SQL] Add scope to show functions

2016-06-27 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/13929 [SPARK-16220][SQL] Add scope to show functions ## What changes were proposed in this pull request? Spark currently shows all functions when issue a `SHOW FUNCTIONS` command. This PR refines

[GitHub] spark pull request #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_ex...

2016-06-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13928 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13928 Merged into master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13928 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13928 **[Test build #61319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61319/consoleFull)** for PR 13928 at commit

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13928 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61319/ Test PASSed. ---

[GitHub] spark issue #13603: [SPARK-15865][CORE] Blacklist should not result in job h...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13603 **[Test build #61320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61320/consoleFull)** for PR 13603 at commit

[GitHub] spark pull request #13727: [SPARK-15982][SPARK-16009][SPARK-16007][SQL] Harm...

2016-06-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/13727#discussion_r68644592 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -135,7 +129,7 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13928 LGTM pending Jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13603: [SPARK-15865][CORE] Blacklist should not result in job h...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/13603 I'll finish looking at this later today after #13826 gets merged, so it's easier to see what's new here. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #13603: [SPARK-15865][CORE] Blacklist should not result i...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13603#discussion_r68642776 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -30,6 +30,7 @@ import org.apache.spark.annotation.DeveloperApi

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13926 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61312/ Test PASSed. ---

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13926 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13926 **[Test build #61312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61312/consoleFull)** for PR 13926 at commit

[GitHub] spark pull request #13921: [SPARK-16140][MLlib][SparkR][Docs] Group k-means ...

2016-06-27 Thread keypointt
Github user keypointt commented on a diff in the pull request: https://github.com/apache/spark/pull/13921#discussion_r68642313 --- Diff: R/pkg/R/mllib.R --- @@ -277,14 +277,32 @@ setMethod("summary", signature(object = "NaiveBayesModel"), #' @param k Number of centers #'

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-27 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r68642373 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/ConsumerStrategy.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed to the

[GitHub] spark pull request #13603: [SPARK-15865][CORE] Blacklist should not result i...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13603#discussion_r68642421 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -30,6 +30,7 @@ import org.apache.spark.annotation.DeveloperApi

[GitHub] spark pull request #13603: [SPARK-15865][CORE] Blacklist should not result i...

2016-06-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13603#discussion_r68642206 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -83,7 +83,7 @@ private[spark] class TaskSetManager( val

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13926 BTW, @rxin @yhuai @marmbrus I am still investigating the potential issues for `Hive As A Data Source`. Thanks! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #13826: [SPARK-16106][CORE] TaskSchedulerImpl should properly tr...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/13826 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #13603: [SPARK-15865][CORE] Blacklist should not result i...

2016-06-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13603#discussion_r68641914 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -30,6 +30,7 @@ import org.apache.spark.annotation.DeveloperApi

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13926 Thanks! @hvanhovell After going over the other functions that called `sparkSession.sessionState.catalog.createTable`, this is the only function that has such an issue. --- If your

[GitHub] spark issue #13027: [SPARK-4452][SPARK-11293][Core][BRANCH-1.6] Shuffle data...

2016-06-27 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/13027 @tgravescs For minor releases, we (the community) usually do not put much effort on the QA, it's risky to pull in large change (like this one) it. At least, I don't have enough confidence to merge

[GitHub] spark pull request #13378: [SPARK-15643] [Doc] [ML] Update spark.ml and spar...

2016-06-27 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/13378#discussion_r68640123 --- Diff: docs/mllib-guide.md --- @@ -121,6 +121,9 @@ Deprecations: We encourage users to use `spark.ml.regression.LinearRegresson` and

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13928 **[Test build #61319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61319/consoleFull)** for PR 13928 at commit

[GitHub] spark issue #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.p...

2016-06-27 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/13928 cc @mengxr @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13928: [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_ex...

2016-06-27 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/13928 [SPARK-16231][PYSPARK][ML][EXAMPLES] dataframe_example.py fails to convert ML style vectors ## What changes were proposed in this pull request? Need to convert ML Vectors to the old MLlib

[GitHub] spark pull request #13888: [SPARK-16187] [ML] Implement util method for ML M...

2016-06-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13888 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #13888: [SPARK-16187] [ML] Implement util method for ML Matrix c...

2016-06-27 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13888 Merged into master and branch-2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13378: [SPARK-15643] [Doc] [ML] Update spark.ml and spark.mllib...

2016-06-27 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/13378 Other than that, this looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13826: [SPARK-16106][CORE] TaskSchedulerImpl should properly tr...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13826 **[Test build #61318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61318/consoleFull)** for PR 13826 at commit

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68639726 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -298,10 +298,10 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13921: [SPARK-16140][MLlib][SparkR][Docs] Group k-means method ...

2016-06-27 Thread junyangq
Github user junyangq commented on the issue: https://github.com/apache/spark/pull/13921 It would be better to group write.ml (KM) into the doc as well. Also please include a full screenshot of the doc page. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13927 @junyangq You can use `dev/lint-r` to check R code style on your local machine. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #13927: [SPARK-16143][R] group AFT survival regression me...

2016-06-27 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/13927#discussion_r68638675 --- Diff: R/pkg/R/mllib.R --- @@ -420,44 +440,96 @@ setMethod("spark.naiveBayes", signature(data = "SparkDataFrame", formula = "form

[GitHub] spark issue #13826: [SPARK-16106][CORE] TaskSchedulerImpl should properly tr...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13826 **[Test build #61317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61317/consoleFull)** for PR 13826 at commit

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68638551 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -298,10 +298,10 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13927 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61315/ Test FAILed. ---

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13927 **[Test build #61315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61315/consoleFull)** for PR 13927 at commit

[GitHub] spark issue #13848: [SPARK-16136][CORE] Fix flaky TaskManagerSuite

2016-06-27 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/13848 Thanks for taking a look, I've updated the comment (just waiting for the sanity checks from jenkins) btw, if you're interested in following up on leaked threads, I opened this:

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13927 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13027: [SPARK-4452][SPARK-11293][Core][BRANCH-1.6] Shuffle data...

2016-06-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/13027 @davies Why haven't we pulled this into branch 1.6? can we upmerge and pull in so that if we do 1.6.3 its there? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #13927: [SPARK-16143][R] group AFT survival regression me...

2016-06-27 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/13927#discussion_r68638138 --- Diff: R/pkg/R/mllib.R --- @@ -420,44 +440,96 @@ setMethod("spark.naiveBayes", signature(data = "SparkDataFrame", formula = "form

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13927 **[Test build #61315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61315/consoleFull)** for PR 13927 at commit

[GitHub] spark issue #13848: [SPARK-16136][CORE] Fix flaky TaskManagerSuite

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13848 **[Test build #61316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61316/consoleFull)** for PR 13848 at commit

[GitHub] spark pull request #13927: [SPARK-16143][R] group AFT survival regression me...

2016-06-27 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/13927#discussion_r68637735 --- Diff: R/pkg/R/mllib.R --- @@ -420,44 +440,96 @@ setMethod("spark.naiveBayes", signature(data = "SparkDataFrame", formula = "form

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13927 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/13927 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68637198 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -293,13 +292,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13826: [SPARK-16106][CORE] TaskSchedulerImpl should properly tr...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13826 **[Test build #61314 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61314/consoleFull)** for PR 13826 at commit

[GitHub] spark issue #13927: [SPARK-16143][R] group AFT survival regression methods d...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13927 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13765 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61309/ Test PASSed. ---

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13765 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13765 **[Test build #61309 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61309/consoleFull)** for PR 13765 at commit

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68636634 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -293,13 +292,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #13927: [SPARK-16143][R] group AFT survival regression me...

2016-06-27 Thread junyangq
Github user junyangq commented on a diff in the pull request: https://github.com/apache/spark/pull/13927#discussion_r68636543 --- Diff: R/pkg/R/mllib.R --- @@ -233,9 +233,10 @@ setMethod("predict", signature(object = "GeneralizedLinearRegressionModel"), # Makes predictions

[GitHub] spark pull request #13927: [SPARK-16143][R] group AFT survival regression me...

2016-06-27 Thread junyangq
GitHub user junyangq opened a pull request: https://github.com/apache/spark/pull/13927 [SPARK-16143][R] group AFT survival regression methods docs in a single Rd ## What changes were proposed in this pull request? This PR groups `spark.survreg`, `summary(AFT)`,

[GitHub] spark issue #13848: [SPARK-16136][CORE] Fix flaky TaskManagerSuite

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/13848 LGTM -- thanks for fixing this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68635883 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -293,13 +292,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #13848: [SPARK-16136][CORE] Fix flaky TaskManagerSuite

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13848#discussion_r68635755 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -157,14 +157,27 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #13848: [SPARK-16136][CORE] Fix flaky TaskManagerSuite

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13848#discussion_r68635687 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -819,6 +833,8 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #13916: [SPARK-16220][SQL] Revert Change to Bring Back SH...

2016-06-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13916 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68634181 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -293,13 +292,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13916: [SPARK-16220][SQL] Revert Change to Bring Back SHOW FUNC...

2016-06-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13916 LGTM - merging to master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13107: [SPARK-13850] Force the sorter to Spill when number of e...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13107 **[Test build #3133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3133/consoleFull)** for PR 13107 at commit

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68632131 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -272,6 +278,45 @@ class TaskSchedulerImplSuite extends

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13494 **[Test build #61313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61313/consoleFull)** for PR 13494 at commit

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68631252 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -272,6 +278,45 @@ class TaskSchedulerImplSuite extends

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13926 Look pretty good. Are there any other places where this can happen? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68630849 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -293,13 +292,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStreams to ...

2016-06-27 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/11863 3. Okay, no strong feelings on this one. 4b. I dont think there is a need for Java tests for methods which need class tags. Those are probably least likely to be used. --- If your project

[GitHub] spark pull request #13826: [SPARK-16106][CORE] TaskSchedulerImpl should prop...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13826#discussion_r68630201 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -293,13 +292,16 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #13925: [SPARK-16226][SQL]change the way of JDBC commit

2016-06-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13925 @lihongliustc could you take a look at the JIRA ticket, and respond to @srowen's very valid comment? I don't think we should do this. --- If your project is set up for it, you can reply to

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61308/ Test PASSed. ---

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE ...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13926 **[Test build #61312 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61312/consoleFull)** for PR 13926 at commit

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13494 **[Test build #61308 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61308/consoleFull)** for PR 13494 at commit

[GitHub] spark pull request #13926: [SPARK-16229] [SQL] Drop Empty Table After CREATE...

2016-06-27 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/13926 [SPARK-16229] [SQL] Drop Empty Table After CREATE TABLE AS SELECT fails What changes were proposed in this pull request? In `CREATE TABLE AS SELECT`, if the `SELECT` query failed, the

[GitHub] spark pull request #11863: [SPARK-12177][Streaming][Kafka] Update KafkaDStre...

2016-06-27 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/11863#discussion_r68629750 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka/ConsumerStrategy.scala --- @@ -0,0 +1,164 @@ +/* + * Licensed to the

[GitHub] spark pull request #13603: [SPARK-15865][CORE] Blacklist should not result i...

2016-06-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/13603#discussion_r68629355 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -30,6 +30,7 @@ import org.apache.spark.annotation.DeveloperApi

[GitHub] spark issue #13720: [SPARK-16004] [SQL] Correctly display "Last Access Time"...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13720 **[Test build #61311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61311/consoleFull)** for PR 13720 at commit

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61307/ Test PASSed. ---

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13494 **[Test build #61307 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61307/consoleFull)** for PR 13494 at commit

[GitHub] spark issue #13816: [SPARK-16114] [SQL] structured streaming network word co...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61310/ Test PASSed. ---

[GitHub] spark issue #13816: [SPARK-16114] [SQL] structured streaming network word co...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13816 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13816: [SPARK-16114] [SQL] structured streaming network word co...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13816 **[Test build #61310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61310/consoleFull)** for PR 13816 at commit

[GitHub] spark issue #13730: [SPARK-16006][SQL] Attemping to write empty DataFrame wi...

2016-06-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13730 Ping @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #13918: [SPARK-16221][SQL] Redirect Parquet JUL logger via SLF4J...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13918 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13918: [SPARK-16221][SQL] Redirect Parquet JUL logger via SLF4J...

2016-06-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13918 Hi, @liancheng . Now, it passes the Jenkins again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #13918: [SPARK-16221][SQL] Redirect Parquet JUL logger via SLF4J...

2016-06-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13918 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61306/ Test PASSed. ---

[GitHub] spark issue #13848: [SPARK-16136][CORE] Fix flaky TaskManagerSuite

2016-06-27 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/13848 @devaraj-kavali @kayousterhout https://github.com/apache/spark/pull/11996 added a flaky test, this fix should be pretty straightforward. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #13918: [SPARK-16221][SQL] Redirect Parquet JUL logger via SLF4J...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13918 **[Test build #61306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61306/consoleFull)** for PR 13918 at commit

[GitHub] spark pull request #13727: [SPARK-15982][SPARK-16009][SPARK-16007][SQL] Harm...

2016-06-27 Thread koertkuipers
Github user koertkuipers commented on a diff in the pull request: https://github.com/apache/spark/pull/13727#discussion_r68624316 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -135,7 +129,7 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #13915: [SPARK-16081][BUILD] Disallow using `l` as variable name

2016-06-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13915 Hi, @rxin and @srowen . I also worry about merge conflicts. It's really annoying for committers. So, what about the stepwise approach? We have 2395 files and found 90 files

[GitHub] spark issue #13816: [SPARK-16114] [SQL] structured streaming network word co...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13816 **[Test build #61310 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61310/consoleFull)** for PR 13816 at commit

[GitHub] spark issue #13765: [SPARK-16052][SQL] Improve `CollapseRepartition` optimiz...

2016-06-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/13765 Hi, @cloud-fan . Now, this PR can handle all combinations of Repartition and RepartitionBy. I updated the description of PR and JIRA, too. Thank you so much for making this PR much

[GitHub] spark pull request #13107: [SPARK-13850] Force the sorter to Spill when numb...

2016-06-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13107#discussion_r68620470 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -72,7 +72,10 @@ private final TaskContext taskContext;

[GitHub] spark issue #13765: [SPARK-16052][SQL] Add `CollapseRepartitionBy` optimizer

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13765 **[Test build #61309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61309/consoleFull)** for PR 13765 at commit

[GitHub] spark issue #13107: [SPARK-13850] Force the sorter to Spill when number of e...

2016-06-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13107 **[Test build #3133 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3133/consoleFull)** for PR 13107 at commit

<    1   2   3   4   5   6   7   >