[GitHub] spark issue #13459: [SPARK-15726] [SQL] Make DatasetBenchmark fairer among D...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13459 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #14473: [SPARK-16495] [MLlib]Add ADMM optimizer in mllib package

2016-08-04 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14473 @ZunwenYou Would you mind addressing the comments in the JIRA first? Adding a new optimization algorithm to an API that is now deprecated definitely warrants more high level discussion before code

[GitHub] spark issue #13459: [SPARK-15726] [SQL] Make DatasetBenchmark fairer among D...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13459 **[Test build #63253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63253/consoleFull)** for PR 13459 at commit

[GitHub] spark issue #14503: [#SPARK-16911] Fix the links in the programming guide

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14503 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-04 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13701 @davies I improved the test case by adding a SQL metrics. Please take a look. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #12753: [SPARK-3767] [CORE] Support wildcard in Spark properties

2016-08-04 Thread devaraj-kavali
Github user devaraj-kavali commented on the issue: https://github.com/apache/spark/pull/12753 @vanzin Thanks for looking into this, I have resolved the conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14500 **[Test build #63242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63242/consoleFull)** for PR 14500 at commit

[GitHub] spark issue #14232: [SPARK-16587] [CORE] The annotation for the abstract Cla...

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14232 @f7753 I think just listing the message types here is not super useful. Yes, new messages were added and not mentioned in the documentation, but then adding just "foo is used by bar" doesn't really

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14500 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63242/ Test FAILed. ---

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14500 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14500 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63243/ Test FAILed. ---

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73623489 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -229,10 +230,8 @@ case class AlterTableSetPropertiesCommand(

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73623449 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -229,10 +230,8 @@ case class AlterTableSetPropertiesCommand(

[GitHub] spark pull request #9571: [SPARK-11373] [CORE] Add metrics to the History Se...

2016-08-04 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9571#discussion_r73623442 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -667,6 +710,123 @@ private[history] class FsHistoryProvider(conf:

[GitHub] spark pull request #9571: [SPARK-11373] [CORE] Add metrics to the History Se...

2016-08-04 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9571#discussion_r73624122 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -667,6 +700,90 @@ private[history] class FsHistoryProvider(conf:

[GitHub] spark issue #14497: [SPARK-16901] Hive settings in hive-site.xml may be over...

2016-08-04 Thread clockfly
Github user clockfly commented on the issue: https://github.com/apache/spark/pull/14497 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14501: [SPARK-16906][SQL] Adds auxiliary info like input class ...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14501 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14501: [SPARK-16906][SQL] Adds auxiliary info like input class ...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14501 **[Test build #63249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63249/consoleFull)** for PR 14501 at commit

[GitHub] spark issue #14501: [SPARK-16906][SQL] Adds auxiliary info like input class ...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14501 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63249/ Test PASSed. ---

[GitHub] spark pull request #14450: [SPARK-16847][SQL] Prevent to potentially read co...

2016-08-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14450#discussion_r73632453 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java --- @@ -204,7 +205,8 @@

[GitHub] spark issue #14491: [SPARK-16886] [EXAMPLES][SQL] structured streaming netwo...

2016-08-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14491 Oh, this is only about streaming examples. I can look at those in a separate PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #12800: [SPARK-15024] NoClassDefFoundError in spark-examples due...

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/12800 @atokhy that's a little weird. The examples module should be inheriting the guava shading from the parent pom. I have a 1.6 build locally (with patches, so I'll check the official package tomorrow)

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r73638020 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -25,30 +25,36 @@ import

[GitHub] spark issue #14502: [SPARK-16909][Spark Core] - Streaming for postgreSQL JDB...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14502 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #13701: [SPARK-15639][SPARK-16321][SQL] Push down filter at RowG...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13701 **[Test build #63254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63254/consoleFull)** for PR 13701 at commit

[GitHub] spark pull request #14503: [#SPARK-16911] Fix the links in the programming g...

2016-08-04 Thread shiv4nsh
GitHub user shiv4nsh opened a pull request: https://github.com/apache/spark/pull/14503 [#SPARK-16911] Fix the links in the programming guide ## What changes were proposed in this pull request? Fix the broken links in the programming guide of the Graphx Migration and

[GitHub] spark pull request #9571: [SPARK-11373] [CORE] Add metrics to the History Se...

2016-08-04 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9571#discussion_r73624337 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -667,6 +710,123 @@ private[history] class FsHistoryProvider(conf:

[GitHub] spark pull request #9571: [SPARK-11373] [CORE] Add metrics to the History Se...

2016-08-04 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9571#discussion_r73624345 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -667,6 +710,123 @@ private[history] class FsHistoryProvider(conf:

[GitHub] spark issue #14499: [HOTFIX] Remove unnecessary imports from #12944 that bro...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14499 **[Test build #63241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63241/consoleFull)** for PR 14499 at commit

[GitHub] spark issue #14499: [HOTFIX] Remove unnecessary imports from #12944 that bro...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14499 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14499: [HOTFIX] Remove unnecessary imports from #12944 that bro...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63241/ Test PASSed. ---

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63248/ Test PASSed. ---

[GitHub] spark issue #14445: [SPARK-16907] [SQL] Fix performance regression for parqu...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14445 thanks. merging to master and 2.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #63252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63252/consoleFull)** for PR 12819 at commit

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #63252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63252/consoleFull)** for PR 12819 at commit

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73622711 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/HiveSerDe.scala --- @@ -42,8 +41,7 @@ object HiveSerDe { HiveSerDe(

[GitHub] spark issue #9571: [SPARK-11373] [CORE] Add metrics to the History Server an...

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/9571 A few minor things left. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14500 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14500 **[Test build #63246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63246/consoleFull)** for PR 14500 at commit

[GitHub] spark issue #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14500 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63246/ Test FAILed. ---

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #63251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63251/consoleFull)** for PR 12819 at commit

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73633166 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -275,238 +269,21 @@ case class

[GitHub] spark pull request #13428: [SPARK-12666][CORE] SparkSubmit packages fix for ...

2016-08-04 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13428#discussion_r73636023 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -891,10 +891,12 @@ private[spark] object SparkSubmitUtils { val

[GitHub] spark pull request #13428: [SPARK-12666][CORE] SparkSubmit packages fix for ...

2016-08-04 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13428#discussion_r73636009 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -891,10 +891,12 @@ private[spark] object SparkSubmitUtils { val

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63252/ Test PASSed. ---

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73644259 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala --- @@ -275,238 +269,21 @@ case class

[GitHub] spark issue #14155: [SPARK-16498][SQL] move hive hack for data source table ...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14155 **[Test build #63257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63257/consoleFull)** for PR 14155 at commit

[GitHub] spark pull request #14483: [SPARK-16880][ML][MLLib] make ann training data p...

2016-08-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14483 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14498: [SPARK-16904] [SQL] Removal of Hive Built-in Hash Functi...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14498 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63240/ Test FAILed. ---

[GitHub] spark issue #14498: [SPARK-16904] [SQL] Removal of Hive Built-in Hash Functi...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14498 **[Test build #63240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63240/consoleFull)** for PR 14498 at commit

[GitHub] spark issue #14498: [SPARK-16904] [SQL] Removal of Hive Built-in Hash Functi...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14498 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14497: [SPARK-16901] Hive settings in hive-site.xml may be over...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14497 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14497: [SPARK-16901] Hive settings in hive-site.xml may be over...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14497 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63238/ Test PASSed. ---

[GitHub] spark issue #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor port rest...

2016-08-04 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/11157 How about the rest of the ports, what would happen to them in that case? We will have only one executor per job right? Right now I get conflicts without the isolator in my local tests. In

[GitHub] spark pull request #14500: [SPARK-] SQL DDL: MSCK REPAIR TABLE

2016-08-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14500#discussion_r73619320 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -389,6 +389,50 @@ case class TruncateTableCommand( }

[GitHub] spark pull request #14109: [SPARK-16404][ML] LeastSquaresAggregators seriali...

2016-08-04 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14109#discussion_r73643402 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -862,27 +873,30 @@ class LinearRegressionSummary

[GitHub] spark issue #14482: [SPARK-16879][SQL] unify logical plans for CREATE TABLE ...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14482 @gatorsmile how about keep working on them after this PR is merged? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14482: [SPARK-16879][SQL] unify logical plans for CREATE TABLE ...

2016-08-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14482 @cloud-fan Sure, will do. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73644536 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -374,20 +373,7 @@ private[hive] class HiveClientImpl(

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73644517 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -70,64 +69,16 @@ private[hive] class

[GitHub] spark pull request #14155: [SPARK-16498][SQL] move hive hack for data source...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14155#discussion_r73644322 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -506,76 +495,16 @@ object DDLUtils { } /**

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73643735 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -420,45 +420,40 @@ private[sql] abstract class

[GitHub] spark issue #14109: [SPARK-16404][ML] LeastSquaresAggregators serializes unn...

2016-08-04 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/14109 @sethah In my opinion, I think using `@transient lazy val` is okay since there are only two places dereferencing the `lazy val`, and we don't use it in the tight loop. LGTM except one small

[GitHub] spark issue #14482: [SPARK-16879][SQL] unify logical plans for CREATE TABLE ...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14482 **[Test build #63256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63256/consoleFull)** for PR 14482 at commit

[GitHub] spark pull request #14109: [SPARK-16404][ML] LeastSquaresAggregators seriali...

2016-08-04 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/14109#discussion_r73644694 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -862,27 +873,30 @@ class LinearRegressionSummary

[GitHub] spark pull request #14493: [#SPARK-16897] Refactored the code and changed th...

2016-08-04 Thread shiv4nsh
Github user shiv4nsh closed the pull request at: https://github.com/apache/spark/pull/14493 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14495: MAINTENANCE. Cleaning up stale PRs.

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14495 Adding #13248 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #14487: [SPARK-16884] Move DataSourceScanExec out of ExistingRDD...

2016-08-04 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14487 LGTM, merging into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13506: [SPARK-15763][SQL] Support DELETE FILE command natively

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13506 @kevinyu98 Could you update the PR and fix merge conflicts? Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14113: [SPARK-16457] [SQL] Fix Wrong Messages when CTAS with a ...

2016-08-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14113 This is not contained in https://github.com/apache/spark/pull/14482. Should I leave it open? or you will merge it into the PR? --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #14113: [SPARK-16457] [SQL] Fix Wrong Messages when CTAS with a ...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14113 **[Test build #63237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63237/consoleFull)** for PR 14113 at commit

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13950 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13680: [SPARK-15962][SQL] Introduce implementation with ...

2016-08-04 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13680#discussion_r73565692 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala --- @@ -18,27 +18,131 @@ package

[GitHub] spark issue #14308: [SPARK-16421][EXAMPLES][ML] Improve ML Example Outputs

2016-08-04 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/14308 This has been updated since fixing the errors in Java @srowen @MLnick . I know most of these changes are trivial, but will hopefully make some of the examples easier to follow. Thanks! ---

[GitHub] spark issue #12753: [SPARK-3767] [CORE] Support wildcard in Spark properties

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/12753 Sorry this has been forgotten for so long. I think instead of adding a new way of referencing variables, it might be better to use the new code I'm adding in #14468; that way it would be easier to

[GitHub] spark issue #14113: [SPARK-16457] [SQL] Fix Wrong Messages when CTAS with a ...

2016-08-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14113 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #12913: [SPARK-928][CORE] Add support for Unsafe-based serialize...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12913 **[Test build #63228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63228/consoleFull)** for PR 12913 at commit

[GitHub] spark issue #12913: [SPARK-928][CORE] Add support for Unsafe-based serialize...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12913 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12913: [SPARK-928][CORE] Add support for Unsafe-based serialize...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12913 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63228/ Test PASSed. ---

[GitHub] spark issue #14258: [Spark-16579][SparkR] add install.spark function

2016-08-04 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14258 I see - So I was thinking that we could merge this into master as well as its not going to fail any tests or affect any users building SparkR from source -- I dont think we make any promises about

[GitHub] spark pull request #14464: [SPARK-16802] [SQL] fix overflow in LongToUnsafeR...

2016-08-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14464 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14239: [SPARK-16593] [CORE] Provide a pre-fetch mechanism to ac...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14239 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14428: [SPARK-16810] Refactor registerSinks with multiple const...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14428 **[Test build #63236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63236/consoleFull)** for PR 14428 at commit

[GitHub] spark issue #14495: MAINTENANCE. Cleaning up stale PRs.

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63230/ Test PASSed. ---

[GitHub] spark issue #13907: [SPARK-16209] [SQL] Convert Hive Tables in PARQUET/ORC t...

2016-08-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13907 This is not contained in https://github.com/apache/spark/pull/14482. Should I leave it open? Or should I fix the conflict after https://github.com/apache/spark/pull/14482 is merged? --- If

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13950 **[Test build #63232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63232/consoleFull)** for PR 13950 at commit

[GitHub] spark issue #11157: [SPARK-11714][Mesos] Make Spark on Mesos honor port rest...

2016-08-04 Thread mgummelt
Github user mgummelt commented on the issue: https://github.com/apache/spark/pull/11157 So I hate to ask for another significant change to this PR, but it strikes me that the source of most of the complexity here is partitioning an offer around a port so we can reuse the rest of the

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-08-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13950 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63232/ Test PASSed. ---

[GitHub] spark pull request #14496: [SPARK-16772] [Python] [Docs] Fix API doc referen...

2016-08-04 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/14496 [SPARK-16772] [Python] [Docs] Fix API doc references to UDFRegistration + Update "important classes" ## Proposed Changes * Update the list of "important classes" in `pyspark.sql` to

[GitHub] spark pull request #14482: [SPARK-16879][SQL] unify logical plans for CREATE...

2016-08-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14482#discussion_r73568017 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -420,45 +420,40 @@ private[sql] abstract class

[GitHub] spark issue #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse proxy ...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13950 **[Test build #63232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63232/consoleFull)** for PR 13950 at commit

[GitHub] spark issue #11228: [SPARK-13356][Streaming]WebUI missing input informations...

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/11228 @jeanlyn do you mind updating the PR to solve conflicts? thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14496: [SPARK-16772] [Python] [Docs] Fix API doc references to ...

2016-08-04 Thread nchammas
Github user nchammas commented on the issue: https://github.com/apache/spark/pull/14496 cc @rxin - Follow-on to #14393. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14487: [SPARK-16884] Move DataSourceScanExec out of Exis...

2016-08-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14487 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #11127: [SPARK-13238][Core] Add ganglia dmax parameter

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/11127 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14495: MAINTENANCE. Cleaning up stale PRs.

2016-08-04 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14495 Adding #9866 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #14239: [SPARK-16593] [CORE] Provide a pre-fetch mechanism to ac...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14239 **[Test build #63235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63235/consoleFull)** for PR 14239 at commit

[GitHub] spark issue #14469: [SPARK-16700] [PYSPARK] [SQL] create DataFrame from dict...

2016-08-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14469 **[Test build #63234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63234/consoleFull)** for PR 14469 at commit

<    1   2   3   4   5   >