[GitHub] spark pull request #17148: [SPARK-17075][SQL][followup] fix filter estimatio...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17148 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17096 Thank you @viirya for your sign-off. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17148: [SPARK-17075][SQL][followup] fix filter estimation issue...

2017-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17148 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17090: [Spark-19535][ML] RecommendForAllUsers RecommendForAllIt...

2017-03-06 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/17090 I commented further on the [JIRA](https://issues.apache.org/jira/browse/SPARK-14409?focusedCommentId=15898855=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15898855).

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104605419 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -17,89 +17,55 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104604870 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -17,89 +17,55 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request #17178: [SPARK-19828][R] Support array type in from_json ...

2017-03-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17178#discussion_r104604621 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1342,28 +1342,52 @@ test_that("column functions", { df <- read.json(mapTypeJsonPath)

[GitHub] spark issue #17146: [SPARK-19806][ML][PySpark] PySpark GeneralizedLinearRegr...

2017-03-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17146 @actuaryzhang would you take a look at this one. If recall, it's one option we considered for R API. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #17146: [SPARK-19806][ML][PySpark] PySpark GeneralizedLin...

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17146#discussion_r104604254 --- Diff: python/pyspark/ml/regression.py --- @@ -1344,40 +1347,53 @@ class GeneralizedLinearRegression(JavaEstimator, HasLabelCol, HasFeaturesCol, Ha

[GitHub] spark issue #16896: [SPARK-19561][Python] cast TimestampType.toInternal outp...

2017-03-06 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/16896 lgtm, will merge it when I get a chance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16290: [SPARK-18817] [SPARKR] [SQL] Set default warehouse dir t...

2017-03-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16290 so base on this comment https://github.com/apache/spark/pull/16330#issuecomment-282101389 doesn't it mean we shouldn't set warehouse dir to under tempdir()? --- If your project is set up

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-03-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17096 Remaining changes LGTM. cc @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17182: [SPARK-19840][SQL] Disallow creating permanent functions...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17182 **[Test build #74078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74078/testReport)** for PR 17182 at commit

[GitHub] spark issue #17186: [SPARK-19846][SQL] Add a flag to disable constraint prop...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17186 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17186: [SPARK-19846][SQL] Add a flag to disable constraint prop...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17186 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74071/ Test PASSed. ---

[GitHub] spark issue #17186: [SPARK-19846][SQL] Add a flag to disable constraint prop...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17186 **[Test build #74071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74071/testReport)** for PR 17186 at commit

[GitHub] spark issue #16856: [SPARK-19516][DOC] update public doc to use SparkSession...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16856 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74077/ Test PASSed. ---

[GitHub] spark issue #16856: [SPARK-19516][DOC] update public doc to use SparkSession...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16856 **[Test build #74077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74077/testReport)** for PR 16856 at commit

[GitHub] spark issue #16856: [SPARK-19516][DOC] update public doc to use SparkSession...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16856 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17144: [SPARK-19803][TEST] flaky BlockManagerReplication...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17144#discussion_r104602172 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala --- @@ -494,7 +494,9 @@ class

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16867 Thanks a lot for comments. I refined accordingly : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #17144: [SPARK-19803][TEST] flaky BlockManagerReplication...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17144#discussion_r104602103 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala --- @@ -494,7 +494,9 @@ class

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74070/ Test PASSed. ---

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16867 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16867 **[Test build #74070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74070/testReport)** for PR 16867 at commit

[GitHub] spark issue #15229: [SPARK-17654] [SQL] Propagate bucketing information for ...

2017-03-06 Thread carlos-verdes
Github user carlos-verdes commented on the issue: https://github.com/apache/spark/pull/15229 Hi @rxin, In Hive you have two levels, the partition and the buckets. The partitons are translated to folders on HDFS, for example: ```bash

[GitHub] spark issue #16896: [SPARK-19561][Python] cast TimestampType.toInternal outp...

2017-03-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16896 +1 LGTM. Could you review and merge this please, @davies ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #16856: [SPARK-19516][DOC] update public doc to use SparkSession...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16856 **[Test build #74077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74077/testReport)** for PR 16856 at commit

[GitHub] spark pull request #16896: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-03-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16896#discussion_r104601283 --- Diff: python/pyspark/sql/types.py --- @@ -189,7 +189,7 @@ def toInternal(self, dt): if dt is not None: seconds =

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74069/ Test PASSed. ---

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16867 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16867: [SPARK-16929] Improve performance when check speculatabl...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16867 **[Test build #74069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74069/testReport)** for PR 16867 at commit

[GitHub] spark pull request #16856: [SPARK-19516][DOC] update public doc to use Spark...

2017-03-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16856#discussion_r104600900 --- Diff: docs/quick-start.md --- @@ -438,8 +412,7 @@ Lines with a: 46, Lines with b: 23 # Where to Go from Here Congratulations on running your

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104600104 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala --- @@ -64,7 +64,7 @@ private[spark] object UIData { var

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104599195 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala --- @@ -54,6 +54,9 @@ private[spark] trait TaskScheduler { //

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104598647 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -239,8 +244,9 @@ private[spark] class Executor( */

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104599383 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104599319 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104600582 --- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala --- @@ -538,10 +538,37 @@ class SparkContextSuite extends SparkFunSuite with

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104600689 --- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala --- @@ -538,10 +538,37 @@ class SparkContextSuite extends SparkFunSuite with

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104599420 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104599899 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -710,7 +710,11 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #17178: [SPARK-19828][R] Support array type in from_json ...

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17178#discussion_r104600695 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1342,28 +1342,52 @@ test_that("column functions", { df <- read.json(mapTypeJsonPath)

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104600696 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -278,6 +278,8 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #17176: [SPARK-19833][SQL]remove SQLConf.HIVE_VERIFY_PARTITION_P...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17176 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74072/ Test FAILed. ---

[GitHub] spark issue #17176: [SPARK-19833][SQL]remove SQLConf.HIVE_VERIFY_PARTITION_P...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17176 **[Test build #74072 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74072/testReport)** for PR 17176 at commit

[GitHub] spark issue #17176: [SPARK-19833][SQL]remove SQLConf.HIVE_VERIFY_PARTITION_P...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17176 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16944: [SPARK-19611][SQL] Introduce configurable table schema i...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16944 **[Test build #74076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74076/testReport)** for PR 16944 at commit

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-03-06 Thread budde
Github user budde commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r104598894 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -476,46 +476,6 @@ object

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104598515 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/executor/MesosExecutorBackend.scala --- @@ -104,7 +104,8 @@ private[spark] class

[GitHub] spark issue #17097: [SPARK-19765][SPARK-18549][SQL] UNCACHE TABLE should un-...

2017-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17097 @gatorsmile I have put the JIRA number in this PR title, without adding a test, because the new behavior this PR introduced can obviously fix that bug. I don't want to add a lot of end

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104598293 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -467,7 +474,7 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark issue #17090: [Spark-19535][ML] RecommendForAllUsers RecommendForAllIt...

2017-03-06 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/17090 @MLnick OK I think I misunderstood some of your comments above then. I see the proposal in SPARK-14409 differs from this PR, so I agree it'd be nice to resolve it. We can make changes to this

[GitHub] spark pull request #16944: [SPARK-19611][SQL] Introduce configurable table s...

2017-03-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16944#discussion_r104598287 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -476,46 +476,6 @@ object

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104598221 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark pull request #17173: [SPARK-19832][SQL]DynamicPartitionWriteTask get p...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17173 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17187: [SPARK-19847][SQL] port hive read to FileFormat API

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17187 **[Test build #74075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74075/testReport)** for PR 17187 at commit

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74068/ Test PASSed. ---

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17173: [SPARK-19832][SQL]DynamicPartitionWriteTask get partitio...

2017-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17173 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15363 **[Test build #74068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74068/testReport)** for PR 15363 at commit

[GitHub] spark issue #17187: [SPARK-19847][SQL] port hive read to FileFormat API

2017-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17187 cc @sameeragarwal @rxin @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17187: [SPARK-19847][SQL] port hive read to FileFormat A...

2017-03-06 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/17187 [SPARK-19847][SQL] port hive read to FileFormat API ## What changes were proposed in this pull request? implement the read logic in `HiveFileFormat`, to unify the table read path

[GitHub] spark issue #16290: [SPARK-18817] [SPARKR] [SQL] Set default warehouse dir t...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16290 **[Test build #74074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74074/testReport)** for PR 16290 at commit

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104596342 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -17,43 +17,70 @@ package

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104596281 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -17,43 +17,70 @@ package

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17166 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74065/ Test PASSed. ---

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74065 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74065/testReport)** for PR 17166 at commit

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104594383 --- Diff: R/pkg/R/generics.R --- @@ -1420,6 +1420,17 @@ setGeneric("spark.posterior", function(object, newData) { standardGeneric("spark #'

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104595228 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104595814 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/FPGrowthWrapper.scala --- @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104594800 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104595735 --- Diff: R/pkg/inst/tests/testthat/test_mllib_fpm.R --- @@ -0,0 +1,74 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104594654 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104594539 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104595392 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104595125 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104595454 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104594212 --- Diff: R/pkg/DESCRIPTION --- @@ -54,5 +55,5 @@ Collate: 'types.R' 'utils.R' 'window.R' -RoxygenNote: 5.0.1

[GitHub] spark pull request #17170: [SPARK-19825][R][ML] spark.ml R API for FPGrowth

2017-03-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17170#discussion_r104594501 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104595706 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark issue #17166: [SPARK-19820] [core] Allow reason to be specified for ta...

2017-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17166 **[Test build #74073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74073/testReport)** for PR 17166 at commit

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104595290 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1197,6 +1199,65 @@ class

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104595023 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -158,7 +158,8 @@ private[spark] class Executor( threadPool.execute(tr)

[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...

2017-03-06 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17164 This pr added an new SQL option `spark.sql.aggregate.preferSortAggregate` to preferably select `SortAggregate` for easy-to-test in `DataFrameAggregateSuite.scala`. In some cases (e.g., input data is

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104595065 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala --- @@ -40,7 +40,8 @@ private[spark] object

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104594970 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark issue #17148: [SPARK-17075][SQL][followup] fix filter estimation issue...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17148 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17148: [SPARK-17075][SQL][followup] fix filter estimation issue...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74067/ Test PASSed. ---

[GitHub] spark pull request #17151: [ML][Minor] Separate estimator and model params f...

2017-03-06 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/17151#discussion_r104594062 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala --- @@ -372,16 +372,18 @@ class

[GitHub] spark pull request #17159: [SPARK-19818][SparkR] rbind should check for name...

2017-03-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17159 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104593920 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala --- @@ -40,7 +40,8 @@ private[spark] object

[GitHub] spark pull request #16826: [SPARK-19540][SQL] Add ability to clone SparkSess...

2017-03-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16826#discussion_r104593862 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala --- @@ -1197,6 +1199,65 @@ class

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104593825 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -732,6 +732,13 @@ class DAGScheduler( } /** + *

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104593790 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -158,7 +158,8 @@ private[spark] class Executor( threadPool.execute(tr)

[GitHub] spark issue #17159: [SPARK-19818][SparkR] rbind should check for name consis...

2017-03-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17159 merged to master. thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104593710 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark pull request #17166: [SPARK-19820] [core] Allow reason to be specified...

2017-03-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/17166#discussion_r104593724 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends Logging { }

[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...

2017-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74066/ Test PASSed. ---

  1   2   3   4   5   6   7   8   >