[GitHub] spark pull request: [SPARK-4009][SQL]HiveTableScan should use make...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2857#issuecomment-59816505 This isn't a valid change since different partitions of a table can have different serializers. As a result I think we should close this issue. --- If your proje

[GitHub] spark pull request: [SPARK-4003] [SQL] add 3 types for java SQL co...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2850#issuecomment-59816592 Can you add some tests too? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SQL] Refactors data type pattern matching

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2764#discussion_r19118466 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -107,20 +107,20 @@ trait HiveTypeCoercion

[GitHub] spark pull request: [SQL] Refactors data type pattern matching

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2764#discussion_r19118592 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/dataTypes.scala --- @@ -319,10 +315,8 @@ case object ByteType extends

[GitHub] spark pull request: [SPARK-3906][SQL] Adds multiple join support f...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2767#issuecomment-59850744 Thanks, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3483][SQL] Special chars in column name...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2804#issuecomment-59852990 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3966][SQL] Fix nullabilities of Cast re...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2820#issuecomment-59853602 Thanks, I've merged this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-3939][SQL] NPE caused by SessionState.o...

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2812#discussion_r19120144 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala --- @@ -69,6 +71,10 @@ object

[GitHub] spark pull request: [SPARK-3814][SQL] Bitwise & does not work in H...

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2789#discussion_r19121230 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -64,6 +64,23 @@ abstract class BinaryArithmetic

[GitHub] spark pull request: [SPARK-3814][SQL] Bitwise & does not work in H...

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2789#discussion_r19121441 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -64,6 +64,23 @@ abstract class BinaryArithmetic

[GitHub] spark pull request: [SPARK-3569][SQL] Add metadata field to Struct...

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2701#discussion_r19121733 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -43,6 +44,9 @@ abstract class Expression extends

[GitHub] spark pull request: [SPARK-3569][SQL] Add metadata field to Struct...

2014-10-20 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2701#discussion_r19121837 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/Metadata.scala --- @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [HOTFIX][SQL] missing golden file for SPARK-38...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2821#issuecomment-59859591 I'm confused. How can Jenkins be passing if a golden file is missing? --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-3569][SQL] Add metadata field to Struct...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2701#issuecomment-59859828 I looked this over and only had a few minor comments. Also, can you update the PR description? I think its pretty out of date. --- If your project is set up for it

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-59859939 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3969][SQL] Optimizer should have a supe...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2825#issuecomment-59860308 Thanks, merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3925][SQL] Do not consider the ordering...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2783#issuecomment-59860581 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-59860616 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3925][SQL] Do not consider the ordering...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2783#issuecomment-59864322 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-59877085 I think this is looking pretty good, but I'm not okay with merging it before the tests are passing for Hive 13. Let me take a look and see how hard that wi

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-20 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-59880136 @scwf, which PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-21 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-59962787 A few comments: I'm not talking only about getting the current tests to pass, but upgrading the test set to include the new files. Also, I hope to updat

[GitHub] spark pull request: [SPARK-2663] [SQL] Support the Grouping Set

2014-10-21 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1567#issuecomment-59969282 Yeah, please do post a design doc. Also, sorry for not reviewing this earlier. This will be a good feature to have. I did a quick pass and I have two high

[GitHub] spark pull request: [SPARK-4050][SQL] Fix caching of temporary tab...

2014-10-23 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/2912 [SPARK-4050][SQL] Fix caching of temporary tables with projections. Previously cached data was found by `sameResult` plan matching on optimized plans. This technique however fails to locate the

[GitHub] spark pull request: [spark-3907][sql] add truncate table support

2014-10-23 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2770#discussion_r19324134 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala --- @@ -121,7 +121,8 @@ private[hive] object HiveQl { // Commands that we do

[GitHub] spark pull request: [SPARK-4074][SQL] Throw exception when droppin...

2014-10-24 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/2923 [SPARK-4074][SQL] Throw exception when dropping nonexistent table. You can merge this pull request into a Git repository by running: $ git pull https://github.com/marmbrus/spark

[GitHub] spark pull request: [SPARK-2706][SQL] Enable Spark to support Hive...

2014-10-24 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2241#issuecomment-60426160 Okay, I've merged this to master. Will file a PR shortly to fix the tests. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-24 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2884#discussion_r19354518 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala --- @@ -81,7 +81,8 @@ case class HiveTableScan

[GitHub] spark pull request: [SPARK-4074][SQL] Throw exception when droppin...

2014-10-24 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2923#issuecomment-60441658 These failures look legitimate. I'll investigate. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: [HOTFIX][SQL] Remove sleep on reset() failure.

2014-10-24 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/2934 [HOTFIX][SQL] Remove sleep on reset() failure. You can merge this pull request into a Git repository by running: $ git pull https://github.com/marmbrus/spark patch-2 Alternatively you can

[GitHub] spark pull request: [SQL] Update Hive test harness to work with bo...

2014-10-24 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/2936 [SQL] Update Hive test harness to work with both Hive 12 and 13 As part of the upgrade I also copy the newest version of the query tests, and whitelist a bunch of new ones that are now passing

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2841#issuecomment-60475393 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2841#issuecomment-60491281 There are style violations. Run `sbt/sbt scalastyle` to see what they are. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2841#issuecomment-60491361 Spaces are required before comments: ``` [error] /Users/marmbrus/workspace/spark/sql/core/src/main/scala/org/apache/spark/sql/parquet

[GitHub] spark pull request: [HOTFIX][SQL] Temporarily turn of hive-server ...

2014-10-26 Thread marmbrus
GitHub user marmbrus opened a pull request: https://github.com/apache/spark/pull/2950 [HOTFIX][SQL] Temporarily turn of hive-server tests. The thirift server is not available in the default (hive13) profile yet which is breaking all SQL only PRs. This turns off these test until

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60533046 Now that Hive 13 is merged in it would be great to get this in ASAP. I looked over this and it seems pretty good. My only high level comment is maybe we should keep

[GitHub] spark pull request: [SPARK-3925][SQL] Do not consider the ordering...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2783#issuecomment-60533106 Thanks! I've merged this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383897 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -460,27 +481,43 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383900 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -460,27 +481,43 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383917 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -450,8 +465,14 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383923 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -450,8 +465,14 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383935 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -450,8 +465,14 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383949 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -460,27 +481,43 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383955 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -106,7 +110,13 @@ case class ParquetTableScan

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2841#discussion_r19383980 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala --- @@ -450,8 +465,14 @@ private[parquet] class

[GitHub] spark pull request: SPARK-3968 Use parquet-mr filter2 api in spark...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2841#issuecomment-60533736 This is pretty awesome. Thanks for working on it! I made a few comments. @mateiz you had also looked into upgrading our version of parquet. Is this missing

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2716#discussion_r19384056 --- Diff: python/pyspark/sql.py --- @@ -995,19 +1038,22 @@ def registerFunction(self, name, f, returnType=StringType

[GitHub] spark pull request: [SPARK-3594] [PySpark] [SQL] take more rows to...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2716#issuecomment-60534136 Minor comment on documentation wording. Otherwise this LGTM! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60534222 @davies does anything else remain to be done here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3816][SQL] Add table properties from st...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2677#issuecomment-60534260 Can you please rebase this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2570#discussion_r19384148 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -160,17 +162,14 @@ private[hive] trait HiveStrategies { def

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2570#discussion_r19384160 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateTableAsSelect.scala --- @@ -30,33 +32,46 @@ import

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60534526 This would be a great feature to get in before 1.2. I made a few minor comments. Also, can you rebase? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-26 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2685#discussion_r19384192 --- Diff: dev/run-tests --- @@ -167,7 +167,7 @@ CURRENT_BLOCK=$BLOCK_SPARK_UNIT_TESTS # If the Spark SQL tests are enabled, run the tests with the

[GitHub] spark pull request: [SPARK-3904] [SQL] add constant objectinspecto...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2762#issuecomment-60534712 How does this compare with #2802? They look very similar. Also can you rebase please? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [HOTFIX][SQL] missing golden file for SPARK-38...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2821#issuecomment-60535502 Thanks for doing this, but I think this has been superseded by the refreshed golden files from the upgrade to hive 13. Unless I'm wrong can we close this

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60536387 Great, thanks for finding this and adding a test. Regarding the implementation, I'd like to try to avoid doing too much string munging as its generally easy to f

[GitHub] spark pull request: [SPARK-4003] [SQL] add 3 types for java SQL co...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2850#issuecomment-60537184 Thanks for working on this! In order for the BigDecimals to work we are going to need to do some conversions as well. This is because the expected internal

[GitHub] spark pull request: [SQL][DOC] Wrong package name "scala.math.sql"...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2873#issuecomment-60537245 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3953][SQL][Minor] Confusable variable n...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2807#issuecomment-60537284 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4052][SQL] Use scala.collection.Map for...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2899#issuecomment-60537355 Thanks @yhuai! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [spark-3907][sql] add truncate table support

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2770#issuecomment-60537463 It appears there are some style violations: ``` [marmbrus@michaels-mbp spark (pr/2770)]$ sbt hive/scalastyle [info] Set current project to spark-parent

[GitHub] spark pull request: [SPARK-3483][SQL] Special chars in column name...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2927#issuecomment-60537528 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3959][SPARK-3960][SQL] SqlParser fails ...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2816#issuecomment-60537666 Awesome, thanks for doing this and for all the test cases! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-3814][SQL] Support for Bitwise AND(&), ...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2926#issuecomment-60537966 This LGTM. Can you please merge with master and push to this branch. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4061][SQL] We cannot use EOL character ...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2908#issuecomment-60538119 Thanks for working on this! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [Spark-4042][SQL]append columns ids and names ...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2885#issuecomment-60538205 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3569][SQL] Add metadata field to Struct...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2701#issuecomment-60538403 @mengxr, thanks for working on this! Overall LGTM. One minor thing: I think we should expose Metadata as a type variable in the `org.apache.spark.sql` package and as

[GitHub] spark pull request: [SPARK-3816][SQL] Add table properties from st...

2014-10-26 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2677#issuecomment-60539054 Hmm, this failure looks related to @liancheng patch on statistics that I just merged. On Oct 26, 2014 5:18 PM, "Apache Spark QA" wrote: &g

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60556909 Yeah, I like that idea. On Oct 26, 2014 8:43 PM, "wangfei" wrote: > @marmbrus <https://github.com/marmbrus>, how about make a new sub

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60641434 @chenghao-intel, because Hive is not compatible with metastores of a different version we will need to support Hive 12 and 13 concurrently for at least a little while

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60705802 @chenghao-intel there is already support for conditional compilation based on hive version. This code can go in `sql/hive/v0.X.0/src/main/scala/org/apache/spark/sql

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2771#issuecomment-60706048 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3816][SQL] Add table properties from st...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2677#issuecomment-60706146 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60706370 Minor comment: In the future please put SPARK- in all capitals in the title so that our merge scripts recognize it. Thanks! Thanks for working on this

[GitHub] spark pull request: [SPARK-3814][SQL] Support for Bitwise AND(&), ...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2961#issuecomment-60706385 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SQL] Correct a variable name in JavaApplySche...

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2869#issuecomment-60706514 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [spark-3907][sql] add truncate table support

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2770#issuecomment-60706609 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [spark-3907][sql] add truncate table support

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2770#issuecomment-60706659 Can you please make the title SPARK- so that the merge script recognizes the JIRA? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-3907][sql] add truncate table support

2014-10-27 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2770#issuecomment-60710381 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-4108][sql]

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2970#issuecomment-60811305 Can you please change the title to "[SPARK-4108][SQL] Fixed usage of deprecated in sql/catalyst/types/datatypes"? Our merge script uses this as the comm

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60815370 @scwf, okay cool that sounds like a good compromise. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60818913 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4133] [SQL] [PySpark] type conversionfo...

2014-10-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2973#discussion_r19498469 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala --- @@ -428,9 +397,10 @@ class SchemaRDD( */ private[sql] def

[GitHub] spark pull request: [SPARK-4133] [SQL] [PySpark] type conversionfo...

2014-10-28 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/2973#discussion_r19498538 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/pythonUdfs.scala --- @@ -141,8 +213,11 @@ case class BatchPythonEvaluation(udf: PythonUDF

[GitHub] spark pull request: [SPARK-4133] [SQL] [PySpark] type conversionfo...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2973#issuecomment-60820309 Thanks for the quick fix! A few minor comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SQL][Doc] Add description for Spark SQL CLI a...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2976#issuecomment-60822257 This README is really more for developers, while this looks like information that should go in the SQL programming guide. I'd be okay with adding a link to the l

[GitHub] spark pull request: [SPARK-4074][SQL] Throw exception when droppin...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2923#issuecomment-60825290 @mateiz points out this is actually configurable in hive an purposefully off by default. Closing this. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-4074][SQL] Throw exception when droppin...

2014-10-28 Thread marmbrus
Github user marmbrus closed the pull request at: https://github.com/apache/spark/pull/2923 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3720][SQL]initial support ORC in spark ...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2576#issuecomment-60826422 It may be difficult to get this in for 1.2.0, mostly because it should use the new external data API and that is going to go in at the last minute. Also, we are pretty

[GitHub] spark pull request: [SPARK-3814][SQL] Support for Bitwise AND(&), ...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2961#issuecomment-60826868 Thanks for implementing this! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2489] [SQL] Parquet support for fixed_l...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1737#issuecomment-60827613 The problem is that dataTypes are a public api so once we add one we are stuck with it for ever. Also, each new datatype adds significant overhead so I'd like

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60827970 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3343] [SQL] Add serde support for CTAS

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2570#issuecomment-60835836 Thanks for working on this! I'm going to merge it in, but we should consider moving the semantic analyzer part to the analysis phase. These execution APIs ar

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60839602 I agree, but I would like to continue to at least build with hive-12 so that we make sure patches do not break the hive 12 build. Perhaps we can build first with hive

[GitHub] spark pull request: [SPARK-3904] [SQL] add constant objectinspecto...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2762#issuecomment-60840095 Also, this needs to be updated again (I think it conflicted with your CTAS patch). --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60840749 I'm afraid that will take too long to do it for each PR. We do plan to set up a separate build that runs the hive 12 tests nightly. --- If your project is set u

[GitHub] spark pull request: [SPARK-4003] [SQL] add 3 types for java SQL co...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2850#issuecomment-60841716 @adrian-wang any chance you will have time to update this before the 1.2 code freeze this Friday? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [WIP][SQL][SPARK-3839] Reimplement Left/Right ...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2723#issuecomment-60841760 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3826][SQL]enable hive-thriftserver to s...

2014-10-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/2685#issuecomment-60843170 I think you have the right idea, but maybe we can isolate the changes a little as I commented on the diff. --- If your project is set up for it, you can reply to this

  1   2   3   4   5   6   7   8   9   10   >