[GitHub] spark issue #16575: [SPARK-19213] DatasourceScanExec uses runtime sparksessi...

2017-01-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16575 @robert3005 why is this needed? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #16581: [SPARK-18589] [SQL] Fix Python UDF accessing attr...

2017-01-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16581#discussion_r96121284 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -86,6 +86,19 @@ trait PredicateHelper

[GitHub] spark issue #16608: [SPARK-13721][SQL] Support outer generators in DataFrame...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16608 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96275903 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -300,7 +311,7 @@ abstract class ExplodeBase

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96275746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -233,11 +244,11 @@ abstract class ExplodeBase

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96276717 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2879,6 +2888,15 @@ object functions { def posexplode(e: Column

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96276562 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala --- @@ -160,9 +160,20 @@ case class GenerateExec

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96276812 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -175,6 +175,9 @@ object FunctionRegistry

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96276686 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2871,6 +2871,15 @@ object functions { def explode(e: Column): Column

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96275919 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -323,7 +334,7 @@ case class Explode(child

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96275197 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -204,6 +204,17 @@ case class Stack(children: Seq

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96275427 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -175,6 +175,9 @@ object FunctionRegistry

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96275937 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -369,3 +380,5 @@ case class Inline(child

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96323578 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionToSQLSuite.scala --- @@ -102,6 +102,9 @@ class ExpressionToSQLSuite extends

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96324322 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -508,4 +511,12 @@ object FunctionRegistry

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96323956 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -104,7 +104,10 @@ case class

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96324372 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -508,4 +511,12 @@ object FunctionRegistry

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96324395 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -508,4 +511,12 @@ object FunctionRegistry

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96323864 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -204,6 +204,17 @@ case class Stack(children: Seq

[GitHub] spark pull request #16608: [SPARK-13721][SQL] Support outer generators in Da...

2017-01-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16608#discussion_r96323976 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2879,6 +2888,15 @@ object functions { def posexplode(e: Column

[GitHub] spark issue #15302: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15302 @dongjoon-hyun I have taken a quick look. Shouldn't we just use Expressions for filtering partitions? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15047 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15047 @tejasapatil I have triggered a new build. I'll merge this as soon as it completes successfully. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #15302: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15302 @dongjoon-hyun I think that `AlterTableDropPartitionCommand` just should take a set of catalyst `Expression`s instead of a `PartitionRangeSpec`. I have added a few things the `Catalog

[GitHub] spark issue #15337: [SPARK-17773] [Input/Output] Add VoidObjectInspector

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15337 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15337: [SPARK-17773] [Input/Output] Add VoidObjectInspector

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15337 @seyfe could we add a little test to see if this is working? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15322: [SPARK-17753][SQL] Allow a complex expression as the inp...

2016-10-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15322 Merging to master/2.0. Thanks for the reviews! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15345: [SPARK-17773] [Input/Output] [branch-2.0] Add VoidObject...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15345 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15348: [SPARK-17758][SQL] Last returns wrong result in c...

2016-10-04 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/15348 [SPARK-17758][SQL] Last returns wrong result in case of empty partition ## What changes were proposed in this pull request? The result of the `Last` function can be wrong when the last

[GitHub] spark issue #15345: [SPARK-17773] [Input/Output] [branch-2.0] Add VoidObject...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15345 LGTM - merging to 2.0. Thanks! Could you close this PR? (our merge tooling does not close backports automatically) --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #15348: [SPARK-17758][SQL] Last returns wrong result in c...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15348#discussion_r81887208 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/aggregate/LastTestSuite.scala --- @@ -0,0 +1,105

[GitHub] spark pull request #15346: [SPARK-17741][SQL] Grammar to parse top level and...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15346#discussion_r81821454 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -593,6 +593,14 @@ colTypeList ; colType

[GitHub] spark pull request #15346: [SPARK-17741][SQL] Grammar to parse top level and...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15346#discussion_r81887963 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala --- @@ -67,9 +86,331 @@ class SparkSqlParserSuite extends

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15047 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15319: [SPARK-17733][SQL] InferFiltersFromConstraints rule neve...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15319 cc @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15044: [WIP][SQL][SPARK-17490] Optimize SerializeFromObject() f...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15044 @kiszk this seems nice, is this ready for review? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15044: [WIP][SQL][SPARK-17490] Optimize SerializeFromObject() f...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15044 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15355: [SPARK-17782][STREAMING]Disable Kafka 010 pattern...

2016-10-04 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/15355 [SPARK-17782][STREAMING]Disable Kafka 010 pattern based subscription test. ## What changes were proposed in this pull request? This PR disables the `pattern based subscription` test in the

[GitHub] spark issue #15355: [SPARK-17782][STREAMING]Disable Kafka 010 pattern based ...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15355 cc @koeninger any idea why this flaky? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14828: [SPARK-17258][SQL] Parse scientific decimal literals as ...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14828 @gatorsmile does this LGTY? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15357: [SPARK-17328][SQL] Fix NPE with EXPLAIN DESCRIBE ...

2016-10-04 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15357#discussion_r81900460 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -265,7 +265,9 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark issue #15357: [SPARK-17328][SQL] Fix NPE with EXPLAIN DESCRIBE TABLE

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15357 LGTM - merging to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15044 @kiszk this PR is. It is much more concise! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r82056906 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/PrimitiveArrayBenchmark.scala --- @@ -0,0 +1,72 @@ +/* + * Licensed

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r82058100 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -441,6 +441,28 @@ object ScalaReflection extends

[GitHub] spark issue #15348: [SPARK-17758][SQL] Last returns wrong result in case of ...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15348 You cannot. Last is not deterministic outside of ordered window functions. The only thing is that you can have inconsistent results if the last physical partition happens to be an empty one

[GitHub] spark issue #14912: [SPARK-17357][SQL] Fix current predicate pushdown

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14912 @viirya TBH this seems hacky to me and I'd rather not merge this. I think we should just focus on having proper CNF in the optimizer. I am sorry to disappoint you. --- If your project i

[GitHub] spark issue #14828: [SPARK-17258][SQL] Parse scientific decimal literals as ...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14828 @gatorsmile we tried that in Spark 2.0. But it turns out that this is quite confusing to an end-user, and that this is problematic in case of testing (due to small numeric differences

[GitHub] spark issue #15363: [SPARK-17791][SQL] Join reordering using star schema det...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15363 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15371: [SPARK-17463] [Core] Fix ConcurrentModificationException...

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15371 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15323: [SPARK-17757][SQL] Remove CreateNamedStruct[Unsaf...

2016-10-05 Thread hvanhovell
Github user hvanhovell closed the pull request at: https://github.com/apache/spark/pull/15323 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #15323: [SPARK-17757][SQL] Remove CreateNamedStruct[Unsafe]

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15323 closing this in favor of https://github.com/apache/spark/pull/1/files --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-10-06 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r82244393 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -441,6 +441,28 @@ object ScalaReflection extends

[GitHub] spark issue #15351: [SPARK-17612][SQL][branch-2.0] Support `DESCRIBE table P...

2016-10-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15351 @dongjoon-hyun it LGTM. It is just a rather big patch to backport, for something that is not a bug fix. But I'll merge it. --- If your project is set up for it, you can reply to this emai

[GitHub] spark issue #15351: [SPARK-17612][SQL][branch-2.0] Support `DESCRIBE table P...

2016-10-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15351 Merging to 2.0. @dongjoon-hyun can you close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15355: [SPARK-17782][STREAMING] Disable Kafka 010 pattern based...

2016-10-06 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15355 I have re-enables the kafka test. This PR now only contains a change to build infrastructure. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15390: [SPARK-17806] [SQL] fix bug in join key rewritten in Has...

2016-10-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15390 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15333: [SPARK-17761][SQL] Remove MutableRow

2016-10-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15333 Merging to master. Thanks for commenting. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15297: [WIP][SPARK-9862]Handling data skew

2016-10-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15297 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #15403: [SPARK-17832][SQL] TableIdentifier.quotedString c...

2016-10-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15403#discussion_r82506382 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/identifiers.scala --- @@ -29,12 +28,23 @@ sealed trait IdentifierWithDatabase

[GitHub] spark issue #15403: [SPARK-17832][SQL] TableIdentifier.quotedString creates ...

2016-10-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15403 @jiangxb1987 This looks pretty good. What is the problem `getTable` and `getFunction`? Could you give an example? assume you are talking about the `org.apache.spark.sql.catalog.Catalog

[GitHub] spark issue #15403: [SPARK-17832][SQL] TableIdentifier.quotedString creates ...

2016-10-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15403 LGTM - merging to master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15346 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14788 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-09 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14788 LGTM - I'll merge as soon as tests complete successfully --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysi...

2016-10-09 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15316#discussion_r82545437 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/AnalysisException.scala --- @@ -43,6 +43,11 @@ class AnalysisException protected[sql

[GitHub] spark pull request #14788: [SPARK-17174][SQL] Add the support for TimestampT...

2016-10-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14788#discussion_r82650345 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2548,16 +2548,20 @@ object functions { def to_date(e: Column

[GitHub] spark pull request #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysi...

2016-10-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15316#discussion_r82650465 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/AnalysisException.scala --- @@ -43,6 +43,11 @@ class AnalysisException protected[sql

[GitHub] spark pull request #15416: [SPARK-17849] [SQL] Fix NPE problem when using gr...

2016-10-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15416#discussion_r82664096 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -298,10 +298,11 @@ class Analyzer

[GitHub] spark pull request #15072: [SPARK-17123][SQL] Use type-widened encoder for D...

2016-10-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15072#discussion_r82726273 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -53,7 +53,15 @@ import org.apache.spark.util.Utils private[sql

[GitHub] spark issue #15412: [SPARK-17844] Simplify DataFrame API for defining frame ...

2016-10-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15412 LGTM - merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15072: [SPARK-17123][SQL] Use type-widened encoder for D...

2016-10-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15072#discussion_r82728542 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -53,7 +53,15 @@ import org.apache.spark.util.Utils private[sql

[GitHub] spark pull request #15072: [SPARK-17123][SQL] Use type-widened encoder for D...

2016-10-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15072#discussion_r82743571 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -53,7 +53,15 @@ import org.apache.spark.util.Utils private[sql

[GitHub] spark issue #15448: [SPARK-17108][SQL]: Fix BIGINT and INT comparison failur...

2016-10-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15448 Could you try to fix this by adding implicit casting to the `GetMapValue` (make it extend `ImplicitCastInputTypes` instead of `ExpectsInputTypes`)? --- If your project is set up for it, you

[GitHub] spark issue #15448: [SPARK-17108][SQL]: Fix BIGINT and INT comparison failur...

2016-10-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15448 Also add unit tests please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-10-11 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/13065#discussion_r82944265 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala --- @@ -99,5 +102,182 @@ case class GenerateExec

[GitHub] spark issue #13065: [SPARK-15214][SQL] Code-generation for Generate

2016-10-12 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13065 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15457: [SPARK-17830][SQL] Annotate remaining SQL APIs with Inte...

2016-10-12 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15457 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15457: [SPARK-17830][SQL] Annotate remaining SQL APIs with Inte...

2016-10-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15457 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14788: [SPARK-17174][SQL] Add the support for TimestampT...

2016-10-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14788#discussion_r83280122 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -916,21 +931,26 @@ case class ToDate

[GitHub] spark issue #15464: [SPARK-17827][SQL]maxColLength type should be Int for St...

2016-10-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15464 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15044: [SQL][SPARK-17490] Optimize SerializeFromObject()...

2016-10-13 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15044#discussion_r83300104 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystTypeConvertersSuite.scala --- @@ -61,4 +63,39 @@ class

[GitHub] spark issue #15416: [SPARK-17849] [SQL] Fix NPE problem when using grouping ...

2016-10-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15416 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15487: [SPARK-17940][SQL] Fixed a typo in LAST function ...

2016-10-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15487#discussion_r83470770 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Last.scala --- @@ -29,15 +29,18 @@ import

[GitHub] spark issue #15512: [SPARK-17930][CORE]The SerializerInstance instance used ...

2016-10-17 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15512 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysis and O...

2016-10-17 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15316 LGTM. Merging to master/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15316: [SPARK-17751] [SQL] Remove spark.sql.eagerAnalysis and O...

2016-10-17 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15316 @gatorsmile I cannot merge this 2.0. Can you open a backport for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15542: [SPARK-17996][SQL] Fix unqualified catalog.getFun...

2016-10-18 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/15542 [SPARK-17996][SQL] Fix unqualified catalog.getFunction(...) ## What changes were proposed in this pull request? Currently an unqualified `getFunction(..)`call returns a wrong result; the

[GitHub] spark issue #15542: [SPARK-17996][SQL] Fix unqualified catalog.getFunction(....

2016-10-18 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15542 cc @srinathshankar --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2016-10-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15544#discussion_r83993309 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/HyperLogLogPlusPlus.scala --- @@ -142,319 +84,37 @@ case class

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84003714 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -229,10 +235,17 @@ class Analyzer( * Group

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84003200 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -229,10 +235,17 @@ class Analyzer( * Group

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84007872 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -488,23 +488,13 @@ class AstBuilder extends

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84001582 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -216,10 +216,16 @@ class Analyzer( * Group

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84012469 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -518,59 +517,64 @@ case class

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84003737 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -229,10 +235,17 @@ class Analyzer( * Group

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84005447 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -281,9 +294,11 @@ class Analyzer( s

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-19 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84006017 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -488,23 +488,13 @@ class AstBuilder extends

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-03-15 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16909 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17244: [SPARK-19889][SQL] Make TaskContext callbacks thr...

2017-03-16 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17244#discussion_r106363391 --- Diff: core/src/main/scala/org/apache/spark/TaskContextImpl.scala --- @@ -52,62 +63,79 @@ private[spark] class TaskContextImpl( @volatile

<    1   2   3   4   5   6   7   8   9   10   >