Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/22205
Turning the whole rule off will disable also the existing
Project(LocalRelation). By invalid you mean they will not test what they are
supposed to test anymore
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/22205#discussion_r213108632
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1349,6 +1353,12 @@ object ConvertToLocalRelation
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/22205#discussion_r213102214
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1349,6 +1353,12 @@ object ConvertToLocalRelation
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/22205#discussion_r212918379
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -130,6 +130,10 @@ abstract class Optimizer
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/22205
cc @gatorsmile @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/22205
To justify copying ConvertToLocalRelation earlier: this code produces 10 ms
after this change vs 45 ms before:
```
val manyCols = (0 until 1000).map { i => s"value
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/22205
[SPARK-25212][SQL] Support Filter in ConvertToLocalRelation
## What changes were proposed in this pull request?
Support Filter in ConvertToLocalRelation, similar to how Project works
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/22201
cc @gatorsmile @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/22201
[SPARK-25209][SQL] Avoid deserializer check in Dataset.apply when Dataset
is actually DataFrame
## What changes were proposed in this pull request?
Dataset.apply calls dataset.deserializer
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/21587
maybe also fix `SinglePartition.satisfies`. It is only checking for
ClusteredDistribution and defaults to true otherwise. Luckily,
`SinglePartition.numPartitions` is 1 so `EnsureRequirements
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/20548
[SPARK-23316][SQL] AnalysisException after max iteration reached for IN
query
## What changes were proposed in this pull request?
Added flag ignoreNullability to DataType.equalsStructurally
Github user bogdanrdc closed the pull request at:
https://github.com/apache/spark/pull/20547
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/20547
[SPARK-23316][SQL] AnalysisException after max iteration reached for IN
query
## What changes were proposed in this pull request?
Added flag ignoreNullability to DataType.equalsStructurally
Github user bogdanrdc closed the pull request at:
https://github.com/apache/spark/pull/18563
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/18563
cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/18563
[SPARK-21228][SQL][BRANCH-2.2] InSet incorrect handling of structs
## What changes were proposed in this pull request?
This is backport of https://github.com/apache/spark/pull/18455
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18455#discussion_r125877364
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/PredicateSuite.scala
---
@@ -35,7 +35,8 @@ class PredicateSuite extends
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18455#discussion_r125662252
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -274,24 +278,32 @@ case class InSet(child
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/18455
cc @hvanhovell @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/18455
cc @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/18455
[SPARK-21228][SQL] InSet incorrect handling of structs
## What changes were proposed in this pull request?
When data type is struct, InSet now uses TypeUtils.getInterpretedOrdering
(similar
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/18217
[SPARK-20854][TESTS] Removing duplicate test case
## What changes were proposed in this pull request?
Removed a duplicate case in "SPARK-20854: select hint syntax with
expres
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r119561805
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -407,7 +407,7 @@ class AstBuilder(conf: SQLConf
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r119087361
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameHintSuite.scala ---
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r119087296
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -381,12 +381,12 @@ querySpecification
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r119087261
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -371,7 +371,7 @@ querySpecification
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r119085351
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/hints.scala
---
@@ -25,7 +25,7 @@ import
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r118939619
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/hints.scala
---
@@ -25,7 +25,7 @@ import
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r118661387
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/hints.scala
---
@@ -25,7 +25,7 @@ import
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/18086#discussion_r118660663
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
---
@@ -533,13 +533,16 @@ class AstBuilder(conf: SQLConf
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/18086
[SPARK-20854][SQL] Extend hint syntax to support expressions
## What changes were proposed in this pull request?
SQL hint syntax:
* support expressions such as strings, numbers, etc
Github user bogdanrdc closed the pull request at:
https://github.com/apache/spark/pull/17720
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/17720
Jenkins, retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/17720
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17720
[SPARK-20407][TESTS][BACKPORT-2.1] ParquetQuerySuite 'Enabling/disabling
ignoreCorruptFiles' flaky test
## What changes were proposed in this pull request?
SharedSQLContext.afterEach
Github user bogdanrdc closed the pull request at:
https://github.com/apache/spark/pull/17718
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/17718
wrong diff
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17718
[SPARK-20407][TESTS][BACKPORT-2.1] ParquetQuerySuite 'Enabling/disabling
ignoreCorruptFiles' flaky test
## What changes were proposed in this pull request?
SharedSQLContext.afterEach
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/17701#discussion_r112461621
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala ---
@@ -84,6 +85,10 @@ trait SharedSQLContext extends SQLTestUtils
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/17701
Fails rarely in our own jenkins. The test I added is just part of an
existing test that is run 100 times, that one fails reliably.
---
If your project is set up for it, you can reply
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17701
[SPARK-20407][TESTS] ParquetQuerySuite 'Enabling/disabling
ignoreCorruptFiles' flaky test
## What changes were proposed in this pull request?
SharedSQLContext.afterEach now calls
Github user bogdanrdc closed the pull request at:
https://github.com/apache/spark/pull/17632
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/17632
cc @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17632
[SPARK-19946][TESTS][BACKPORT-2.1] DebugFilesystem.assertNoOpenStreams
should report the open streams to help debugging
## What changes were proposed in this pull request?
Backport for PR
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/17591#discussion_r110646607
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -94,13 +94,25 @@ private class
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/17591#discussion_r110639886
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala
---
@@ -94,13 +94,25 @@ private class
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17592
[SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams thread race
## What changes were proposed in this pull request?
Synchronize access to openStreams map.
## How
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17591
[SPARK-20280][CORE] FileStatusCache Weigher integer overflow
## What changes were proposed in this pull request?
Weigher.weigh needs to return Int but it is possible for an
Array
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17292
DebugFilesystem.assertNoOpenStreams should report the open streams to help
debugging
## What changes were proposed in this pull request?
DebugFilesystem.assertNoOpenStreams throws
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/17026
cc @rxin @hvanhovell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/17026
[SPARK-13721][SQL] Make GeneratorOuter unresolved.
## What changes were proposed in this pull request?
This is a small change to make GeneratorOuter always unresolved. It is
mostly
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/16958
IMO I'd leave it as it is.
The problem with `resolved = false` is that `ResolveAliases` rule will no
longer handle an expression like `UnresolvedAlias(GeneratorOuter(Generator)))`
which
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/16958
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user bogdanrdc closed the pull request at:
https://github.com/apache/spark/pull/16875
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/16875
[BACKPORT-2.1][SPARK-19512][SQL] codegen for compare structs fails #16852
## What changes were proposed in this pull request?
Set currentVars to null in GenerateOrdering.genComparisons
Github user bogdanrdc commented on the issue:
https://github.com/apache/spark/pull/16852
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/16852
[SPARK-19512][SQL] codegen for compare structs fails
## What changes were proposed in this pull request?
Set currentVars to null in GenerateOrdering.genComparisons before genCode
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96323081
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala ---
@@ -160,9 +160,20 @@ case class GenerateExec
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96322905
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
---
@@ -300,7 +311,7 @@ abstract class ExplodeBase
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96322755
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
---
@@ -233,11 +244,11 @@ abstract class ExplodeBase
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96322655
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -175,6 +175,9 @@ object FunctionRegistry
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r9636
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
---
@@ -233,11 +244,11 @@ abstract class ExplodeBase
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96298756
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -175,6 +175,9 @@ object FunctionRegistry
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96298581
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
---
@@ -204,6 +204,17 @@ case class Stack(children: Seq
Github user bogdanrdc commented on a diff in the pull request:
https://github.com/apache/spark/pull/16608#discussion_r96298059
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
---
@@ -300,7 +311,7 @@ abstract class ExplodeBase
GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/16608
[SPARK-13721][SQL] Support outer generators in DataFrame API
## What changes were proposed in this pull request?
Added outer_explode, outer_posexplode, outer_inline functions
66 matches
Mail list logo
