This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from cd39cd4 [SPARK-28560][SQL][FOLLOWUP] support the build side to local
shuffle reader as far as possible in BroadcastHashJoin
add faf220a [SPARK-29277][SQL][test-hadoop3.2] Add early DSv2 filter and
projection pushdown
No new revisions were added by this update.
Summary of changes:
.../spark/sql/catalyst/analysis/Analyzer.scala | 14 ++-
.../sql/catalyst/analysis/CheckAnalysis.scala | 14 +++
.../spark/sql/catalyst/optimizer/Optimizer.scala | 14 ++-
.../sql/catalyst/plans/logical/v2Commands.scala | 6 +-
.../sql/connector/catalog/CatalogV2Util.scala | 7 +-
.../datasources/v2/DataSourceV2Relation.scala | 52 +++++++--
.../main/scala/org/apache/spark/sql/Dataset.scala | 4 +-
.../spark/sql/execution/SparkOptimizer.scala | 12 +-
.../datasources/v2/DataSourceV2Strategy.scala | 123 +++------------------
.../execution/datasources/v2/PushDownUtils.scala | 103 +++++++++++++++++
.../datasources/v2/V2ScanRelationPushDown.scala | 66 +++++++++++
.../spark/sql/FileBasedDataSourceSuite.scala | 10 +-
.../spark/sql/connector/DataSourceV2Suite.scala | 8 +-
.../sql/execution/datasources/orc/OrcTest.scala | 11 +-
.../datasources/orc/OrcV2SchemaPruningSuite.scala | 2 +-
.../datasources/parquet/ParquetFilterSuite.scala | 11 +-
.../execution/datasources/orc/OrcFilterSuite.scala | 10 +-
.../execution/datasources/orc/OrcFilterSuite.scala | 9 +-
18 files changed, 308 insertions(+), 168 deletions(-)
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownUtils.scala
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
