This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from be4d825 [SPARK-30312][SQL][FOLLOWUP] Rename conf by adding `.enabled`
add 4e50f02 [SPARK-30323][SQL] Support filters pushdown in CSV datasource
No new revisions were added by this update.
Summary of changes:
.../apache/spark/sql/catalyst/csv/CSVFilters.scala | 191 +++++++++++++++++++++
.../spark/sql/catalyst/csv/UnivocityParser.scala | 58 +++++--
.../sql/catalyst/expressions/csvExpressions.scala | 2 +-
.../org/apache/spark/sql/internal/SQLConf.scala | 7 +
.../spark/sql/catalyst/csv/CSVFiltersSuite.scala | 134 +++++++++++++++
.../sql/catalyst/csv/UnivocityParserSuite.scala | 50 ++++++
sql/core/benchmarks/CSVBenchmark-jdk11-results.txt | 8 +
sql/core/benchmarks/CSVBenchmark-results.txt | 8 +
.../org/apache/spark/sql/DataFrameReader.scala | 2 +-
.../execution/datasources/csv/CSVFileFormat.scala | 11 +-
.../v2/csv/CSVPartitionReaderFactory.scala | 7 +-
.../sql/execution/datasources/v2/csv/CSVScan.scala | 12 +-
.../datasources/v2/csv/CSVScanBuilder.scala | 26 ++-
.../execution/datasources/csv/CSVBenchmark.scala | 46 +++++
.../sql/execution/datasources/csv/CSVSuite.scala | 77 ++++++++-
15 files changed, 608 insertions(+), 31 deletions(-)
create mode 100644
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVFilters.scala
create mode 100644
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/csv/CSVFiltersSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
