This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.
from 817d1d7 [SPARK-37769][SQL][FOLLOWUP] Filtering files if metadata
columns are present in the data filter
add 3a45981 [SPARK-37896][SQL] Implement a ConstantColumnVector and
improve performance of the hidden file metadata
No new revisions were added by this update.
Summary of changes:
.../execution/vectorized/ConstantColumnVector.java | 292 +++++++++++++++++++++
.../spark/sql/execution/DataSourceScanExec.scala | 6 +-
.../sql/execution/datasources/FileScanRDD.scala | 45 ++--
.../vectorized/ConstantColumnVectorSuite.scala | 205 +++++++++++++++
4 files changed, 515 insertions(+), 33 deletions(-)
create mode 100644
sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ConstantColumnVector.java
create mode 100644
sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ConstantColumnVectorSuite.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
