This is an automated email from the ASF dual-hosted git repository.
hongze pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
from 7a036dd6e [GLUTEN-5939][CH] Support java timezone id named 'GMT+8' or
'GMT+08:00' (#5940)
add ad817ed51 [GLUTEN-5414] [VL] Support arrow csv option and schema
(#5850)
No new revisions were added by this update.
Summary of changes:
.github/workflows/velox_docker.yml | 71 +-
.github/workflows/velox_docker_cache.yml | 8 +-
.../gluten/datasource/ArrowCSVFileFormat.scala | 111 ++-
.../datasource/ArrowCSVOptionConverter.scala | 62 ++
.../gluten/datasource/ArrowConvertorRule.scala | 12 +-
.../v2/ArrowCSVPartitionReaderFactory.scala | 79 +-
.../gluten/datasource/v2/ArrowCSVTable.scala | 12 +
.../gluten/extension/ArrowScanReplaceRule.scala | 3 +-
.../resources/datasource/csv/student_option.csv | 4 +
.../datasource/csv/student_option_schema.csv | 4 +
.../csv/{student.csv => student_option_str.csv} | 2 +-
.../org/apache/gluten/execution/TestOperator.scala | 235 ++++--
ep/build-velox/src/build_velox.sh | 10 +-
ep/build-velox/src/get_velox.sh | 2 +
.../src/modify_arrow_dataset_scan_option.patch | 883 +++++++++++++++++++++
ep/build-velox/src/modify_velox.patch | 3 +-
.../execution/WholeStageTransformerSuite.scala | 20 +
gluten-data/pom.xml | 4 +-
.../scala/org/apache/gluten/utils/ArrowUtil.scala | 147 +---
gluten-ut/spark32/pom.xml | 2 +-
.../gluten/utils/velox/VeloxTestSettings.scala | 9 +
gluten-ut/spark33/pom.xml | 2 +-
.../gluten/utils/velox/VeloxTestSettings.scala | 9 +
gluten-ut/spark34/pom.xml | 2 +-
.../gluten/utils/velox/VeloxTestSettings.scala | 9 +
gluten-ut/spark35/pom.xml | 2 +-
.../gluten/utils/velox/VeloxTestSettings.scala | 11 +
.../execution/datasources/csv/GlutenCSVSuite.scala | 65 +-
gluten-ut/test/pom.xml | 2 +-
pom.xml | 1 +
30 files changed, 1472 insertions(+), 314 deletions(-)
create mode 100644
backends-velox/src/main/scala/org/apache/gluten/datasource/ArrowCSVOptionConverter.scala
create mode 100644
backends-velox/src/test/resources/datasource/csv/student_option.csv
create mode 100644
backends-velox/src/test/resources/datasource/csv/student_option_schema.csv
copy backends-velox/src/test/resources/datasource/csv/{student.csv =>
student_option_str.csv} (78%)
create mode 100644 ep/build-velox/src/modify_arrow_dataset_scan_option.patch
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
