felipepessoto commented on code in PR #13131:
URL: https://github.com/apache/gluten/pull/13131#discussion_r4320739457
##########
.github/workflows/util/delta-spark-ut/apply-delta-test-patches.sh:
##########
@@ -82,6 +82,34 @@ cherry_pick_delta_fix
46bd45d57eadd7e528002a0ae7bd36ce5a456eca "#7104 (ScanRepor
cherry_pick_delta_fix 959e00e15f41f56afc1c9bb95d160c55c6dc7068 "#7105 (9 more
test suites)"
echo "::endgroup::"
+echo "::group::Using Spark's writer for the Hadoop-only mock S3 fixture"
+# This test maps s3: paths to local files through a Hadoop FileSystem mock.
+# Velox's native writer cannot use that mock. Only prepare the fixture with
+# Spark's writer; keep Gluten enabled for the conversion and path-scheme check.
+if ! sed 's/^__BLANK_CONTEXT__$/ /' <<'PATCH' | git -C "$DELTA_DIR" apply -
+diff --git
a/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
b/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
+---
a/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
++++
b/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
+@@ -1322,7 +1322,9 @@
+ withTempDir { dir =>
+ withTable("externalTable") {
+ withSQLConf(("fs.s3.impl",
classOf[S3LikeLocalFileSystem].getCanonicalName)) {
+- sql(s"CREATE TABLE externalTable USING parquet LOCATION 's3://$dir'
AS SELECT 1")
++ withSQLConf("spark.gluten.sql.native.writer.enabled" -> "false") {
Review Comment:
The scoped statement creates a Parquet table (`CREATE TABLE ... USING
parquet ... AS SELECT 1`), not a Delta table.
`spark.gluten.sql.native.writer.enabled` is the flag read by
`VeloxBackend.enableNativeWriteFiles()`, and
`Validators.FallbackByBackendSettings` rejects native `WriteFilesExec` when it
is false. The Delta-specific key would not disable this preparatory Parquet
write. The subsequent Delta conversion and path-scheme assertion remain outside
the override, so the existing flag is correct here.
##########
.github/workflows/util/delta-spark-ut/apply-delta-test-patches.sh:
##########
@@ -82,6 +82,34 @@ cherry_pick_delta_fix
46bd45d57eadd7e528002a0ae7bd36ce5a456eca "#7104 (ScanRepor
cherry_pick_delta_fix 959e00e15f41f56afc1c9bb95d160c55c6dc7068 "#7105 (9 more
test suites)"
echo "::endgroup::"
+echo "::group::Using Spark's writer for the Hadoop-only mock S3 fixture"
+# This test maps s3: paths to local files through a Hadoop FileSystem mock.
+# Velox's native writer cannot use that mock. Only prepare the fixture with
+# Spark's writer; keep Gluten enabled for the conversion and path-scheme check.
+if ! sed 's/^__BLANK_CONTEXT__$/ /' <<'PATCH' | git -C "$DELTA_DIR" apply -
+diff --git
a/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
b/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
+---
a/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
++++
b/spark/src/test/scala/org/apache/spark/sql/delta/ConvertToDeltaSuiteBase.scala
+@@ -1322,7 +1322,9 @@
+ withTempDir { dir =>
+ withTable("externalTable") {
+ withSQLConf(("fs.s3.impl",
classOf[S3LikeLocalFileSystem].getCanonicalName)) {
+- sql(s"CREATE TABLE externalTable USING parquet LOCATION 's3://$dir'
AS SELECT 1")
++ withSQLConf("spark.gluten.sql.native.writer.enabled" -> "false") {
++ sql(s"CREATE TABLE externalTable USING parquet LOCATION
's3://$dir' AS SELECT 1")
++ }
+__BLANK_CONTEXT__
Review Comment:
The single space is the unified-diff context prefix for an unchanged empty
line. `git apply` consumes that prefix; it does not require a space in the
Scala source. I checked the exact `sed` output against Delta v4.4.1 with `git
apply --check --whitespace=error-all` and applied it successfully; the
resulting source line is empty. Deleting the context line outright without
adjusting the hunk counts instead produces `corrupt patch`. Keeping the current
substitution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
