philo-he commented on code in PR #12279:
URL: https://github.com/apache/gluten/pull/12279#discussion_r3399295427
##########
gluten-ut/spark33/src/test/scala/org/apache/spark/sql/gluten/GlutenFallbackSuite.scala:
##########
@@ -59,6 +60,22 @@ class GlutenFallbackSuite extends GlutenSQLTestsTrait with
AdaptiveSparkPlanHelp
}
}
+ testGluten("fail job on fallback when failOnFallback is enabled") {
+ withTable("t") {
+ spark.range(10).write.format("parquet").saveAsTable("t")
+ withSQLConf(GlutenConfig.COLUMNAR_FILESCAN_ENABLED.key -> "false") {
+ // Default failOnFallback=false: same fallback should not throw.
+ sql("SELECT * FROM t").collect()
+ withSQLConf(GlutenConfig.FALLBACK_FAIL_ON_FALLBACK.key -> "true") {
+ val ex = intercept[GlutenException] {
+ sql("SELECT * FROM t").collect()
+ }
+
assert(ex.getMessage.contains(GlutenConfig.FALLBACK_FAIL_ON_FALLBACK.key))
+ }
+ }
+ }
+ }
Review Comment:
Since this test is independent of any Spark version, can we move it to the
gluten-ut/test module—perhaps as a new test suite—so we only need to maintain a
single copy of the test code?
##########
gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala:
##########
@@ -1430,6 +1432,14 @@ object GlutenConfig extends ConfigRegistry {
.booleanConf
.createWithDefault(true)
+ val FALLBACK_FAIL_ON_FALLBACK =
+ buildConf("spark.gluten.sql.columnar.failOnFallback")
+ .doc(
+ "When true, throw an exception if any operator falls back to Spark" +
+ " instead of running on the native engine.")
+ .booleanConf
Review Comment:
Maybe add `.internal()` to mark this config as not intended for end users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
