PHILO-HE commented on code in PR #6672:
URL: https://github.com/apache/incubator-gluten/pull/6672#discussion_r1708822806
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxBackend.scala:
##########
@@ -73,7 +74,14 @@ object VeloxBackendSettings extends BackendSettingsApi {
format: ReadFileFormat,
fields: Array[StructField],
partTable: Boolean,
+ rootPaths: Seq[String],
paths: Seq[String]): ValidationResult = {
+ if (
+ !rootPaths.isEmpty &&
!VeloxFileSystemValidationJniWrapper.supportedPaths(rootPaths.toArray)
Review Comment:
@zhli1142015, I note Spark accepts data from two or more kinds of file
systems, e.g.,
`df = spark.read.format("parquet").load("file:///path/to/local/data",
"s3a://bucket/path/to/s3/data")`.
Not sure whether we need to also consider other files other than the first
one, which definitely brings some cost. Does your production env. have such
usage? If it is rarely used, we can just put some comments to clarify. cc
@wForget
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
