MaxGekk commented on code in PR #36500:
URL: https://github.com/apache/spark/pull/36500#discussion_r871467515
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala:
##########
@@ -618,6 +618,7 @@ case class FileSourceScanExec(
}.groupBy { f =>
BucketingUtils
.getBucketId(new Path(f.filePath).getName)
+ // TODO(SPARK-39163): Throw an exception w/ error class for an
invalid bucket file
Review Comment:
It captures the exception, actually. That's why I have to change the related
tests. But I do believe we should throw another exception here because this
case is not an illegal state. The bucket can be removed from the file system by
someone else, and Spark shouldn't consider this as its illegal state and show
an internal error to users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
