andygrove commented on code in PR #255:
URL:
https://github.com/apache/arrow-datafusion-comet/pull/255#discussion_r1570663482
##########
spark/src/main/scala/org/apache/comet/CometSparkSessionExtensions.scala:
##########
@@ -91,6 +98,30 @@ class CometSparkSessionExtensions
scanExec.copy(scan = cometScan),
runtimeFilters = scanExec.runtimeFilters)
+ // unsupported parquet data source V2
+ case scanExec: BatchScanExec if
scanExec.scan.isInstanceOf[ParquetScan] =>
+ val requiredSchema =
scanExec.scan.asInstanceOf[ParquetScan].readDataSchema
+ var info1: Option[String] = None
+ if (isSchemaSupported(requiredSchema)) {
+ info1 = Some(s"Schema $requiredSchema is not supported")
+ }
Review Comment:
I wonder if we could add a helper method to reduce the boilerplate for each
of these sections for creating the `Option[String]`.
We could add a method like this:
```scala
def createMessage(condition: Boolean, message: => String): Option[String] = {
if (condition) {
Some(message)
} else {
None
}
}
```
The call site could then be a one-liner:
```scala
val info1 = createMessage(isSchemaSupported(requiredSchema), s"Schema
$requiredSchema is not supported")
```
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
