andygrove commented on code in PR #412:
URL: https://github.com/apache/datafusion-comet/pull/412#discussion_r1597065878
##########
spark/src/main/spark-3.x/org/apache/comet/shims/ShimCometScanExec.scala:
##########
@@ -69,6 +69,8 @@ trait ShimCometScanExec {
readSchema: StructType,
options: ParquetOptions): FileScanRDD =
classOf[FileScanRDD].getDeclaredConstructors
+ // workaround for aws spark 3.4 implementation
+ .filter(c => List(3,5,6).contains(c.getParameterCount()) )
Review Comment:
I'm not sure that we should be adding anything specific to closed-source
Spark versions without first considering the implications for the project.
I filed https://github.com/apache/datafusion-comet/issues/414 so that we can
start that discussion.
This change has no unit test, and I assume that it would be difficult to add
a unit test without adding a dependency on the AWS Spark jars, which are maybe
not even available except for when we are building on the AWS platform.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
