mbutrovich commented on issue #2611: URL: https://github.com/apache/datafusion-comet/issues/2611#issuecomment-3427123109
I think it would be great to separate testing concerns a bit. For this issue in particular, I think testing expressions should be somewhat independent of testing scans. When I implemented `decode` and `regexp_replace` I wanted to test a lot of types and corner case values, so I stuck it in `CometFuzzTestSuite` but it didn't really belong there. I think We should break that out to a generic DataFrame generator and push more tests to use the InMemoryTableScan and SparkToColumnar logic and exercise them directly. This would save test time (and SSD write cycles) for all of the Parquet files our tests generate. It also makes the data generator more agnostic for things like Iceberg support. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
