uros-db commented on PR #47620: URL: https://github.com/apache/spark/pull/47620#issuecomment-2278017099
@cloud-fan, @MaxGekk and I have discussed this offline, so here's a quick rundown: over the past few months, we've had many untested and mishandled cases for collation support in various expressions - so to avoid things like this, I propose that we should stay on the conservative side of testing with collations (at least for now) given that this suite takes ~10s total to run on my machine locally, I think this is not too bad and that benefits outweight the extra computation load - to recap, `CollationStringExpressionsSuite.scala` currently handles two types of tests: - Spark expression-level unit tests, that are used to test both code paths (interpreted and genCode) - e2e SQL tests with literals, that are used to conveniently verify only the basic test case results the other stuff, such as: - e2e SQL tests with explicit `collate` expressions - e2e SQL tests that verify collation type coercion & collation mismatches - e2e SQL tests with table columns instead of literals will be moved to the golden file `collations.sql` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
