dcoliversun commented on issue #9300: URL: https://github.com/apache/incubator-gluten/issues/9300#issuecomment-2803592099
@marin-ma I would like to discuss the implementation plan with you. After reviewing the code in Spark and Gluten, I found that it is not feasible to enumerate replacement expressions for the following two reasons: 1. Currently, Spark does not provide an API to enumerate expressions, so we cannot filter out the desired expressions by checking if they have a `replacement` method. 2. Some expressions' replacements are not other expressions but rather objects implemented by Spark, such as `urlencode`[1]. My solution is to maintain a map in `expressionMap` to record the replacement expressions. I believe this approach will not introduce significant maintenance overhead. The main task would be to check whether new expressions inherit from `RuntimeReplaceable` when updating Spark to a supported version. Please let me know your thoughts or if you have any alternative suggestions. Thank you! [1] https://github.com/apache/spark/blob/898a0b46a0ab40bf201307a5d25f6028775740f2/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/urlExpressions.scala#L57-L63 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
