cloud-fan commented on PR #43751: URL: https://github.com/apache/spark/pull/43751#issuecomment-1968294284
This is a hard decision. Technically the behavior of LIKE in many commands (`SHOW TABLES LIKE ...`) relies on the underlying catalog, which can be HMS of different versions, or a Hive-compatible metastore service. This is out of Spark's control. From @panbingkun's [investigation](https://github.com/apache/spark/pull/43751#issuecomment-1953619718), the Hive behavior is actually very weird and different from other main-stream SQL systems (they follow the same behavior of the LIKE expression). Hive 4.0 also switches to the more common behavior. There are some commands that we implement the LIKE filtering by our own, following the Hive behavior. Now we are in a hard position: 1. If we do nothing, then Spark's behavior of LIKE in commands is non-standard and different from other databases. We may also hit future behavior changes if we upgrade to Hive 4.0. 2. If we change the LIKE filtering behavior now, it's a breaking change, and also lead to inconsistent behaviors as some commands use Hive to do LIKE filtering. cc @srielau -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
