alamb opened a new issue, #10919: URL: https://github.com/apache/datafusion/issues/10919
### Is your feature request related to a problem or challenge? Part of https://github.com/apache/datafusion/issues/10918, `[StringViewArray`](https://docs.rs/arrow/latest/arrow/array/type.StringViewArray.html) support in DataFusion There are several queries in the [clickbench](https://github.com/apache/datafusion/blob/11b7b5c215012231e5768fc5be3445c0254d0169/benchmarks/queries/clickbench/queries.sql#L11-L15) suite like follows: ```sql SELECT "MobilePhone", "MobilePhoneModel", COUNT(DISTINCT "UserID") AS u FROM hits WHERE "MobilePhoneModel" <> '' GROUP BY "MobilePhone", "MobilePhoneModel" ORDER BY u DESC LIMIT 10; SELECT "SearchPhrase", COUNT(*) AS c FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY c DESC LIMIT 10; SELECT "SearchPhrase", COUNT(DISTINCT "UserID") AS u FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchPhrase" ORDER BY u DESC LIMIT 10; SELECT "SearchEngineID", "SearchPhrase", COUNT(*) AS c FROM hits WHERE "SearchPhrase" <> '' GROUP BY "SearchEngineID", "SearchPhrase" ORDER BY c DESC LIMIT 10; ``` where `"MobilePhoneModel"` and `"SearchPhrase"` are string columns with predicates (in this case checking for empty string) ### Describe the solution you'd like In order to improve performance of these queries we will need the ability to actually compare `StringViewArrays` to constant strings (and likely to each other) Thus I would like to be able to run `StringViewColumn = scalar` `StringViewColumn = StringViewColumn` (and likewise for BinaryView) I basically want to to run the following queries (where table `foo` has `StringView` columns) ```sql > create table foo as values ('Andrew', 'X'), ('Xiangpeng', 'Xiangpeng'), ('Raphael', 'R'); 0 row(s) fetched. Elapsed 0.002 seconds. > select * from foo where column1 = 'Andrew'; +---------+---------+ | column1 | column2 | +---------+---------+ | Andrew | X | +---------+---------+ 1 row(s) fetched. Elapsed 0.003 seconds. > select * from foo where column1 <> 'Andrew'; +-----------+-----------+ | column1 | column2 | +-----------+-----------+ | Xiangpeng | Xiangpeng | | Raphael | R | +-----------+-----------+ 2 row(s) fetched. Elapsed 0.001 seconds. > select * from foo where column1 = column2; +-----------+-----------+ | column1 | column2 | +-----------+-----------+ | Xiangpeng | Xiangpeng | +-----------+-----------+ 1 row(s) fetched. Elapsed 0.002 seconds. > select * from foo where column1 <> column2; +---------+---------+ | column1 | column2 | +---------+---------+ | Andrew | X | | Raphael | R | +---------+---------+ 2 row(s) fetched. Elapsed 0.001 seconds. ``` ### Describe alternatives you've considered I suspect we will need to update the coercion logic and maybe also the arrow equality kernels like https://docs.rs/arrow/latest/arrow/compute/kernels/cmp/fn.eq.html ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
