kz930 opened a new issue, #8977: URL: https://github.com/apache/texera/issues/8977
### What happened? Filter's exported Python compares the column to the value with pandas operators, while `FilterPredicate` compares by type: a STRING field and the value are compared as numbers when both parse as one, a BOOLEAN as its lower-case text, and a DOUBLE with `Double.compareTo`. Run against `SpecializedFilterOpExec` on main, all of these differ: - STRING column `["1", "2", "10", "abc"]`, `= 1`: the engine keeps "1", the export keeps nothing; `> 2`: the engine keeps "10" and "abc", the export raises `TypeError`. - BOOLEAN column, `= True`: the engine keeps true, the export keeps nothing; `= 1`: the reverse. - DOUBLE column with NaN and -0.0: `Double.compareTo` ranks NaN above every number and equal to itself, and -0.0 below 0.0, so `> 0`, `< 0`, `= 0` and `!= 0` all keep different rows. A value of `NaN` or `Infinity` makes the export raise `NameError`. ### How to reproduce? Filter a STRING column holding "1", "2", "10" and "abc" with `= 1`. The engine returns one row, the exported script returns none. ### Version/Branch 1.4.0-incubating-SNAPSHOT (main) ### Commit Hash (Optional) 6a29c59c6 ### What browsers are you seeing the problem on? _No response_ ### Relevant log output _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
