peter-toth commented on PR #59916: URL: https://github.com/apache/spark/pull/59916#issuecomment-6604349298
@dongjoon-hyun, thanks again! Round 2 is addressed in [a3b03fe](https://github.com/apache/spark/pull/59916/commits/a3b03fe2a9fa6d4073e138c844271597c2d9c6bd): - 1 to 3: for a key whose DPP filters are all dropped, the rule adds no bloom filter if the creation side reads a CTE or keeps an `onlyInBroadcast` DPP filter that counts as kept, or if it cannot follow the key into a union branch. Scratch runs of your three queries at 2420b70 lost rows (1) and failed with `CAST_INVALID_INPUT` (2 and 3). Each is a test now. Item 3 also happens on master for a key without a DPP filter, which is SPARK-60842. - 4: a numeric up-cast of an attribute, and the normalization of a floating point key, count as safe keys now. - 13: the description adds a run with the application-side threshold at 0. - The rest is answered in the threads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
