Github user DaimonPl commented on the issue:
https://github.com/apache/spark/pull/22357
@mallman regarding
"But how do we know that pushing down IsNotNull(employer) does not negate
that instruction? "
isn't it pretty obvious that when you read 'employer.id' with predicate
'employer.id = 0' then you can safely skip all records where 'employer' is
null? If 'employer' is null then 'employer.id' is null as well so it would
never satisfy provided predicate
Of course I agree that pushing down for 'employer.id' might be even better
(if it's possible)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
