Github user bersprockets commented on the issue:
https://github.com/apache/spark/pull/21144
@hvanhovell @maropu As it turns out, there are at least two places where an
InterpretedPredicate is created but never initialized:
SimpleTextSource.buildReader, and ExternalCatalogUtils.prunePartitionsByFilter.
It seems unlikely that nondeterministic expressions would be used for
partitions, but maybe I am not imaginative enough. Also, these seem like bugs
in those two classes, not in InterpretedPredicate.
Also, I reran the mostly-interpreted SQL unit tests, this time with
SortPrefix implemented, and got the error count down from 1270 to 114. None of
the errors were 'requirement failed' exceptions (so no uninitialized
nondeterministic expressions).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
