aokolnychyi commented on code in PR #6077:
URL: https://github.com/apache/iceberg/pull/6077#discussion_r1008755739
##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkCopyOnWriteScan.java:
##########
@@ -102,6 +103,14 @@ public NamedReference[] filterAttributes() {
@Override
public void filter(Filter[] filters) {
+ Preconditions.checkState(
Review Comment:
I initially thought about not doing runtime filtering if the table has been
refreshed to avoid failing the query. However, we change such a critical place.
I think it is better to let the user know rather than proceeding with the
query. Otherwise, jobs that leverage multiple threads may take much more
resources. If runtime filtering stops working, jobs will simply start taking
tons of time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
