kbendick opened a new pull request #3268: URL: https://github.com/apache/iceberg/pull/3268
Removes a redundant call to Preconditions.checkState in the Spark 3 MicroBatchStream `shouldProcess` method. Also stores and reuses the result from checking if its an append operation, and places that first in the list of checks, as this is by far the most common case for anybody using the Spark 3 streaming source at present. This method was a bit confusing for me on first read and given that the APPEND check is the most common case, processing it first and storing the results means that the check will likely involve far fewer string comparisons. I was going to allow for also skipping OVERWRITE if users set a flag, but there is already open discussion around this and some older PRs. So closing my original PR in favor of just simplify the check and continuing the discussion elsewhere: https://github.com/apache/iceberg/pull/3267 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
