Github user adrian-ionescu commented on the issue:
https://github.com/apache/spark/pull/21289
I just took a quick look... Do I understand correctly that your flag
basically controls whether the refresh is eager (`true`, current behavior) vs
lazy (`false`)? If so, let's rename the flag accordingly.
Also, with lazy mode we just delay file listing from immediately after an
append operation until the next Spark job involving that table, right? It
doesn't seem like we gain much overall..
What's the general use-case that motivated you to make this change? Please
explain that more clearly in the PR description. Also, let's add a test that
verifies the intended behavior.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
