Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19404
BTW, perf wise: hflush() is required to block until the flush has got to
the store (visible to others), and with hsync actually saved to the durable
store. So it will take time, but if you want durability, that's that price.
Without that hsync call though, there's no guarantee anything will be written
to the store. If you need this log to recover from failures: hsync is what you
are going to need, even though it currently only works on HDFS and WASB.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
