@hzxa21 Good point. In general I agree it is easier to ensure correctness by always getting the read lock when reading from the file. I think the main concern here is degraded performance. Currently log read operation is essentially lock free and thus does not interfere with log write operation. If we use the read/write lock for every access to the log, then log read operation and log write operation will block each other. My intuition is that this will considerably reduce the the Kafka cluster's read/write throughput.
This will also considerably increase the ProduceRequest latency, e.g. `LogCleaner.cleanSegments()` will have to grab the per-log read lock as it goes through each segment in the log, which is going to take tens of segments for large log. And ProduceRequest will have to wait for lock during this period, which is probably not good. What do you think? [ Full content available at: https://github.com/apache/kafka/pull/5591 ] This message was relayed via gitbox.apache.org for [email protected]
