sidharthdaga commented on PR #22585: URL: https://github.com/apache/kafka/pull/22585#issuecomment-5447290460
Thanks for the review. I checked where the time goes, and it is not the mmap. The time indexes in this benchmark are 12-byte single-entry files, not pre-allocated 10MB ones, so there is no 10MB mapping or 2560-page read in the delta. Materializing and checking a real TimeIndex costs about the same for a 12-byte and a 10MB index, roughly 110 and 125 microseconds warm, because the map is independent of length and sanityCheck reads only the first and last entry. The cost is opening the file; the rest of the per-segment number is the surrounding UnifiedLog load. So the zero probe would not help. It still opens the file, which is the cost, and there is no whole-file read to shrink. Adding it to the offset index only doubles the opens, and the offset index is excluded because a pre-allocated one does not fail the roll. The filter that helps is the file length, since the corruption only exists at the full pre-allocated size and a trimmed index is smaller. I pushed this: sanityCheck now materializes only when File.length matches the pre-allocated size, so a healthy segment is a stat with no open, about 14 microseconds against the 110 to 125 to materialize, and only a pre-allocated-size index is opened and checked. I updated the description to match, so it no longer frames this as index I/O. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
