RajaVinoth96 commented on PR #19577: URL: https://github.com/apache/pinot/pull/19577#issuecomment-6897221690
@cypherean @tarun11Mavani I've opened #20263, which builds on this PR. It keeps your commit unchanged and adds two commits on top: - **Review fixes**: the CLP temp-dir cleanup, using the nested `try`/`finally` suggested inline; the read-only/empty/closed guard in `MmapMemory.flush()`; and exception-safety fixes in `BitmapInvertedIndexWriter` and the chunk forward-index writers. - **`BenchmarkSegmentCreation`**, with Linux results. In short, typical segments see no measurable slowdown. For a 2.1 GB segment, the new fsyncs add about 5.5 s of waiting per build on a disk that sustains ~450 MB/s, and almost all of that time is spent syncing the v1 temp files that the v3 conversion copies and then deletes. The PR description has the full numbers, links to the workflow runs, and a couple of options (the bitmap resize msync, and syncing temp files versus final files) that I'd welcome your views on. Happy to fold these commits into this PR instead if you'd prefer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
