diegomrsantos opened a new issue, #4177: URL: https://github.com/apache/iggy/issues/4177
Part of #4174. Affected areas: Iggy server, Performance. Measure how the owner serves disk poll completions when several reads and ordinary work are active. The purpose is to identify any admission or service bottleneck before changing completion capacity or drain policy. In [router.rs](https://github.com/apache/iggy/blob/8f1c0e260076f9fbe49539e904e43895bc24c3fe/core/shard/src/router.rs), the owner services one completion after ordinary work and also has a completion select arm. [completion.rs](https://github.com/apache/iggy/blob/8f1c0e260076f9fbe49539e904e43895bc24c3fe/core/shard/src/poll/completion.rs) reserves capacity before disk I/O and retains it until dequeue or discard. The [configured limit](https://github.com/apache/iggy/blob/8f1c0e260076f9fbe49539e904e43895bc24c3fe/core/configs/src/server_config/sharding.rs) counts running reads plus queued completions, including reads that continue after their requester times out. It counts operations, not retained bytes. A bounded return path does not guarantee a short wait behind a long owner operation. An earlier diagnostic used base `0d862e36dcc42fd1fffca2f4504886718bf7757f` with an instrumented port of the archived lane. This was not the current reserved completion implementation. At configured capacity 1024 it observed: | Observation | Without writes | With writes | | --- | ---: | ---: | | Maximum ordinary inbox occupancy sampled at pump starts | 1 | 2 | | Maximum candidate reservation occupancy | 1 | 1 | | Maximum candidate completion queue occupancy | 1 | 1 | | Rejected candidate reservations | 0 | 0 | These were eight diagnostic captures of 10,000 completed polls each, using one shard, one sequential TCP consumer, explicit offsets, one message with a 256 byte payload per poll and automatic commits disabled. The optional producer targeted 50,000,000 payload bytes per second in batches of 100. Captures ran in Docker Desktop's ARM64 Linux VM on an M1 Pro. Pump samples can miss brief occupancy peaks. These captures neither demonstrate saturation nor establish a benefit from greater capacity, and their instrumented timings are not a clean performance comparison. The latest aggregate campaign linked from #4174 also used a sequential consumer, so it does not close this concurrency gap. The older diagnostic report and raw captures remain archived locally. The investigation should: 1. Reuse the parent's initial stage capture. If owner service appears material, or a representative concurrency requirement warrants it, run a short ramp that exercises several simultaneous reads and ordinary work. 2. Record offered and completed load, active reads, reserved capacity, queued completions, admission rejections, retained bytes, write progress and tick delay. 3. Separate read completion → enqueue → owner acceptance → reply resumption. A ready reply is not evidence that its caller has resumed. Correlate waiting with actual owner work and service intervals. 4. Use independent request arrivals when that matches the intended service claim. A sequential consumer reduces its offered load when the server slows, so it cannot represent that arrival model. 5. If a specific service interval or capacity boundary explains material delay, compare one bounded policy change at matched load. Keep clean timing separate from diagnostics and report completion latency, achieved write throughput and tick latency as guardrails. Larger drains can delay consensus, other replies or shutdown. Greater capacity can admit more detached I/O and retain more data. Preserve bounded admission, reservation release, history validation and progress acceptance on the owner. Moving acceptance to a worker is not an equivalent optimization. Complete this issue with a workload and capacity characterization plus a supported decision. An observation that the existing policy is adequate is a valid outcome. Coordinate stage timestamps with the tail investigation under #4174 rather than starting an unrelated tracing campaign. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
