hubcio opened a new pull request, #4314: URL: https://github.com/apache/iggy/pull/4314
Warmup ran at full speed and left each consumer where it stopped. A paced run started with a warmup burst far above the measured rate. A consumer that had to read all the data never finished, because the warmup had already read part of it. Warmup now runs under the limiter of the measured phase, and the consumer kinds that time latency locally rewind every partition after it. The reset deletes the stored offset, because next() starts at stored + 1, so a stored zero skips the first message. The high-level client also rebuilds its consumer, because the SDK refuses messages it already delivered. The origin-timestamp kinds keep their position, because re-read warmup messages count the whole warmup as latency. The producing consumer charged sent plus received bytes and waited only after a receive, so a produce-only actor ran unthrottled. It now charges only what it sends, and it waits when it can send again. The wait never sits between a send and the poll that reads the batch back, where it adds to the recorded latency. In these kinds, --rate-limit now caps sent bytes only. Pacing the warmup needed the limiter fixed first. Quota::per_second truncates the refill period to whole nanoseconds, so 350 MB/s ran 43% fast and rates past 1 GB/s clamped to 1 GB/s. Its burst was a full second of quota, and a rate below one batch per second panicked on the unwrap. The quota now counts cells, about a million per second, with a 100 ms burst, and the limiter takes a larger charge in successive waits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
