hubcio commented on PR #4092: URL: https://github.com/apache/iggy/pull/4092#issuecomment-5621040296
@diegomrsantos your comment is valid, let me can explain,it grew in 3 steps. first we fixed the producer ACK. ACKs didn't always wait for recoverable storage, so we added the `durability` and `consumer_offset_durability` topic options. they're part of the `TopicOptions` API, which is why every SDK changed too. Then config changes: no more fsync settings, and `system.*` got flattened along the way. when we ran it on AWS EC2 i4i, persisted perf wasn't what we expected. a 3-node cluster missed an 800 MB/s target and p999 sat around XX ms. after that we added 40b6e6d08 (persisted bodies no longer go to both the wal and the segment) and 20ee88498 (drops an extra memset per produce request). p999 latency dropped to single digit. the PR description is behind on all this, I'll fix it today. we want to deliver it as is to not prolong the release due to github actions being constantly overloaded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
