Hi Tom,

Thanks for adding this performance test results.
So basically, for acks=1 case, it's what we expected.

I have some comments about it, I'll reply in the discussion thread in
KIP-1176.

Thank you.
Luke

On Thu, Aug 28, 2025 at 4:53 AM Thomas Thornton
<tthorn...@salesforce.com.invalid> wrote:

> Hi Luke,
>
> Thanks for creating this discussion on these KIPs.
>
> I'm collaborating with Henry on KIP-1176. We have deployed the Kafka fork
> to one of our Kafka clusters. We collected performance data (full results
> here
> <
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=354454090#KIP1176:TieredStorageforActiveLogSegment-Appendix:PerformanceData
> >)
> for the acks=1 case. Overall we see comparable performance (throughput &
> latency) for the producer path between a standard topic and one with
> KIP-1176 code. Consumer latency is slightly higher as data must travel an
> additional hop to/from S3E1Z. There are additional tunings we are aware of
> that could further boost performance (e.g., tuning the interval that the
> leader/follower remote WAL tasks read/write data to/from cloud storage).
>
> Are there any other benchmarks the community would find useful? Any
> questions on these findings?
>
> Thanks,
> Tom
>
> On Tue, Aug 5, 2025 at 1:31 AM Luke Chen <show...@gmail.com> wrote:
>
> > Hi all,
> >
> > The Kafka community is currently seeing an unprecedented situation with
> > three KIPs (KIP-1150, IP-1176, KIP-1183) simultaneously addressing the
> same
> > challenge of high replication costs when running Kafka across multiple
> > cloud availability zones. Each KIP offers a different solution to this
> > issue. While diversity of innovative ideas is a key strength of
> open-source
> > projects, it creates a burden for reviewers and users who must compare
> and
> > comment on multiple proposals simultaneously. Furthermore, discussion
> > around the three KIPs has stalled for over two months now. This could be
> > due to the authors being hesitant to proceed due to the existence of
> > alternative, potentially conflicting, solutions. Addressing replication
> > cost is a key concern of Kafka’s userbase and we should try to move the
> > conversation forward if we can.
> >
> > From what I understand, these three KIPs are not mutually exclusive. But
> > adopting all three KIPs in the community might not be what we expect.
> Thus,
> > I would like to *start a discussion on how we could move the conversation
> > forward*.
> >
> > To save time for the KIP readers/reviewers, I have created this document
> > <
> >
> https://cwiki.apache.org/confluence/display/KAFKA/The+Path+Forward+for+Saving+Cross-AZ+Replication+Costs+KIPs
> > >[1]
> > to help summarize each of the KIPs and describe their current status.
> *Hope
> > to get some suggestions/feedback from the community*.
> >
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/The+Path+Forward+for+Saving+Cross-AZ+Replication+Costs+KIPs
> >
> > KIP-1150:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1150%3A+Diskless+Topics
> > KIP-1176
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1150%3A+Diskless+TopicsKIP-1176
> >
> > :
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1176%3A+Tiered+Storage+for+Active+Log+Segment
> > KIP-1183
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1176%3A+Tiered+Storage+for+Active+Log+SegmentKIP-1183
> >
> > :
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1183%3A+Unified+Shared+Storage
> >
> >
> > Thank you.
> > Luke
> >
>

Reply via email to