Hi Fred and Greg, Thanks for your feedback and it really not straightforward but interesting! There are some behavior I expect.
The current producer uses the *RecordAccumulator* to gather records, and the sender thread sends them in batches. We can track each record’s acknowledgment setting as it appends to the *RecordAccumulator*, allowing the *sender *to group batches by acknowledgment levels and topicPartition when processing. Regarding the statement, "Callbacks for records being sent to the same partition are guaranteed to execute in order," this is ensured when *max.inflight.request *is set to 1. We can send records with different acknowledgment levels in the order of acks-0, acks=1, acks=-1. Since we need to send batches with different acknowledgment levels batches to the broker, the callback will execute after each request is completed. In response to, "If so, are low-acks records subject to head-of-line blocking from high-acks records?," I believe an additional configuration is necessary to control this behavior. We could allow records to be either sync or async, though the callback would still execute after each batch with varying acknowledgment levels completes. To measure behavior across acknowledgment levels, we could also include acks in *ProducerIntercepor*. Furthermore, before this KIP, a producer could only include one acks level so sequence is premised. However, with this change, we can *ONLY* guarantee the sequence within records of the same acknowledgment level because we may send up to three separate requests to brokers. Best, TaiJuWu TaiJu Wu <tjwu1...@gmail.com> 於 2024年11月6日 週三 上午10:01寫道： > Hi Fred and Greg, > > Apologies for the delayed response. > Yes, you’re correct. > I’ll outline the behavior I expect. > > Thanks for your feedback! > > Best, > TaiJuWu > > > Greg Harris <greg.har...@aiven.io.invalid> 於 2024年11月6日 週三 上午9:48寫道： > >> Hi TaiJuWu, >> >> Thanks for the KIP! >> >> Can you explain in the KIP about the behavior when the number of acks is >> different for individual records? I think the current description using >> the >> word "straightforward" does little to explain that, and may actually be >> hiding some complexity. >> >> For example, the send() javadoc contains this: "Callbacks for records >> being >> sent to the same partition are guaranteed to execute in order." Is this >> still true when acks vary for records within the same partition? >> If so, are low-acks records subject to head-of-line-blocking from >> high-acks >> records? It seems to me that this feature is useful when acks is specified >> per-topic, but introduces a lot of edge cases that are underspecified. >> >> Thanks, >> Greg >> >> >> On Tue, Nov 5, 2024 at 4:52 PM TaiJu Wu <tjwu1...@gmail.com> wrote: >> >> > Hi Chia-Ping, >> > >> > Thanks for your feedback. >> > I have updated KIP based on your suggestions. >> > >> > Best, >> > Stanley >> > >> > Chia-Ping Tsai <chia7...@apache.org> 於 2024年11月5日 週二 下午4:41寫道： >> > >> > > hi TaiJuWu, >> > > >> > > Q0: Could you please add getter (Short acks()) to "public interface" >> > > section? >> > > >> > > Q1: Could you please add RPC json reference to prove "been available >> at >> > > the RPC-level," >> > > >> > > Q2: Could you please add link to producer docs to prove "share a >> single >> > > producer instance across multiple threads" >> > > >> > > Thanks, >> > > Chia-Ping >> > > >> > > On 2024/11/05 01:28:36 吳岱儒 wrote: >> > > > Hi all, >> > > > >> > > > I open a KIP-1107: Adding record-level acks for producers >> > > > < >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1107%3A++Adding+record-level+acks+for+producers >> > > > >> > > > to >> > > > reduce the limitation associated with reusing KafkaProducer. >> > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1107%3A++Adding+record-level+acks+for+producers >> > > > >> > > > Feedbacks and suggestions are welcome. >> > > > >> > > > Thanks, >> > > > TaiJuWu >> > > > >> > > >> > >> >
