[
https://issues.apache.org/jira/browse/KAFKA-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16798792#comment-16798792
]
Spyridon Ninos commented on KAFKA-3539:
---------------------------------------
Hi [[email protected]],
indeed the producer does not comply with the Future contract, however I'm not
sure that this should be considered critical because it violates the contract
only when the system has issues (i.e. the producer cannot communicate with the
cluster).
Should we modify the producer to strictly comply with the Future contract under
any and all circumstances, we may hit a practical wall when it comes to code
complexity. One rough analysis showed that in order for the user thread not to
block, we should at least remove the partitioner from the user thread and push
the functionality somewhere between the user thread and the sender thread (or,
worse, the sender thread itself). Unblocking the user thread when the sender
thread cannot send the messages to kafka, means that the messages should be
stored somewhere locally (i.e. producer side) until the sender thread can push
them to the cluster. This, in turn, changes the memory requirements of the
producer, and increases the risk to lose more messages than the amount you can
lose with the current implementation, in case the producer dies unexpectedly.
I understand the concern people have about the contract - the question is,
though, do we really need the Future contract at all? What are your thoughts on
it?
As a side note, my comment targeted mainly the status of the ticket; since no
work is done currently, having it at the "in progress" status is misleading.
Thanks
> KafkaProducer.send() may block even though it returns the Future
> ----------------------------------------------------------------
>
> Key: KAFKA-3539
> URL: https://issues.apache.org/jira/browse/KAFKA-3539
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Reporter: Oleg Zhurakousky
> Priority: Critical
>
> You can get more details from the [email protected] by searching on the
> thread with the subject "KafkaProducer block on send".
> The bottom line is that method that returns Future must never block, since it
> essentially violates the Future contract as it was specifically designed to
> return immediately passing control back to the user to check for completion,
> cancel etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
