[
https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-4254:
-----------------------------------
Comment: was deleted
(was: [~ijuma] I'm sympathetic to your concern, but it seems a little difficult
to justify the current behavior of raising an exception based on stale
metadata. As a user, I expect that I can produce to an existing partition even
if it has just been created, so requiring me to catch exceptions until the
producer's metadata refresh interval expires seems a bit clunky. Of course, the
producer doesn't necessarily need to block the full {{max.block.ms}} until the
partition is created, maybe one metadata refresh prior to raising would be
sufficient?)
> Questionable handling of unknown partitions in KafkaProducer
> ------------------------------------------------------------
>
> Key: KAFKA-4254
> URL: https://issues.apache.org/jira/browse/KAFKA-4254
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Fix For: 0.10.1.1
>
>
> Currently the producer will raise an {{IllegalArgumentException}} if the user
> attempts to write to a partition which has just been created. This is caused
> by the fact that the producer does not attempt to refetch topic metadata in
> this case, which means that its check for partition validity is based on
> stale metadata.
> If the topic for the partition did not already exist, it works fine because
> the producer will block until it has metadata for the topic, so this case is
> primarily hit when the number of partitions is dynamically increased.
> A couple options to fix this that come to mind:
> 1. We could treat unknown partitions just as we do unknown topics. If the
> partition doesn't exist, we refetch metadata and try again (timing out when
> max.block.ms is reached).
> 2. We can at least throw a more specific exception so that users can handle
> the error. Raising {{IllegalArgumentException}} is not helpful in practice
> because it can also be caused by other error.s
> My inclination is to do the first one since the producer seems incorrect to
> tell the user that the partition is invalid.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
