Jason Gustafson resolved KAFKA-4254.
       Resolution: Fixed
    Fix Version/s:     (was:

Issue resolved by pull request 1995

> Questionable handling of unknown partitions in KafkaProducer
> ------------------------------------------------------------
>                 Key: KAFKA-4254
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4254
>             Project: Kafka
>          Issue Type: Bug
>          Components: producer 
>            Reporter: Jason Gustafson
>            Assignee: Konstantine Karantasis
>             Fix For:
> Currently the producer will raise an {{IllegalArgumentException}} if the user 
> attempts to write to a partition which has just been created. This is caused 
> by the fact that the producer does not attempt to refetch topic metadata in 
> this case, which means that its check for partition validity is based on 
> stale metadata.
> If the topic for the partition did not already exist, it works fine because 
> the producer will block until it has metadata for the topic, so this case is 
> primarily hit when the number of partitions is dynamically increased. 
> A couple options to fix this that come to mind:
> 1. We could treat unknown partitions just as we do unknown topics. If the 
> partition doesn't exist, we refetch metadata and try again (timing out when 
> max.block.ms is reached).
> 2. We can at least throw a more specific exception so that users can handle 
> the error. Raising {{IllegalArgumentException}} is not helpful in practice 
> because it can also be caused by other error.s
> My inclination is to do the first one since the producer seems incorrect to 
> tell the user that the partition is invalid.

This message was sent by Atlassian JIRA

Reply via email to