Jason Gustafson created KAFKA-4254:
--------------------------------------
Summary: Questionable handling of unknown partitions in
KafkaProducer
Key: KAFKA-4254
URL: https://issues.apache.org/jira/browse/KAFKA-4254
Project: Kafka
Issue Type: Bug
Components: producer
Reporter: Jason Gustafson
Assignee: Jason Gustafson
Fix For: 0.10.1.1
Currently the producer will raise an {{IllegalArgumentException}} if the user
attempts to write to a partition which has just been created. This is caused by
the fact that the producer does not attempt to refetch topic metadata in this
case, which means that its check for partition validity is based on stale
metadata.
If the topic for the partition did not already exist, it works fine because the
producer will block until it has metadata for the topic, so this case is
primarily hit when the number of partitions is dynamically increased.
A couple options to fix this that come to mind:
1. We could treat unknown partitions just as we do unknown topics. If the
partition doesn't exist, we refetch metadata and try again (timing out when
max.block.ms is reached).
2. We can at least throw a more specific exception so that users can handle the
error. Raising {{IllegalArgumentException}} is not helpful in practice because
it can also be caused by other error.s
My inclination is to do the first one since the producer seems incorrect to
tell the user that the partition is invalid.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
