Jake Maes created SAMZA-953:
-------------------------------
Summary: User-friendly error when a message with an empty key is
sent to a log compacted Kafka topic
Key: SAMZA-953
URL: https://issues.apache.org/jira/browse/SAMZA-953
Project: Samza
Issue Type: Bug
Reporter: Jake Maes
While verifying the 10.1 release with a test job, I started getting a
CORRUPT_MESSAGE exception from Kafka while flushing the changelog.
Turns out, the job was writing an empty string as a Key in the store. After
serialization, the string translated to an empty byte[], which is not allowed
for log compacted topics.
The error message isn't intuitive for users. Ideally we'd catch the issue in
the KV store, but the key hasn't been serialized at that point. The next best
place to catch it is right after serialization, where we can measure the key
size and throw if sending to a log compacted topic.
We could add this check in the current KafkaSystemProducer, but there is
discussion about moving to the high-level kafka system producer which, I
believe, performs serialization. We should coordinate with the kafka team to
see if they have a more useful error message in the high-level producer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
