[
https://issues.apache.org/jira/browse/KAFKA-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256890#comment-14256890
]
Joel Koshy commented on KAFKA-1755:
-----------------------------------
Thanks for the comments. The issue with 3a is that once we do have compression
support for compacted topics it will be very ugly to implement that check on
message arrival. This is because we need to do a deep iteration on incoming
messages to look at the key field. The only time we do that currently on the
broker is when assigning offsets. However, this code is in ByteBufferMessageSet
which is fairly low-level and has no log-config information associated with it.
We would have to "leak" some flag indicating whether non-keyed messages are
allowed or not which is ugly. That is why I prefer not doing that check on
message arrival and just have the log cleaner drop/ignore non-keyed messages
with a warning. Ultimately, the effect is the same. However, the benefit of
rejecting is that the producer is made aware of it. So I guess I changed my
mind with regard to my earlier comment - i.e., I would recommend against doing
this unless we can think of an elegant implementation. 3b is easy to do and we
can implement that until KAFKA-1374 is in place.
> Improve error handling in log cleaner
> -------------------------------------
>
> Key: KAFKA-1755
> URL: https://issues.apache.org/jira/browse/KAFKA-1755
> Project: Kafka
> Issue Type: Bug
> Reporter: Joel Koshy
> Assignee: Joel Koshy
> Labels: newbie++
> Fix For: 0.8.3
>
>
> The log cleaner is a critical process when using compacted topics.
> However, if there is any error in any topic (notably if a key is missing)
> then the cleaner exits and all other compacted topics will also be adversely
> affected - i.e., compaction stops across the board.
> This can be improved by just aborting compaction for a topic on any error and
> keep the thread from exiting.
> Another improvement would be to reject messages without keys that are sent to
> compacted topics although this is not enough by itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
