Ricky Saltzer created FLUME-2499:
------------------------------------
Summary: Include Kafka Message Key in Event Header, Updated
Comments
Key: FLUME-2499
URL: https://issues.apache.org/jira/browse/FLUME-2499
Project: Flume
Issue Type: Improvement
Components: Sinks+Sources
Reporter: Ricky Saltzer
Priority: Minor
I believe it would be useful to include a message's key in the event header, as
users will often want to partition based on this key when ingesting into a sink.
*Example:*
{code}
sfdc.sinks.hdfs.hdfs.path =
/data/testing/kafka_flume/dataset=%{key}/year=%Y/month=%m/day=%d/
{code}
In this case, the first part of the partition key is the Kafka message key,
which will ensure messages are organized by key.
I also updated some of the comments and cleaned up the code slightly.
Cheers,
Ricky
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
