All,
I am very new to kafka . We have a case where we need to ingest multiple
avro record types . These avro record types vary vastly in volume and size
and I am thinking of sending each of these message types to a different
topic and creating partitions based on volume and through put needed. What
the kafka consumer has to do is read the record of from partitions and
write to different hdfs locations based on record type . I am guessing
we should at least start with one consumer per topic . Is this
understanding correct or is there a better way to look at it?
