Hi Users , We have planned to use Kafka for one of the use to collect data from different server and persist into Message Bus ..
Flow Would Be : Source --> Kafka --> Streaming Engine --> Reports We like to store different types of data in the same topic , same time data should be accessed easily .. here is sample data : {"code" :"100" , "type" : "a" , "data" : " hello"} {"code" :"100" , "type" : "b" , "data" : " hello"} {"code" :"100" , "type" : "c" , "data" : " hello"} This case we want to create topic called : *topic_100* and store all data but the access pattern is using type. Example : 1) Read only *Type : "a"* data There is an Option to Partition the data using type so that all types goes to same partition. The problem is the data is not distributed across cluster. What is the preferred approach to Use Same Topic but different types of data ? --Senthil