This is more of a general design question but what is the preferred way of importing logs from Kafka to HDFS when you want your data segmented by hour or day? Is there anyway to say "Import only this {hour|day} of logs" or does one need to create their topics around the way they would like to import them.. ie Topic: "search_logs/2011/11/06". If its the latter, is there any documentation/best practices on topic/key design?

Thanks

Reply via email to