i am playing around with "log.file.size"(controls the size of a segment file in a partition) and "log.retention.hours" with the following config. log.file.size=500 log.retention.hours=168
Observation - i see multiple files getting generated within the same partition. Example : my topic name is revenue feed and i see the following ls -lh /tmp/kafka-logs/revenuefeed-0/* -rw-r--r-- 1 inder users 537 Oct 25 01:38 /tmp/kafka-logs/revenuefeed-0/00000000000000000000.kafka -rw-r--r-- 1 inder users 512 Oct 25 01:39 /tmp/kafka-logs/revenuefeed-0/00000000000000000537.kafka Questions -------------- 1. Shouldn't these two properties go hand in hand 2. Why would you want to have multiple files within a partition. Broker has to store more info to figure the right file among a partition. 3. Is it to achieve mmap kinda optimization and allowing the broker to do less I/O in case a feed is really huge or any thing else. -- Inder
