Having the OS cache the data in Kafka's log files is useful since it means that data doesn't need to be read back from disk when consumed. This is good for the latency and throughput of consumers. Usually this caching works out pretty well, keeping the latest data from your topics in cache and only pulling older data into memory if a consumer reads data from earlier in the log. In other words, by leveraging OS-level caching of files, Kafka gets an in-memory caching layer for free.
Generally you shouldn't need to clear this data -- the OS should only be using memory that isn't being used anyway. Is there a particular problem you're encountering that clearing the cache would help with? -Ewen On Mon, Jul 27, 2015 at 2:33 AM, Nilesh Chhapru < nilesh.chha...@ugamsolutions.com> wrote: > Hi All, > > I am facing issues with kafka broker process taking a lot of cache > memory, just wanted to know if the process really need that much of > cache memory, or can i clear the OS level cache by setting a cron. > > Regards, > Nilesh Chhapru. > -- Thanks, Ewen
