Thank you!
On Mon, Jul 27, 2015 at 1:43 PM, Ewen Cheslack-Postava e...@confluent.io
wrote:
As I mentioned, adjusting any settings such that files are small enough
that you don't get the benefits of append-only writes or file
creation/deletion become a bottleneck might affect performance. It
If I want to get higher throughput, should I increase the
log.segment.bytes?
I don't see log.retention.check.interval.ms, but there is
log.cleanup.interval.mins, is that what you mean?
If I set log.roll.ms or log.cleanup.interval.mins too small, will it hurt
the throughput? Thanks.
On Fri, Jul
As I mentioned, adjusting any settings such that files are small enough
that you don't get the benefits of append-only writes or file
creation/deletion become a bottleneck might affect performance. It looks
like the default setting for log.segment.bytes is 1GB, so given fast enough
cleanup of old
Thank you! what performance impacts will it be if I change
log.segment.bytes? Thanks.
On Mon, Jul 27, 2015 at 1:25 PM, Ewen Cheslack-Postava e...@confluent.io
wrote:
I think log.cleanup.interval.mins was removed in the first 0.8 release. It
sounds like you're looking at outdated docs. Search
I think log.cleanup.interval.mins was removed in the first 0.8 release. It
sounds like you're looking at outdated docs. Search for
log.retention.check.interval.ms here:
http://kafka.apache.org/documentation.html
As for setting the values too low hurting performance, I'd guess it's
probably only
Hi,
I am testing the kafka producer performance. So I created a queue and
writes a large amount of data to that queue.
Is there a way to delete the data automatically after some time, say
whenever the data size reaches 50GB or the retention time exceeds 10
seconds, it will be deleted so my disk
You can configure that in the Configs by setting log retention :
http://kafka.apache.org/07/configuration.html
Thanks,
Mayuresh
Sent from my iPhone
On Jul 24, 2015, at 12:49 PM, Yuheng Du yuheng.du.h...@gmail.com wrote:
Hi,
I am testing the kafka producer performance. So I created a
You'll want to set the log retention policy via
log.retention.{ms,minutes,hours} or log.retention.bytes. If you want really
aggressive collection (e.g., on the order of seconds, as you specified),
you might also need to adjust log.segment.bytes/log.roll.{ms,hours} and
