Just the default... ############################# Log Retention Policy #############################
# The following configurations control the disposal of log segments. The policy can # be set to delete segments after a period of time, or after a given size has accumulated. # A segment will be deleted whenever *either* of these criteria are met. Deletion always happens # from the end of the log. # The minimum age of a log file to be eligible for deletion log.retention.hours=168 # A size-based retention policy for logs. Segments are pruned from the log as long as the remaining # segments don't drop below log.retention.bytes. #log.retention.bytes=1073741824 # The maximum size of a log segment file. When this size is reached a new log segment will be created. log.segment.bytes=536870912 # The interval at which log segments are checked to see if they can be deleted according # to the retention policies log.retention.check.interval.ms=60000 # By default the log cleaner is disabled and the log retention policy will default to just delete segments after their retention expires. # If log.cleaner.enable=true is set the cleaner will be enabled and individual logs can then be marked for log compaction. log.cleaner.enable=false On Wed, Sep 3, 2014 at 1:26 PM, Harsha <[email protected]> wrote: > Hi Nick, > Whats your log.retention set on kafka. It might be that kafka is > deleting your data before KafkaSpout is able to consume. > -Harsha > > > On Wed, Sep 3, 2014, at 10:01 AM, Nick Beenham wrote: > > We have started to see a lot of these errors within the logs, and the > tuples being emitted but not transferred from the spout to the bolt. > > > Any ideas? > > > 2014-09-03 16:18:28 s.k.KafkaUtils [WARN] Got fetch request with offset > out of range: [9551]; retrying with default start offset time from > configuration. configured start offset time: [-2] offset: [0] > > 2014-09-03 16:18:28 s.k.KafkaUtils [WARN] Got fetch request with offset > out of range: [616248]; retrying with default start offset time from > configuration. configured start offset time: [-2] offset: [0] > > > Thanks, > > > Nick > > >
