Thank you for pointing out to JIRA. On 7/9/14, 6:38 PM, "Jun Rao" <jun...@gmail.com> wrote:
>This is being worked on in >https://issues.apache.org/jira/browse/KAFKA-1325 > >Thanks, > >Jun > > >On Wed, Jul 9, 2014 at 11:42 AM, Virendra Pratap Singh < >vpsi...@yahoo-inc.com.invalid> wrote: > >> Well currently the log rollup is controlled via >> >> log.roll.hours >> Or >> log.segment.bytes >> >> Given that we now have support to log retention in minutes, I guess it >> would be apt to have rollup also have capability to be available in >> minutes. Whom/where should I ask to have that coded in. >> >> One a similar note, I would genuinely want all the size and time based >> parameters to be defined at bytes and ms level. This would make it >>generic >> enough to have user choose and define kind of setting they want and not >> require someone to go and change the code to support a new use case. >> Having hours/minutes in config will just cause proliferation/bloating of >> config parameters. What is a good place to air this suggestion. Anyone? >> >> Virendra >> >> On 7/9/14, 7:21 AM, "Jun Rao" <jun...@gmail.com> wrote: >> >> >Actually, Kafka only removes old segments. The last (active) segment is >> >never removed. So, f you want to have a 10 min retention, you need to >> >configure log rolling such that log segments are rolled at least every >>10 >> >mins. >> > >> >Thanks, >> > >> >Jun >> > >> > >> >On Tue, Jul 8, 2014 at 10:04 PM, Virendra Pratap Singh < >> >vpsi...@yahoo-inc.com.invalid> wrote: >> > >> >> That's correct. The server where in I was running 0.8.1.1 was not >> >>honoring >> >> this parameter, despite the fact it was set in it server.properties. >> >> Not sure if this fact would play any role, the server which was >>running >> >> 0.8.0 was the leader for all the topics and partition in my setup. >>And >> >>the >> >> second server running 0.8.1.1 has all the replicas (follower). >> >> >> >> Virendra >> >> >> >> On 7/8/14, 12:54 PM, "Guozhang Wang" <wangg...@gmail.com> wrote: >> >> >> >> >Server properties should affect on only the local instance >>separately. >> >>Are >> >> >you saying the property is not honored even on the 0.8.1 machines? >> >> > >> >> >Guozhang >> >> > >> >> >On Mon, Jul 7, 2014 at 3:55 PM, Virendra Pratap Singh < >> >> >vpsi...@yahoo-inc.com.invalid> wrote: >> >> > >> >> >> By setting this property >> >> >> log.retention.mins=10 >> >> >> in the server.properties file, which is passed as argument when >> >>starting >> >> >> the broker. >> >> >> >> >> >> Virendra >> >> >> >> >> >> On 7/7/14, 3:31 PM, "Guozhang Wang" <wangg...@gmail.com> wrote: >> >> >> >> >> >> >How do you set the retention.minutes property? Is it through >> >>zk-based >> >> >> >topics tool? >> >> >> > >> >> >> >Guozhang >> >> >> > >> >> >> > >> >> >> >On Mon, Jul 7, 2014 at 3:07 PM, Virendra Pratap Singh < >> >> >> >vpsi...@yahoo-inc.com.invalid> wrote: >> >> >> > >> >> >> >> I am running a mixed cluster as I mentioned earlier. 1 broker >> >>0.8.0 >> >> >>and >> >> >> >> the other 0.8.1.1. Should the retention of topics for >>partitions >> >> >> >> owned/replicated by the broker running 0.8.1.1 not enforce the >> >>server >> >> >> >> properties settings as defined for that server. >> >> >> >> >> >> >> >> So this brings an interesting question, in case of >>heterogeneous >> >> >> >> environment (as is in my case, which system parameters will >>take >> >> >> >> preference/precedence). >> >> >> >> >> >> >> >> Virendra >> >> >> >> >> >> >> >> On 6/30/14, 9:19 AM, "Guozhang Wang" <wangg...@gmail.com> >>wrote: >> >> >> >> >> >> >> >> >The retention.minute property is only introduced in 0.8.1: >> >> >> >> > >> >> >> >> >https://issues.apache.org/jira/browse/KAFKA-918 >> >> >> >> > >> >> >> >> >if you are running 0.8.0 then it will not be recognized. >> >> >> >> > >> >> >> >> >Guozhang >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> >On Fri, Jun 27, 2014 at 2:13 PM, Virendra Pratap Singh < >> >> >> >> >vpsi...@yahoo-inc.com.invalid> wrote: >> >> >> >> > >> >> >> >> >> Running a mixed 2 broker cluster. Mixed as in one of the >> >>broker1 >> >> >>is >> >> >> >> >> running 0.8.0 and broker2 one 0.8.1.1 (from the apache >>release >> >> >>link. >> >> >> >> >> Directly using the tar ball, no local build used). >> >> >> >> >> >> >> >> >> >> I have set the log.retention.minutes=10. However the broker >>is >> >>not >> >> >> >> >> honoring the setting. I see its not cleaning the log.dir at >> >>all. >> >> >> >> >> >> >> >> >> >> However when I set the log.retention.hours=1, then it starts >> >> >>cleaning >> >> >> >> >>the >> >> >> >> >> log. >> >> >> >> >> >> >> >> >> >> When I have the log.retention.minutes set in the >> >>server.properties >> >> >> >>then >> >> >> >> >>I >> >> >> >> >> see this logged in server.log: >> >> >> >> >> >> >> >> >> >> Š.. >> >> >> >> >> [2014-06-27 19:21:06,633] WARN Property >>log.retention.minutes >> >>is >> >> >>not >> >> >> >> >>valid >> >> >> >> >> (kafka.utils.VerifiableProperties) >> >> >> >> >> [2014-06-27 19:21:06,633] WARN Property >>log.retention.minutes >> >>is >> >> >>not >> >> >> >> >>valid >> >> >> >> >> (kafka.utils.VerifiableProperties) >> >> >> >> >> ŠŠ >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> I have set these properties too: >> >> >> >> >> >> >> >> >> >> log.cleaner.enable=true >> >> >> >> >> log.cleanup.policy=delete >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> But I see similar warning logged for these properties too. >> >> >> >> >> >> >> >> >> >> Regards, >> >> >> >> >> Virendra >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> > >> >> >> >> >-- >> >> >> >> >-- Guozhang >> >> >> >> >> >> >> >> >> >> >> > >> >> >> > >> >> >> >-- >> >> >> >-- Guozhang >> >> >> >> >> >> >> >> > >> >> > >> >> >-- >> >> >-- Guozhang >> >> >> >> >> >>
