Hi Eric, I am using chukwa 0.4 . i have ceated a bug http://portals.zones.apache.org:8080/webcontent/rproxy/secure/issues_apache/jira/browse/CHUKWA-580
________________________________ From: Eric Yang [[email protected]] Sent: Saturday, January 15, 2011 3:01 AM To: [email protected] Subject: Re: Checkpoints increasing without sending data to collector This looks like a regression bug on the CharFileTailingAdaptorUTF8 adaptor. Which version of chukwa are you using? Please open a jira, and we will look into the cause. Thanks Regards, Eric On 1/14/11 7:43 AM, "Stuti Awasthi" <[email protected]<https://puneexchange.persistent.co.in/owa/UrlBlockedError.aspx>> wrote: Hi all, I have a query regarding the checkpoints in chukwa. According to theory : Every few minutes, each agent process polls a collector to find the length of each file to which data is being written. The length of the file is then compared with the offset at which each chunk was to be written. If the file length exceeds this value, then the data has been committed and the agent process advances its checkpoint accordingly.(Note that the length returned by the filesystem is the amount of data that has been successfully replicated.) This means that chukwa_agent_checkpoint would increase only when the agent receivers and ack from the collectors. But in case of dirtailing adapter, this is not correct. I have done the following steps to test this : * Started agent with some dummy collector which was not present. * Added dirtailing adapter with Charfile tailing adapter I can see the following output in my checkpoint file : ADD adaptor_67653208e8dea46c798e46753fc19dad = org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8 Stuti 0 /root/Stuti/yum.log 0 ADD adaptor_b505db62647203ffa3cfe17374042870 = org.apache.hadoop.chukwa.datacollection.adaptor.DirTailingAdaptor Stuti /root/Stuti filetailer.CharFileTailingAdaptorUTF8 1295014173306 Since data is not getting sent to collector, so checkpoints should not increase. Please Suggest Stuti Awasthi DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
