Yogesh, I think to get desired result you need to group by both cid and date , time. If you don't want to do the group by both then break the original dataset into two and keep the date, time separate from the dataset where you are running the group by and later point join them back bases on cid to get the desired result.
Hope this will work. Thank You, Manish -----Original Message----- From: [email protected] [mailto:[email protected]] Sent: Wednesday, September 26, 2012 11:24 AM To: [email protected] Subject: How to use DISTINCT for this case Hi all, I am not getting the idea to use distinct for this, or how else we can perform this I have a file DATAFILE have content like. Date, time, ssitename, sip, csusername, cip (2012-01-17,00:04:50,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:50,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:51,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:51,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:51,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:51,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:51,W3SVC1970829965,10.100.195.16,-,180.150.248.195) (2012-01-17,00:04:56,W3SVC1970829965,10.100.195.16,-,174.6.225.223) (2012-01-17,00:04:57,W3SVC1970829965,10.100.195.16,-,10.100.195.16) I did group and count operation with filter condition. like A = limit DATAFILE 400; Datagroup = group A by cip; Datacount = foreach Datagroup generate group, A.date, COUNT(A) as number; and got output like this. (66.249.72.75,{(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17)},7) (66.249.72.76,{(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17)},7) (77.88.30.248,{(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17)},5) (92.97.183.58,{(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17)},4) (94.242.214.7,{(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17),(2012-01-17)},5) ............................ ....................... I want it to be as . ( Want to see date appears only at once, not for the number of it repeats as count) (66.249.72.75,{(2012-01-17)},7) (66.249.72.76,{(2012-01-17),7) (77.88.30.248,{(2012-01-17),5) (92.97.183.58,{(2012-01-17),4) (94.242.214.7,{(2012-01-17),5) Please Suggest and Help Thanks & Regards Yogesh Kumar 08147912904 Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
