On Tue, May 11, 2010 at 08:25:43AM +0000, [email protected] wrote: > >>To be more specific, I need to find how many distinct records are there in > >>say column#1? > > awk '{print $1}' filename | sort -u | wc -l > > This will show how many unique entries are present in column one (use awk -F > to change delimiter e.g awk -F ":" for : delimiter) > > >> How can I filter out the distinct records with number of occurances less > >> than a pre-determined threshold? > > I don't quite understand this part. > > awk '{print $1}' filename | sort | uniq -c | sort -rn > > Will give you a number of occurrences (reverse numerically sorted) of uniq > data from column one. > > Now I think you want to put that through a loop and only show those that are > less than threshold?
If I understand correctly, you can pipe your output to: `awk '{a=$1} {if
(a > 3) print a}''. `a' is awk variable. `$1' is first column of awk
input so you probably need to change it.
--
Dominik Zyla
pgp8kTia5mitm.pgp
Description: PGP signature
_______________________________________________ CentOS mailing list [email protected] http://lists.centos.org/mailman/listinfo/centos
