On Thu, 2002-09-05 at 19:06, David Relson wrote: > At 06:50 PM 9/5/02, Ric Tibbetts wrote: > >Ok, it's been a long time since I've needed to do this, and maybe I just > >forgot how. > > > >But: > > > >To sort out duplicate lines in a text file, didn't "sort -u" do the > >trick? Or am I getting fuzzy & old? > > > >I have a large text file that I receive on a regular basis ( a couple > >times a week). I need to clean it up for import into a database. I'm > >stuck on getting duplicate lines out. I swear, I just used to use "sort > >-n", but that isn't working now. > > > >Can someone shed a little light on how to do this? > > > >Thanks in advance > > > > Ric > > "man sort" will show you what the "-n" option does. > > To remove duplicates, you can do "sort <in.file | uniq >out.file" or "sort > -u <in.file >out.file".
Yep, exactly as I remembered it. But -u didn't seem to be working. It turned out to be a "very" minor difference in the lines, in a very large file. The first two places I looked were "man sort", and "info sort". Both said that -u should work. I suspect that it does. The fault may have been mine on this one. Thanks! Ric
Want to buy your Pack or Services from MandrakeSoft? Go to http://www.mandrakestore.com
