That's a doozy! Quoting can be a bit tricky with parallel. I couldn't figure it out after a few minutes of playing with it either. You may just want to throw your awk program into a file and have parallel run that.
On Thu, Sep 8, 2011 at 11:34 AM, LU Zen <[email protected]> wrote: > I’m trying to process a big messy csv file with the --pipe option but I keep > getting errors such as unexpected EOF while looking for matching `"' or awk: > cmd. line:1: ^ unexpected newline or end of string. I suspect I’m not using > the quoting correctly. 2 of the commands I’ve tried: > > > > $ cat big.csv | parallel --pipe --files 'awk -v FS="\",\"" '{print $1, $3, > $4, $5, $9, $14}' | grep -v "#" | sed -e '1d' -e 's/\"//g' -e > 's/\/\/\//\t/g' | cut -f1-6,11 | sed -e 's/\/\//\t/g' -e 's/ /\t/g'' | > parallel -Xj1 sort -k1 {} ';' rm {} > big_modified_parallel.csv > > > > $ cat big.csv | parallel --pipe --files 'awk -v FS="\",\"" '{OFS="\\t\"\; > print $1, $3, $4, $5, $9, $14}' | grep -v "#" | sed -e '1d' -e 's/\"//g' -e > 's/\/\/\//\t/g' | cut -f1-6,11 | sed -e 's/\/\//\t/g' -e 's/ /\t/g'' | > parallel -Xj1 sort -k1 {} ';' rm {} > big_modified_parallel.csv > > > > Can someone point out where I might have made a mistake? > > > > Thank you. > > > > > > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > >
