On Wed, Oct 5, 2022 at 7:55 AM Nagle, Michael F
<[email protected]> wrote:
>
> Correct. I have a few lines of bash code to automatically prepare the jobs
> file I just sent. Immediately after that file is made, it is passed as the
> $job_list_name variable to...
>
> parallel --results output -a $job_list_name
>
> Please let me know if any other information would be helpful, and thanks for
> all the suggestions and thought into this...
I have the feeling your problem is that the filesystem you use has a
restriction on the length of a dir:
mkdir
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456
fails on ext4 and tmpfs.
The \_ is (as you guessed) by design to keep the value in a single
dir, so that GNU Parallel can assume a structure like:
parallel --header : --results foo echo {a} {b} \
::: a I II ::: b III IIII
foo/a/II/b/III/seq
foo/a/II/b/III/stderr
foo/a/II/b/III/stdout
foo/a/II/b/IIII/seq
foo/a/II/b/IIII/stderr
foo/a/II/b/IIII/stdout
foo/a/I/b/III/seq
foo/a/I/b/III/stderr
foo/a/I/b/III/stdout
foo/a/I/b/IIII/seq
foo/a/I/b/IIII/stderr
foo/a/I/b/IIII/stdout
What you can do is to use a replacement string to shorten your value:
parallel --results output/'{=s/gemma.*pheno_files//=}' -a $job_list_name
Or you could use GNU Parallel to generate the commands:
doit() {
file="$1"
outname=$(echo $file | sed -e "s/\\.noheader\\.pheno//g")-ResidOverPhase
outname=$(basename $outname)
gemma -bfile ./1323_cohort_maf01_geno10.snp.pass \
-p ./pheno_files/$file \
-k ./1323_cohort_maf01_geno10.cXX.txt \
-lmm 1 \
-n 3 \
-miss 0.1 \
-o "$outname"
}
export -f doit
ls ./pheno_files | grep -i "noheader" |
parallel --results output/ doit
/Ole
