If you're trying to get around the maximum argument limit, maybe this will help: https://www.gnu.org/software/parallel/man.html#example-inserting-multiple-arguments Or this: https://www.gnu.org/software/parallel/man.html#example-processing-a-big-file-using-more-cpus
Parallel can break up the calls to a command, limiting the number of arguments to the maximum allowed on the command line. But then you won't have one sorted file in the end. Try the examples, use "-m -j1" or "-X -j1", and do a final `sort -u` on the output file. -- Joe On Tue, Nov 30, 2021 at 3:25 AM Hongyi Zhao <hongyi.z...@gmail.com> wrote: > > As far as the following command is concerned: > > $ find ./source -type f | xargs sort -o bbb -u > > Lew Pitcher told me that it has the following limitation [1]: > > The above command depends on xargs(1) to provide sort(1) with a list > of input files. As the size of this list depends on both the number of > files found, /and/ the maximum size of an argument vector (argv[]), there > is a chance that the file bbb will only contain the sorted contents of > a subset of the files found by find(1). > > [1] https://groups.google.com/g/comp.unix.shell/c/ha5t3U54GmY/m/M3tIUPtPBAAJ > > So, I want to know whether the parallel tool still has the similar problem. > > Regards, > HZ >