On Mon, Aug 12, 2013 at 1:05 AM, Linda Walsh <g...@tlinx.org> wrote: >> When run in --semaphore mode GNU Parallel uses hard links to create a >> counting semaphore. If your filesystem does not support hard links the >> behaviour is undefined - and that is likely what you see. > > NTFS supports hard links. However, file accesses are infinitely slower than > semaphores in memory. > > Ideally, the program could test to see if the system has > semaphores (they work on cygwin and linux and any posix compatible > using the POSIX::RT::Semaphore package.
It has a heavy downside: It seems not to be installed by default on: aix centos debian dragonfly freebsd hpux hurd irix mandriva minix miros netbsd openbsd openindiana qnx raspberrypi redhat scosysv solaris suse tru64 ubuntu ultrix unixware GNU Parallel has a design goal to not require extra packages installed. Installation should even on older systems simply be copy the file 'parallel' to the path. sem does 20 read accesses and 24 write access to do the semaphore using the filesystem (fewer if it is not the only sem running). sem takes 0.3 seconds to run on my machine. It is hardly a speed devil as it is. So comparing it to xargs (or even parallel without --semaphore) will always go against sem. GNU Parallel is designed to work on multiple computers at the same time if $HOME/.parallel is shared between the computers. So sem should block on computerA if computerB has the semaphore (e.g. if they both try to access the same file on an NFS based dir). A semaphore that is only respected by computerA is thus not backwards compatible with the current design. Compared to the other 24 platforms Windows in general has more quirks and has always been supported in GNU Parallel as a nice-to-have platform - not a need-to-have. Not having ssh access to a Windows/Cygwin environment also makes development for that platform harder. Compromising the design goal on at least 24 platforms is not worth it to me to get --semaphore to work better on Windows. If we can find a way that will work fast on Windows/Cygwin without compromising the 1-file-install design goal and the shared $HOME semaphore, I will definitely be interested. /Ole
