Hi Martin --
Martin Guy wrote:
2007/8/21, David Liontooth <[EMAIL PROTECTED]>:
Is it possible to use transcode in cluster mode on a beowulf cluster?
Does anyone here have experience with this?
Yes it is, I did this the other week.
All right, I lucked out!
There are two mechanisms supported: PVM and -W ("cluster mode").
There are two documents in the distribution about these two modes.
I found PVM support to be broken in transcode.
It always gives some message about "Unable to send" or something
similar, and a google for the same message says other people find the
same thing.
I ran cluster mode (-W) a coupla weeks ago on 14 nodes of varying speeds.
-W seems to be designed to spread the work between N otherwise idle
nodes of the same speed, whereas mine range from 400MHz to 1.7GHz
share a common home filesystem and all have the same OS installation
(Debian with custom transcode)
What was the effect when you ran transcode in cluster mode on nodes of
different speeds and loads?
I wrote a program "distribute" to spread the work that is sort of
like a generalised distcc. It reads a list of command lines (60 of
them in my example) and sends one to each node via rsh (ie ssh), then
when a node finished its task its is given another one, so on until
all are done.
It's all pretty hacky and error-prone at present but you're welcome
to try with it. It reduced my encoding time from many hours to 29
minutes.
Thanks -- much appreciated!
The cluster I'm using is a Redhat 8.0 with few or no updates since
2003. I'm not the sysadmin, and don't expect they'll want to do a lot
up upgrades.
The gcc is ancient:
$ gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --host=i386-redhat-linux --with-system-zlib
--enable-__cxa_atexit
Thread model: posix
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)
Is compiling transcode under gcc 3.2 worth doing, or should I install a
local version in my user?
I'd like to use it for x264, let me know if you've used this codec
within transcode or separately.
Dave
