Re: [OMPI devel] failures runing mpi4py testsuite, perhaps Comm.Split()
On 7/11/07, George Bosilca wrote: The two errors you provide are quite different. The first one has been addresses few days ago in the trunk (https://svn.open-mpi.org/ trac/ompi/changeset/15291). If instead of the 1.2.3 you use anything after r15291 you will be safe in a threading case. Please, take into account that in this case I not used MPI_Init_tread() ... In any case, sorry for making noise if this was already reported. I have other issues to report, but perhaps I should try the svn version. Please, understand me, I am really busy with many things as to be up-to-date with every source code I use. Sorry again. The second is different. The problem is that memcpy is a lot faster than memmove, and that's why we use it. Yes, of course. The case where the 2 data overlap are quite minimal. I'll take a look to see exactly what happened there. Initially, I though it was my error, but next realized that this seems to happen in Comm.Split() internals. -- Lisandro Dalcín --- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594
Re: [OMPI devel] failures runing mpi4py testsuite, perhaps Comm.Split()
Lisandro, The two errors you provide are quite different. The first one has been addresses few days ago in the trunk (https://svn.open-mpi.org/ trac/ompi/changeset/15291). If instead of the 1.2.3 you use anything after r15291 you will be safe in a threading case. The second is different. The problem is that memcpy is a lot faster than memmove, and that's why we use it. The case where the 2 data overlap are quite minimal. I'll take a look to see exactly what happened there. george. On Jul 11, 2007, at 8:08 PM, Lisandro Dalcin wrote: Ups, sended to wrong list, forwarded here... -- Forwarded message -- From: Lisandro Dalcin Date: Jul 11, 2007 8:58 PM Subject: failures runing mpi4py testsuite, perhaps Comm.Split() To: Open MPI Hello all, after a long time I'm here again. I am improving mpi4py in order to support MPI threads, and I've found some problem with latest version 1.2.3 I've configured with: $ ./configure --prefix /usr/local/openmpi/1.2.3 --enable-mpi-threads --disable-dependency-tracking However, for the following fail, MPI_Init_thread() was not used. This test creates a intercommunicator by using Comm.Split() followed by Intracomm.Create_intercomm(). When running in two or more procs (for one proc this test is skipped), I got (sometimes) the following trace [trantor:06601] *** Process received signal *** [trantor:06601] Signal: Segmentation fault (11) [trantor:06601] Signal code: Address not mapped (1) [trantor:06601] Failing at address: 0xa8 [trantor:06601] [ 0] [0x958440] [trantor:06601] [ 1] /usr/local/openmpi/1.2.3/lib/openmpi/mca_btl_sm.so (mca_btl_sm_component_progress+0x1483) [0x995553] [trantor:06601] [ 2] /usr/local/openmpi/1.2.3/lib/openmpi/mca_bml_r2.so (mca_bml_r2_progress+0x36) [0x645d06] [trantor:06601] [ 3] /usr/local/openmpi/1.2.3/lib/libopen-pal.so.0(opal_progress+0x58) [0x1a2c88] [trantor:06601] [ 4] /usr/local/openmpi/1.2.3/lib/libmpi.so.0(ompi_request_wait_all+0xea) [0x140a8a] [trantor:06601] [ 5] /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so (ompi_coll_tuned_sendrecv_actual+0xc8) [0x22d6e8] [trantor:06601] [ 6] /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so (ompi_coll_tuned_allgather_intra_bruck+0xf2) [0x231ca2] [trantor:06601] [ 7] /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so (ompi_coll_tuned_allgather_intra_dec_fixed+0x8b) [0x22db7b] [trantor:06601] [ 8] /usr/local/openmpi/1.2.3/lib/libmpi.so.0(ompi_comm_split+0x9d) [0x12d92d] [trantor:06601] [ 9] /usr/local/openmpi/1.2.3/lib/libmpi.so.0(MPI_Comm_split+0xad) [0x15a53d] [trantor:06601] [10] /u/dalcinl/lib/python/mpi4py/_mpi.so [0x508500] [trantor:06601] [11] /usr/local/lib/libpython2.5.so.1.0(PyCFunction_Call+0x14d) [0xe150ad] [trantor:06601] [12] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x64af) [0xe626bf] [trantor:06601] [13] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [14] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x5a43) [0xe61c53] [trantor:06601] [15] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x6130) [0xe62340] [trantor:06601] [16] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [17] /usr/local/lib/libpython2.5.so.1.0 [0xe01450] [trantor:06601] [18] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [19] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x42eb) [0xe604fb] [trantor:06601] [20] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [21] /usr/local/lib/libpython2.5.so.1.0 [0xe0137a] [trantor:06601] [22] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [23] /usr/local/lib/libpython2.5.so.1.0 [0xde6de5] [trantor:06601] [24] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [25] /usr/local/lib/libpython2.5.so.1.0 [0xe2abc9] [trantor:06601] [26] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [27] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x1481) [0xe5d691] [trantor:06601] [28] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [29] /usr/local/lib/libpython2.5.so.1.0 [0xe01450] [trantor:06601] *** End of error message *** As the problem seems to originate in Comm.Split(), I've written a small python script to test it:: from mpi4py import MPI # true MPI_COMM_WORLD_HANDLE BASECOMM = MPI.__COMM_WORLD__ BASE_SIZE = BASECOMM.Get_size() BASE_RANK = BASECOMM.Get_rank() if BASE_RANK < (BASE_SIZE // 2) : COLOR = 0 else: COLOR = 1 INTRACOMM = BASECOMM.Split(COLOR, key=0) print 'Done!!!' This seems always work, but running it under valgrind (note valgrind-py below is just an alias adding a suppression file for python) I get the following: mpiexec -n 3 valgrind-py python test.py =6727== Warning: set address range perms: large range 134217728 (defined) ==6727== Source and destination overlap in memc
[OMPI devel] failures runing mpi4py testsuite, perhaps Comm.Split()
Ups, sended to wrong list, forwarded here... -- Forwarded message -- From: Lisandro Dalcin List-Post: devel@lists.open-mpi.org Date: Jul 11, 2007 8:58 PM Subject: failures runing mpi4py testsuite, perhaps Comm.Split() To: Open MPI Hello all, after a long time I'm here again. I am improving mpi4py in order to support MPI threads, and I've found some problem with latest version 1.2.3 I've configured with: $ ./configure --prefix /usr/local/openmpi/1.2.3 --enable-mpi-threads --disable-dependency-tracking However, for the following fail, MPI_Init_thread() was not used. This test creates a intercommunicator by using Comm.Split() followed by Intracomm.Create_intercomm(). When running in two or more procs (for one proc this test is skipped), I got (sometimes) the following trace [trantor:06601] *** Process received signal *** [trantor:06601] Signal: Segmentation fault (11) [trantor:06601] Signal code: Address not mapped (1) [trantor:06601] Failing at address: 0xa8 [trantor:06601] [ 0] [0x958440] [trantor:06601] [ 1] /usr/local/openmpi/1.2.3/lib/openmpi/mca_btl_sm.so(mca_btl_sm_component_progress+0x1483) [0x995553] [trantor:06601] [ 2] /usr/local/openmpi/1.2.3/lib/openmpi/mca_bml_r2.so(mca_bml_r2_progress+0x36) [0x645d06] [trantor:06601] [ 3] /usr/local/openmpi/1.2.3/lib/libopen-pal.so.0(opal_progress+0x58) [0x1a2c88] [trantor:06601] [ 4] /usr/local/openmpi/1.2.3/lib/libmpi.so.0(ompi_request_wait_all+0xea) [0x140a8a] [trantor:06601] [ 5] /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so(ompi_coll_tuned_sendrecv_actual+0xc8) [0x22d6e8] [trantor:06601] [ 6] /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so(ompi_coll_tuned_allgather_intra_bruck+0xf2) [0x231ca2] [trantor:06601] [ 7] /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so(ompi_coll_tuned_allgather_intra_dec_fixed+0x8b) [0x22db7b] [trantor:06601] [ 8] /usr/local/openmpi/1.2.3/lib/libmpi.so.0(ompi_comm_split+0x9d) [0x12d92d] [trantor:06601] [ 9] /usr/local/openmpi/1.2.3/lib/libmpi.so.0(MPI_Comm_split+0xad) [0x15a53d] [trantor:06601] [10] /u/dalcinl/lib/python/mpi4py/_mpi.so [0x508500] [trantor:06601] [11] /usr/local/lib/libpython2.5.so.1.0(PyCFunction_Call+0x14d) [0xe150ad] [trantor:06601] [12] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x64af) [0xe626bf] [trantor:06601] [13] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [14] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x5a43) [0xe61c53] [trantor:06601] [15] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x6130) [0xe62340] [trantor:06601] [16] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [17] /usr/local/lib/libpython2.5.so.1.0 [0xe01450] [trantor:06601] [18] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [19] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x42eb) [0xe604fb] [trantor:06601] [20] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [21] /usr/local/lib/libpython2.5.so.1.0 [0xe0137a] [trantor:06601] [22] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [23] /usr/local/lib/libpython2.5.so.1.0 [0xde6de5] [trantor:06601] [24] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [25] /usr/local/lib/libpython2.5.so.1.0 [0xe2abc9] [trantor:06601] [26] /usr/local/lib/libpython2.5.so.1.0(PyObject_Call+0x37) [0xddf5c7] [trantor:06601] [27] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalFrameEx+0x1481) [0xe5d691] [trantor:06601] [28] /usr/local/lib/libpython2.5.so.1.0(PyEval_EvalCodeEx+0x7c4) [0xe63814] [trantor:06601] [29] /usr/local/lib/libpython2.5.so.1.0 [0xe01450] [trantor:06601] *** End of error message *** As the problem seems to originate in Comm.Split(), I've written a small python script to test it:: from mpi4py import MPI # true MPI_COMM_WORLD_HANDLE BASECOMM = MPI.__COMM_WORLD__ BASE_SIZE = BASECOMM.Get_size() BASE_RANK = BASECOMM.Get_rank() if BASE_RANK < (BASE_SIZE // 2) : COLOR = 0 else: COLOR = 1 INTRACOMM = BASECOMM.Split(COLOR, key=0) print 'Done!!!' This seems always work, but running it under valgrind (note valgrind-py below is just an alias adding a suppression file for python) I get the following: mpiexec -n 3 valgrind-py python test.py =6727== Warning: set address range perms: large range 134217728 (defined) ==6727== Source and destination overlap in memcpy(0x4C93EA0, 0x4C93EA8, 16) ==6727==at 0x4006CE6: memcpy (mc_replace_strmem.c:116) ==6727==by 0x46C59CA: ompi_ddt_copy_content_same_ddt (in /usr/local/openmpi/1.2.3/lib/libmpi.so.0.0.0) ==6727==by 0x4BADDCE: ompi_coll_tuned_allgather_intra_bruck (in /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so) ==6727==by 0x4BA9B7A: ompi_coll_tuned_allgather_intra_dec_fixed (in /usr/local/openmpi/1.2.3/lib/openmpi/mca_coll_tuned.so) ==6727==by 0x46A692C: ompi_comm_split (in /usr/local/openmpi/1.2.3/lib/libmpi.so.0.0.0) ==6
[OMPI devel] Notes on building and running Open MPI on Red Storm
Some supplementary information to the wiki at https://svn.open-mpi.org/trac/ompi/wiki/CrayXT3. I. Accessing the Open MPI source: * Subversion is installed on redstorm in /projects/unsupported/bin * Reddish has subversion in the default path (you don't have to load a module) * The proxy information in the wiki is accurate, and works on both redstorm and reddish II. Building Open MPI on reddish: * 'module load PrgEnv-pgi-xc' to cross compile for redstorm * Reddish and redstorm do not have recent enough version of autotools, so you must build your own (currently available in /project/openmpi/rbbrigh/tools) * Suggested configuration: 'configure CC=qk-gcc CXX=qk-pgCC F77=qk-pgf77 FC=qk-pgf90 --disable-mpi-profile --with-platform=redstorm --host=x86_64-cray-linux-gnu --build=x86_64-unknown-linux-gnu --disable-mpi-f90 --disable-mem-debug --disable-mem-profile --disable-debug build_alias=x86_64-unknown-linux-gnu host_alias=x86_64-cray-linux-gnu' III. Building with Open MPI: * No executables will be installed in $PREFIX/bin, so must compile without mpicc, e.g. 'qk-gcc -I$PREFIX/include *.c -L$PREFIX/lib -lmpi -lopen-rte -lopen-pal' * When linking with libopen-pal, the following warning is normal: 'In function `checkpoint_response': warning: mkfifo is not implemented and will always fail' IV. Running on Redstorm: * scp your executable from reddish to redstorm * Use 'qsub' to submit job and 'yod' to launch job (if you do an interactive session, you can bypass PBS) * qsub expects project/task information - you can either provide this with -A option or set it in $XT_ACCOUNT environmental variable * Sample job script for qsub with 8 nodes/processes and 10 minute runtime: #!/bin/sh #PBS -lsize=8 #PBS -lwalltime=10:00 cd $PBS_O_WORKDIR yod a.out * Use 'xtshowmesh' and 'qstat' to query job status and cluster configuration
Re: [OMPI devel] Multi-environment builds
On Jul 11, 2007, at 8:09 AM, Terry D. Dontje wrote: Jeff Squyres wrote: On Jul 10, 2007, at 1:26 PM, Ralph H Castain wrote: 2. It may be useful to have some high-level parameters to specify a specific run-time environment, since ORTE has multiple, related frameworks (e.g., RAS and PLS). E.g., "orte_base_launcher=tm", or somesuch. I was just writing this up in an enhancement ticket when the though hit me: isn't this aggregate MCA parameters? I.e.: mpirun --am tm ... Specifically, we'll need to make a "tm" AMCA file (and whatever other ones we want), but my point is: does AMCA already give us what we want? The above sounds like a possible solution as long as we are going to deliver a set of such files and not require each site to create their own. Also, can one pull in multiple AMCA files for one run thus you can specify a tm AMCA and possibly some other AMCA file that the user may want? Yep. You can put a ':' between different parameters. So: shell$ mpirun -am tm:foo:bar ... will pull in the three AMCA files 'tm' 'foo' 'bar' in that order of precedence. Meaning that 'tm' can override a MCA parameter in 'foo', and 'foo' can override a MCA parameter in 'bar'. And any '-mca' command line options take a higher precedence than AMCA parameter files, so could override MCA parameters set by any of 'tm' 'foo' or 'bar'. I'll put it on my list to make a faq entry for AMCA usage, as I don't see one. -- Josh --td ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel Josh Hursey jjhur...@open-mpi.org http://www.open-mpi.org/
Re: [OMPI devel] [OMPI users] warning:regcache incompatible with malloc
Scott Atchley wrote: On Jul 10, 2007, at 3:24 PM, Tim Prins wrote: On Tuesday 10 July 2007 03:11:45 pm Scott Atchley wrote: On Jul 10, 2007, at 2:58 PM, Scott Atchley wrote: Tim, starting with the recently released 1.2.1, it is the default. To clarify, MX_RCACHE=1 is the default. It would be good for the default to be something where there is no warning printed (i.e. 0 or 2). I see the warning on the current trunk. Tim After further discussion in-house, the warning can be avoided if - lmyriexpress is included when linking the app (i.e. if it is in mpicc when linking). We cannot do this since we create network agnostic executables so that users can select networks at runtime. Doing -lmyriexpress would put an artificial dependency on the myrinet library, even if the user does not want to use it. Another clarification, the regache does work with several replacement malloc libraries. If the user application overloads mmap(), munmap() and sbrk(), then it may or may not work. In this case, the user should use MX_RCACHE=0. This sounds to me like a lot to ask the user to do... My opinion is that if MX_RCACHE is not explicitly set by the user, Open MPI should set it to either 0 or 2 automatically. An explicit goal Open MPI is for it to automatically do the right thing in most cases. Letting a ton of warnings be spit out at the user, in my opinion, is the wrong thing. Tim
Re: [OMPI devel] patch for btl_sm.c fixing segmentation fault
On Wed, Jul 11, 2007 at 01:17:02PM +0200, Christoph Niethammer wrote: > Hello, > > > Since some time I'm testing Open MPI at the HRLS. My main topic there is the > thread support of Open MPI. > > Some time ago I found a segmentation fault when running the svn-trunk > Version. > Thanks to the help of Sven I could locate it now to be in the shared memory > btl. (ompi/mca/btl/sm/btl_sm.c) > There the addresses of the fifos were modified because of the different > memory > mapping for the threads. Unfortunately this modification was not applied for > the head_locks. > > The attached patch should fix this problem. > Maybe someone could have a look on it? I see that Sven is already committed the fix to trunk r15291, but it seems the better fix would be to allocate tail_lock and head_lock not from shared memory at all but in a local memory of a process that is going to use respective lock. -- Gleb.
Re: [OMPI devel] Multi-environment builds
Interesting point - no reason why we couldn't use that functionality for this purpose. Good idea! On 7/11/07 5:38 AM, "Jeff Squyres" wrote: > On Jul 10, 2007, at 1:26 PM, Ralph H Castain wrote: > >>> 2. It may be useful to have some high-level parameters to specify a >>> specific run-time environment, since ORTE has multiple, related >>> frameworks (e.g., RAS and PLS). E.g., "orte_base_launcher=tm", or >>> somesuch. > > I was just writing this up in an enhancement ticket when the though > hit me: isn't this aggregate MCA parameters? I.e.: > > mpirun --am tm ... > > Specifically, we'll need to make a "tm" AMCA file (and whatever other > ones we want), but my point is: does AMCA already give us what we want?
Re: [OMPI devel] Multi-environment builds
Jeff Squyres wrote: On Jul 10, 2007, at 1:26 PM, Ralph H Castain wrote: 2. It may be useful to have some high-level parameters to specify a specific run-time environment, since ORTE has multiple, related frameworks (e.g., RAS and PLS). E.g., "orte_base_launcher=tm", or somesuch. I was just writing this up in an enhancement ticket when the though hit me: isn't this aggregate MCA parameters? I.e.: mpirun --am tm ... Specifically, we'll need to make a "tm" AMCA file (and whatever other ones we want), but my point is: does AMCA already give us what we want? The above sounds like a possible solution as long as we are going to deliver a set of such files and not require each site to create their own. Also, can one pull in multiple AMCA files for one run thus you can specify a tm AMCA and possibly some other AMCA file that the user may want? --td
Re: [OMPI devel] Multi-environment builds
On Jul 10, 2007, at 1:26 PM, Ralph H Castain wrote: 2. It may be useful to have some high-level parameters to specify a specific run-time environment, since ORTE has multiple, related frameworks (e.g., RAS and PLS). E.g., "orte_base_launcher=tm", or somesuch. I was just writing this up in an enhancement ticket when the though hit me: isn't this aggregate MCA parameters? I.e.: mpirun --am tm ... Specifically, we'll need to make a "tm" AMCA file (and whatever other ones we want), but my point is: does AMCA already give us what we want? -- Jeff Squyres Cisco Systems
[OMPI devel] patch for btl_sm.c fixing segmentation fault
Hello, Since some time I'm testing Open MPI at the HRLS. My main topic there is the thread support of Open MPI. Some time ago I found a segmentation fault when running the svn-trunk Version. Thanks to the help of Sven I could locate it now to be in the shared memory btl. (ompi/mca/btl/sm/btl_sm.c) There the addresses of the fifos were modified because of the different memory mapping for the threads. Unfortunately this modification was not applied for the head_locks. The attached patch should fix this problem. Maybe someone could have a look on it? Regards Christoph Index: ompi/mca/btl/sm/btl_sm.c === --- ompi/mca/btl/sm/btl_sm.c (Revision 15143) +++ ompi/mca/btl/sm/btl_sm.c (Arbeitskopie) @@ -516,8 +516,11 @@ /* Calculate the difference as (my_base - their_base) */ diff = tmp_ptr[mca_btl_sm_component.my_smp_rank] - tmp_ptr[j]; mca_btl_sm_component.fifo[j] = (ompi_fifo_t*)((char*)fifo_tmp[j]+diff); + +mca_btl_sm_component.fifo[j][mca_btl_sm_component.my_smp_rank].head_lock = + (opal_atomic_lock_t*) ((char*)mca_btl_sm_component.fifo[j][mca_btl_sm_component.my_smp_rank].head_lock + diff); + mca_btl_sm_component.sm_offset[j] = diff; - } for( j=mca_btl_sm_component.num_smp_procs ; j < pgpDKJG53Fh1y.pgp Description: PGP signature
[OMPI devel] Off topic: ptpd for time synchronization
This is a bit off-topic for this list, but I was wondering if anyone has any experience working with ptpd (precision time protocol daemon; following the IEEE 1588 spec). The point of ptpd is to give better time precision than NTP; NTP gives accuracy on the order of miliseconds where ptpd/IEEE 1588 gives accuracy on the order of microseconds. http://ptpd.sourceforge.net/ Having coordination accuracy within miliseconds could be quite helpful to MPI in multiple ways: giving more accurate MPI tracing outputs, the possibility of scheduling communication (particularly for MPI collectives) in oversubscribed networks, etc. I'd like to give ptpd a whirl, but there's very little documentation and I can't find any mailing lists or other points of contact where to ask a few questions. In particular, I would like to run ptpd in a way that I'm guessing would be fairly common in HPC environments: use NTP to get the time to my cluster's head node and then use ptpd to synchronize my cluster to the NTP'ed head node. However, it's not clear to me how ptpd works -- how do I designate one head node as the "master"? What, exactly, do all the command line options to ptpd mean? (there's only a limited "--help" kind of message to explain them) And so on. I have a busy/active cluster, so I don't want to muck up the clock (and therefore potentially muck up NFS file timestamps) -- some level of experimentation is ok, but I don't want to unintentionally cause a large/bad effect (particularly in terms of NFS) if possible. I'm also curious as to how much network overhead ptpd incurs, both at startup and in its steady state operation. If anyone has any insight or experience with ptpd, I'd love to hear it. Thanks! -- Jeff Squyres Cisco Systems
