Re: [OMPI devel] Warning
Luis,
With some low frequency we remove warnings from the code. In this
particular instance the meaning of the code is correct, the ompi_info_t
structure starts with an opal_info_t, but removing the warnings is good
policy.
In general we can either cast the ompi_info_t pointer directly to an
opal_info_t pointer, or access the super field in the ompi_info_t structure
(as it is done in ompi/mpi/c/win_create_dynamic.c). As in this instance we
are explicitly using one of the MPI predefined info keys (without
equivalent at the OPAL level) it is more clear if we cast it.
George.
On Fri, May 15, 2020 at 6:37 AM Luis via devel
wrote:
> Hi OMPI devs,
>
> I was wondewring if this warning is expected, if not, how should we
> internally call ompi_win_create_dynamic?
>
> res = ompi_win_create_dynamic(MPI_INFO_NULL, comm, &win);
> ^
> In file included from pnbc_osc_internal.h:40,
> from pnbc_osc_iallreduce.c:21:
> ../../../../ompi/win/win.h:143:42: note: expected ‘opal_info_t *’ {aka
> ‘struct opal_info_t *’} but argument is of type ‘struct ompi_info_t *’
> int ompi_win_create_dynamic(opal_info_t *info, ompi_communicator_t
> *comm, ompi_win_t **newwin);
>
>
> Regards,
> Luis
> The University of Edinburgh is a charitable body, registered in Scotland,
> with registration number SC005336.
>
Re: [OMPI devel] Warning
Luis,
you can do this:
struct ompi_info_t * info = &ompi_mpi_info_null.info;
ret = ompi_win_create_dynamic(&info_null->super, comm, win);
Cheers,
Gilles
On Fri, May 15, 2020 at 7:38 PM Luis via devel wrote:
>
> Hi OMPI devs,
>
> I was wondewring if this warning is expected, if not, how should we
> internally call ompi_win_create_dynamic?
>
> res = ompi_win_create_dynamic(MPI_INFO_NULL, comm, &win);
> ^
> In file included from pnbc_osc_internal.h:40,
> from pnbc_osc_iallreduce.c:21:
> ../../../../ompi/win/win.h:143:42: note: expected ‘opal_info_t *’ {aka
> ‘struct opal_info_t *’} but argument is of type ‘struct ompi_info_t *’
> int ompi_win_create_dynamic(opal_info_t *info, ompi_communicator_t
> *comm, ompi_win_t **newwin);
>
>
> Regards,
> Luis
> The University of Edinburgh is a charitable body, registered in Scotland,
> with registration number SC005336.
[OMPI devel] Warning
Hi OMPI devs,
I was wondewring if this warning is expected, if not, how should we
internally call ompi_win_create_dynamic?
res = ompi_win_create_dynamic(MPI_INFO_NULL, comm, &win);
^
In file included from pnbc_osc_internal.h:40,
from pnbc_osc_iallreduce.c:21:
../../../../ompi/win/win.h:143:42: note: expected ‘opal_info_t *’ {aka
‘struct opal_info_t *’} but argument is of type ‘struct ompi_info_t *’
int ompi_win_create_dynamic(opal_info_t *info, ompi_communicator_t
*comm, ompi_win_t **newwin);
Regards,
Luis
The University of Edinburgh is a charitable body, registered in Scotland, with
registration number SC005336.
pEpkey.asc
Description: application/pgp-keys
Re: [OMPI devel] #warning "Including liblustreapi.h is deprecated. Include lustreapi.h directly."
I'll put it on my to do list to write the configure logic for that, shouldn't be too difficult. Thanks for the report.Edgar___ devel mailing list devel@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/devel
[OMPI devel] #warning "Including liblustreapi.h is deprecated. Include lustreapi.h directly."
Dear Open MPI developer, trying to compile Open MPI 2.0.1 using PGI compilers (19.9 and 16.7) we ran into an error the root of one is not known yet, but we see also this warning: #warning "Including liblustreapi.h is deprecated. Include lustreapi.h directly." in the /usr/include/lustre/liblustreapi.h file, included from 'openmpi-2.0.1/ompi/mca/fs/lustre/fs_lustre.c' file (line 46, doh). well, it is about you on change or keep the way the Lustre headers being included in Open MPI. Just my $2%. Have a nice day, Paul Kapinos pk224850@lnm001:/w0/tmp/pk224850/OpenMPI/2.0.1/pgi_16.9./openmpi-2.0.1/ompi/mca/fs/lustre[527]$ make CC fs_lustre.lo pgcc -DHAVE_CONFIG_H -I. -I../../../../opal/include -I../../../../ompi/include -I../../../../oshmem/include -I../../../../opal/mca/hwloc/hwloc1112/hwloc/include/private/autogen -I../../../../opal/mca/hwloc/hwloc1112/hwloc/include/hwloc/autogen -I../../../../ompi/mpiext/cuda/c -I../../../.. -I../../../../orte/include -I/w0/tmp/pk224850/OpenMPI/2.0.1/pgi_16.9./openmpi-2.0.1/opal/mca/hwloc/hwloc1112/hwloc/include -I/w0/tmp/pk224850/OpenMPI/2.0.1/pgi_16.9./openmpi-2.0.1/opal/mca/event/libevent2022/libevent -I/w0/tmp/pk224850/OpenMPI/2.0.1/pgi_16.9./openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include -DNDEBUG -O0 -c fs_lustre.c -MD -fpic -DPIC -o .libs/fs_lustre.o PGC-W-0267-#warning -- "Including liblustreapi.h is deprecated. Include lustreapi.h directly." (/usr/include/lustre/liblustreapi.h: 41) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 58) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 58) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 61) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 61) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 65) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 65) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 68) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 68) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 72) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 72) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 75) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 75) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 91) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 91) PGC-W-0114-More than one type specified (/usr/include/libcfs/posix/posix-types.h: 94) PGC-W-0143-Useless typedef declaration (no declarators present) (/usr/include/libcfs/posix/posix-types.h: 94) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 157) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 157) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 158) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 158) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 159) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 159) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 160) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 160) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 161) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 161) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 162) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 162) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 163) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 163) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 164) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 164) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 211) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 211) PGC-S-0040-Illegal use of symbol, u_int64_t (/usr/include/sys/quota.h: 212) PGC-W-0156-Type not specified, 'int' assumed (/usr/include/sys/quota.h: 212) PGC-W-0095-Type cast required for this conversion (fs_lustre.c: 93) PGC/x86-64 Linux 16.9-0: compilation completed with severe errors make: *** [fs_lustre.lo] Error 1 -- Dipl.-Inform. Paul Kapinos - High Performance Computing, RWTH Aachen University, IT Center Seffenter Weg 23, D 52074 Aachen (Germany) Tel: +49 241/80-24915 smime.p7s Description: S/MIME Cryptographic Signature __
[OMPI devel] warning in openib BTL
I'm also seeing these warnings this morning: connect/btl_openib_connect_rdmacm.c:369:5: warning: "BTL_OPENIB_RDMACM_IB_ADDR" is not defined -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] Warning in fcoll
I think this is one of those local functions I defined. I wanted to generalize this for all collective operations, such that its a permanent fix. I can do this today. Regards, Vish > I'll look into this... > > Edgar > > On 5/29/2012 3:24 PM, Ralph Castain wrote: >> Not entirely sure who this might belong to, but thought I should pass it >> along - seen during an optimized build on Linux: >> >> fcoll_static_file_read_all.c: In function >> mca_fcoll_static_file_read_all: >> fcoll_static_file_read_all.c:74: warning: sorted_file_offsets may be >> used uninitialized in this function >> >> >> >> >> ___ >> devel mailing list >> de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/devel > > -- > Edgar Gabriel > Associate Professor > Parallel Software Technologies Lab http://pstl.cs.uh.edu > Department of Computer Science University of Houston > Philip G. Hoffman Hall, Room 524Houston, TX-77204, USA > Tel: +1 (713) 743-3857 Fax: +1 (713) 743-3335 > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] Warning in fcoll
I'll look into this... Edgar On 5/29/2012 3:24 PM, Ralph Castain wrote: > Not entirely sure who this might belong to, but thought I should pass it > along - seen during an optimized build on Linux: > > fcoll_static_file_read_all.c: In function ‘mca_fcoll_static_file_read_all’: > fcoll_static_file_read_all.c:74: warning: ‘sorted_file_offsets’ may be used > uninitialized in this function > > > > > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Edgar Gabriel Associate Professor Parallel Software Technologies Lab http://pstl.cs.uh.edu Department of Computer Science University of Houston Philip G. Hoffman Hall, Room 524Houston, TX-77204, USA Tel: +1 (713) 743-3857 Fax: +1 (713) 743-3335 signature.asc Description: OpenPGP digital signature
[OMPI devel] Warning in fcoll
Not entirely sure who this might belong to, but thought I should pass it along - seen during an optimized build on Linux: fcoll_static_file_read_all.c: In function ‘mca_fcoll_static_file_read_all’: fcoll_static_file_read_all.c:74: warning: ‘sorted_file_offsets’ may be used uninitialized in this function
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
Excellent points Ken; thanks! I expanded the FAQ entry here to include these points: http://www.open-mpi.org/faq/?category=openfabrics#ofa-fork On Nov 30, 2010, at 9:52 AM, Ken Cain wrote: > Hi Jeff, > > We have had some recent experience with this in an Open MPI 1.4.x version and > thought it would be useful to contribute to the discussion. Please see below. > > Jeff Squyres wrote: >> On Nov 29, 2010, at 6:25 PM, George Bosilca wrote: >>> The main problem is that openib require to pin memory pages in order to >>> take advantage of RMA features. There is a major issues with these pinned >>> pages and fork, leading to segmentation fault in some specific cases. >>> However, we only pin the pages on the MPI calls related to data transfers. >>> Therefore, if you call fork __before__ any other MPI data transfer function >>> (but after MPI_Init as you use the process rank), your application should >>> be safe. >> Note that Open MPI also pins some internal memory during MPI_INIT, but that >> memory is totally internal to libmpi, so you should be safe (i.e., you >> should never be able to find it and therefore never be able to try to touch >> it). > > This is what we believe happened in our testing: > > 1. MPI_init allocated and pinned down some memory. This memory was 64 byte > aligned and not page-aligned to 4096 bytes. So an allocation that ideally > should have resulted in 2 pages being pinned, actually had 3 pages pinned > with lots of unused memory on the 3rd page. > > 2. A child process created via popen tried to allocate some memory (perhaps a > byproduct of popen execution itself) and was allocated memory on that last > page with lots of unused memory. When the child tried to touch the > allocation, there was seg fault. > > We could reduce the probability of this happenning by changing the alignment > of MPI allocations to 4096 bytes. But since MPI allocations are not sized to > be multiple of page size, this isn't a foolproof method. > > One way (agreed not ideal) to avoid the potential seg fault is to set the MCA > parameter btl_openib_want_fork_suppoort = 0. But then you are "trusting" any > child processes to not intentionally or as a result of a bug, touch the > memory regions that have been registered/pinned by the parent. > How can one be sure that the disabling the warning is ok? Could you please elaborate on what makes forks vulnerable? May be that will guide the developers to make an informed decision on whether to disable them or find another alternative. >>> No way to know at 100%. Now for an elaborate answer: Once upon a time ... >>> The fork story is a long and boring one, we would all have preferred to >>> never heard about it (believe me). A quick and compressed version can be >>> found on the QLogic download page >>> (http://filedownloads.qlogic.com/files/driver/70277/release_QLogicIB-Basic_4400_Rev_A.html). >> That's a good summary. The issue is with OFED itself, not with Open MPI. >> Note, too, that calling popen() should also be safe (even though we'll warn >> about it -- our atfork hook has no way of knowing whether you're calling >> system, popen, or something else). > > Thanks, > > -Ken > -- > Ken Cain > Mercury Computer Systems, Inc. (http://www.mc.com) > > This message is intended only for the designated recipient(s) and may > contain confidential or proprietary information of Mercury Computer > Systems, Inc. This message is solely intended to facilitate business > discussions and does not constitute an express or implied offer to sell > or purchase any products, services, or support. Any commitments must be > made in writing and signed by duly authorized representatives of each > party. > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
Hi Jeff, We have had some recent experience with this in an Open MPI 1.4.x version and thought it would be useful to contribute to the discussion. Please see below. Jeff Squyres wrote: On Nov 29, 2010, at 6:25 PM, George Bosilca wrote: The main problem is that openib require to pin memory pages in order to take advantage of RMA features. There is a major issues with these pinned pages and fork, leading to segmentation fault in some specific cases. However, we only pin the pages on the MPI calls related to data transfers. Therefore, if you call fork __before__ any other MPI data transfer function (but after MPI_Init as you use the process rank), your application should be safe. Note that Open MPI also pins some internal memory during MPI_INIT, but that memory is totally internal to libmpi, so you should be safe (i.e., you should never be able to find it and therefore never be able to try to touch it). This is what we believe happened in our testing: 1. MPI_init allocated and pinned down some memory. This memory was 64 byte aligned and not page-aligned to 4096 bytes. So an allocation that ideally should have resulted in 2 pages being pinned, actually had 3 pages pinned with lots of unused memory on the 3rd page. 2. A child process created via popen tried to allocate some memory (perhaps a byproduct of popen execution itself) and was allocated memory on that last page with lots of unused memory. When the child tried to touch the allocation, there was seg fault. We could reduce the probability of this happenning by changing the alignment of MPI allocations to 4096 bytes. But since MPI allocations are not sized to be multiple of page size, this isn't a foolproof method. One way (agreed not ideal) to avoid the potential seg fault is to set the MCA parameter btl_openib_want_fork_suppoort = 0. But then you are "trusting" any child processes to not intentionally or as a result of a bug, touch the memory regions that have been registered/pinned by the parent. How can one be sure that the disabling the warning is ok? Could you please elaborate on what makes forks vulnerable? May be that will guide the developers to make an informed decision on whether to disable them or find another alternative. No way to know at 100%. Now for an elaborate answer: Once upon a time ... The fork story is a long and boring one, we would all have preferred to never heard about it (believe me). A quick and compressed version can be found on the QLogic download page (http://filedownloads.qlogic.com/files/driver/70277/release_QLogicIB-Basic_4400_Rev_A.html). That's a good summary. The issue is with OFED itself, not with Open MPI. Note, too, that calling popen() should also be safe (even though we'll warn about it -- our atfork hook has no way of knowing whether you're calling system, popen, or something else). Thanks, -Ken -- Ken Cain Mercury Computer Systems, Inc. (http://www.mc.com) This message is intended only for the designated recipient(s) and may contain confidential or proprietary information of Mercury Computer Systems, Inc. This message is solely intended to facilitate business discussions and does not constitute an express or implied offer to sell or purchase any products, services, or support. Any commitments must be made in writing and signed by duly authorized representatives of each party.
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
On Nov 30 2010, Ralph Castain wrote: Here is what one IB vendor says about the issue on their web site (redacted to protect the innocent): "At the time of this release, the (redacted-openib) driver has issues with buffers sharing pages when fork( ) is used. Pinned (locked in memory) pages are normally marked copy-on-write during a fork. That is TRULY demented! It is almost always precisely the wrong thing to do. If a page is pinned before a fork and subsequently written to while RDMA operations are being performed on the same page, silent data corruption can occur as RDMA operations continue to stream data to a page that has moved. To avoid this, the (redacted-openib) driver does not use copy-on-write behavior during a fork for pinned pages. Instead, access to these pages by the child process will result in a segmentation violation." That is sane. Not user-friendly, but at least sane. While there is some variation, I believe you will find that all IB comm shares this problem. So it is wise to avoid using fork if you want to use the openib transport. Yes and no. Some such communication may allow RDMA only to shared memory, which solves the problem in another way. Several specialist HPC networks were (are?) like that, and I can see no reason why an IB driver should not use the same design. That, of course, means that most MPI transfers need a copy. Hence the warning. Ignoring it is purely a "user beware" situation, but we provide that mechanism for the truly adventurous...or IB developers who want to someday resolve the problem. Well, there is a much simpler case where it will "just work", which is very probably what the OP was doing. When the fork is immediately followed by an exec in the child process, there isn't an issue. We all know the history, but the mainframe designs of having a proper spawn primitive were much cleaner. However, that's not what we've got. It might be worth adding to the note that this is the ONLY case when the ordinary user is advised to use that facility. Or it might not, depending on the level of Clue that readers are expected to have. Regards, Nick Maclaren.
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
On Nov 29, 2010, at 6:25 PM, George Bosilca wrote: > The main problem is that openib require to pin memory pages in order to take > advantage of RMA features. There is a major issues with these pinned pages > and fork, leading to segmentation fault in some specific cases. However, we > only pin the pages on the MPI calls related to data transfers. Therefore, if > you call fork __before__ any other MPI data transfer function (but after > MPI_Init as you use the process rank), your application should be safe. Note that Open MPI also pins some internal memory during MPI_INIT, but that memory is totally internal to libmpi, so you should be safe (i.e., you should never be able to find it and therefore never be able to try to touch it). >> How can one be sure that the disabling the warning is ok? Could you please >> elaborate on what makes forks vulnerable? May be that will guide the >> developers to make an informed decision on whether to disable them or find >> another alternative. > > No way to know at 100%. Now for an elaborate answer: Once upon a time ... The > fork story is a long and boring one, we would all have preferred to never > heard about it (believe me). A quick and compressed version can be found on > the QLogic download page > (http://filedownloads.qlogic.com/files/driver/70277/release_QLogicIB-Basic_4400_Rev_A.html). That's a good summary. The issue is with OFED itself, not with Open MPI. Note, too, that calling popen() should also be safe (even though we'll warn about it -- our atfork hook has no way of knowing whether you're calling system, popen, or something else). -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
Here is what one IB vendor says about the issue on their web site (redacted to protect the innocent): "At the time of this release, the (redacted-openib) driver has issues with buffers sharing pages when fork( ) is used. Pinned (locked in memory) pages are normally marked copy-on-write during a fork. If a page is pinned before a fork and subsequently written to while RDMA operations are being performed on the same page, silent data corruption can occur as RDMA operations continue to stream data to a page that has moved. To avoid this, the (redacted-openib) driver does not use copy-on-write behavior during a fork for pinned pages. Instead, access to these pages by the child process will result in a segmentation violation." While there is some variation, I believe you will find that all IB comm shares this problem. So it is wise to avoid using fork if you want to use the openib transport. Hence the warning. Ignoring it is purely a "user beware" situation, but we provide that mechanism for the truly adventurous...or IB developers who want to someday resolve the problem. On Nov 29, 2010, at 3:44 PM, wrote: > George > > Thanks for the explanation. I am trying to understand the following line in > your mail: > > “In fact, any fork done prior to the communication is a non-issue, but it is > difficult to identify. Therefore, we output the warning as soon as we detect > a fork after MPI_Init.” > > Does it mean that if I have a fork() after the communication (ie; mpi_send or > mpi_receive etc), I may have to relook at a different implementation to be at > safe side? I don’t want to suppress the messages if they result in any > corruption later. > > How can one be sure that the disabling the warning is ok? Could you please > elaborate on what makes forks vulnerable? May be that will guide the > developers to make an informed decision on whether to disable them or find > another alternative. > > > > Thanks > > Ananda > > ------ PREVIOUS MESSAGE - > > Subject: Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!! > From: George Bosilca (bosilca_at_[hidden]) > Date: 2010-11-29 12:22:15 > > Next message: Jeff Squyres: "Re: [OMPI devel] Warning on fork() disappears if > I use MPI threads!!" > Previous message: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning on > fork() disappears if I use MPI threads!!" > In reply to: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning on fork() > disappears if I use MPI threads!!" > Next in thread: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() disappears > if I use MPI threads!!" > Reply: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() disappears if I use > MPI threads!!" > If your code doesn't exactly what is described in the code snippet attached > to your previous email, then you can safely ignore the warning. In fact, any > fork done prior to the communication is a non-issue, but it is difficult to > identify. Therefore, we output the warning as soon as we detect a fork after > MPI_Init. > > You can find more information about the usage of fork in Open MPI at > http://www.open-mpi.de/faq/?category=tuning#fork-warning > > george. > > On Nov 29, 2010, at 12:12 , wrote: > > > I am posting this question again as it was sent before the long weekend and > > didn’t see any responses so far. Can anyone please explain the discrepancy > > I am observing with the scenario explained in the post below? > > > > Thanks > > Ananda > > Sent: Tuesday, November 23, 2010 2:24 PM > > To: devel_at_[hidden] > > Subject: Warning on fork() disappears if I use MPI threads!! > > > > Hi > > > > I am running into a very wierd problem. > > > > If I initialize MPI normally ie; with MPI_Init(), and make one of the MPI > > process to do "popen()" call, I get the following warning/error message: > > > > == Message start === > > An MPI process has executed an operation involving a call to the > > "fork()" system call to create a child process. Open MPI is currently > > operating in a condition that could result in memory corruption or > > other system errors; your MPI job may hang, crash, or produce silent > > data corruption. The use of fork() (or system() or other calls that > > create child processes) is strongly discouraged. > > == Message end > > > > However this error message goes away, if I initialize MPI with threads ie; > > MPI_Init_thread(). Can anyone explain this discrepancy? > > > > I am giving a snippet of th
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
On Nov 29, 2010, at 17:44 , wrote: > George > > Thanks for the explanation. I am trying to understand the following line in > your mail: > > “In fact, any fork done prior to the communication is a non-issue, but it is > difficult to identify. Therefore, we output the warning as soon as we detect > a fork after MPI_Init.” > > Does it mean that if I have a fork() after the communication (ie; mpi_send or > mpi_receive etc), I may have to relook at a different implementation to be at > safe side? I don’t want to suppress the messages if they result in any > corruption later. The main problem is that openib require to pin memory pages in order to take advantage of RMA features. There is a major issues with these pinned pages and fork, leading to segmentation fault in some specific cases. However, we only pin the pages on the MPI calls related to data transfers. Therefore, if you call fork __before__ any other MPI data transfer function (but after MPI_Init as you use the process rank), your application should be safe. > How can one be sure that the disabling the warning is ok? Could you please > elaborate on what makes forks vulnerable? May be that will guide the > developers to make an informed decision on whether to disable them or find > another alternative. No way to know at 100%. Now for an elaborate answer: Once upon a time ... The fork story is a long and boring one, we would all have preferred to never heard about it (believe me). A quick and compressed version can be found on the QLogic download page (http://filedownloads.qlogic.com/files/driver/70277/release_QLogicIB-Basic_4400_Rev_A.html). > Arbitrary fork() support is not supported in the OpenFabrics software stack. > If you use fork() in your application, you must not touch any registered > memory before calling some form of exec() to launch another process. Calling > system() is safe. This limitation includes both native InfiniBand > applications and MPI applications using a RDMA based implementation of MPI > (such as OFED's openmpi and mvapich). george. > > > > Thanks > > Ananda > > ------ PREVIOUS MESSAGE - > > Subject: Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!! > From: George Bosilca (bosilca_at_[hidden]) > Date: 2010-11-29 12:22:15 > > • Next message: Jeff Squyres: "Re: [OMPI devel] Warning on fork() > disappears if I use MPI threads!!" > • Previous message: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning > on fork() disappears if I use MPI threads!!" > • In reply to: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning on > fork() disappears if I use MPI threads!!" > • Next in thread: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() > disappears if I use MPI threads!!" > • Reply: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() disappears > if I use MPI threads!!" > If your code doesn't exactly what is described in the code snippet attached > to your previous email, then you can safely ignore the warning. In fact, any > fork done prior to the communication is a non-issue, but it is difficult to > identify. Therefore, we output the warning as soon as we detect a fork after > MPI_Init. > > You can find more information about the usage of fork in Open MPI at > http://www.open-mpi.de/faq/?category=tuning#fork-warning > > george. > > On Nov 29, 2010, at 12:12 , wrote: > > > I am posting this question again as it was sent before the long weekend and > > didn’t see any responses so far. Can anyone please explain the discrepancy > > I am observing with the scenario explained in the post below? > > > > Thanks > > Ananda > > Sent: Tuesday, November 23, 2010 2:24 PM > > To: devel_at_[hidden] > > Subject: Warning on fork() disappears if I use MPI threads!! > > > > Hi > > > > I am running into a very wierd problem. > > > > If I initialize MPI normally ie; with MPI_Init(), and make one of the MPI > > process to do "popen()" call, I get the following warning/error message: > > > > == Message start === > > An MPI process has executed an operation involving a call to the > > "fork()" system call to create a child process. Open MPI is currently > > operating in a condition that could result in memory corruption or > > other system errors; your MPI job may hang, crash, or produce silent > > data corruption. The use of fork() (or system() or other calls that > > create child processes) is strongly discouraged. > > == Message end > > > > However this e
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
George
Thanks for the explanation. I am trying to understand the following line
in your mail:
"In fact, any fork done prior to the communication is a non-issue, but
it is difficult to identify. Therefore, we output the warning as soon as
we detect a fork after MPI_Init."
Does it mean that if I have a fork() after the communication (ie;
mpi_send or mpi_receive etc), I may have to relook at a different
implementation to be at safe side? I don't want to suppress the messages
if they result in any corruption later.
How can one be sure that the disabling the warning is ok? Could you
please elaborate on what makes forks vulnerable? May be that will guide
the developers to make an informed decision on whether to disable them
or find another alternative.
Thanks
Ananda
-- PREVIOUS MESSAGE -
Subject: Re: [OMPI devel] Warning on fork() disappears if I use MPI
threads!!
From: George Bosilca (bosilca_at_[hidden])
List-Post: devel@lists.open-mpi.org
Date: 2010-11-29 12:22:15
* Next message: Jeff Squyres: "Re: [OMPI devel] Warning on fork()
disappears if I use MPI threads!!"
<http://www.open-mpi.org/community/lists/devel/2010/11/8730.php>
* Previous message: ananda.mudar_at_[hidden]: "Re: [OMPI devel]
Warning on fork() disappears if I use MPI threads!!"
<http://www.open-mpi.org/community/lists/devel/2010/11/8728.php>
* In reply to: ananda.mudar_at_[hidden]: "Re: [OMPI devel] Warning
on fork() disappears if I use MPI threads!!"
<http://www.open-mpi.org/community/lists/devel/2010/11/8728.php>
* Next in thread: N.M. Maclaren: "Re: [OMPI devel] Warning on
fork() disappears if I use MPI threads!!"
<http://www.open-mpi.org/community/lists/devel/2010/11/8731.php>
* Reply: N.M. Maclaren: "Re: [OMPI devel] Warning on fork()
disappears if I use MPI threads!!"
<http://www.open-mpi.org/community/lists/devel/2010/11/8731.php>
If your code doesn't exactly what is described in the code snippet
attached to your previous email, then you can safely ignore the warning.
In fact, any fork done prior to the communication is a non-issue, but it
is difficult to identify. Therefore, we output the warning as soon as we
detect a fork after MPI_Init.
You can find more information about the usage of fork in Open MPI at
http://www.open-mpi.de/faq/?category=tuning#fork-warning
george.
On Nov 29, 2010, at 12:12 , wrote:
> I am posting this question again as it was sent before the long
weekend and didn't see any responses so far. Can anyone please explain
the discrepancy I am observing with the scenario explained in the post
below?
>
> Thanks
> Ananda
> Sent: Tuesday, November 23, 2010 2:24 PM
> To: devel_at_[hidden]
> Subject: Warning on fork() disappears if I use MPI threads!!
>
> Hi
>
> I am running into a very wierd problem.
>
> If I initialize MPI normally ie; with MPI_Init(), and make one of the
MPI process to do "popen()" call, I get the following warning/error
message:
>
> == Message start ===
> An MPI process has executed an operation involving a call to the
> "fork()" system call to create a child process. Open MPI is currently
> operating in a condition that could result in memory corruption or
> other system errors; your MPI job may hang, crash, or produce silent
> data corruption. The use of fork() (or system() or other calls that
> create child processes) is strongly discouraged.
> == Message end
>
> However this error message goes away, if I initialize MPI with threads
ie; MPI_Init_thread(). Can anyone explain this discrepancy?
>
> I am giving a snippet of the program that causes this problem:
>
> == Code snippet start ==
> if ( rank == 0) {
> output = popen("ls -l", "r");
> while((c=getc(output))!=EOF)
> printf("%c",c);
> pclose(output);
> }
> == Code snippet end ==
>
> If this is a design constraint, how can I overcome this problem.
>
> Thanks
> Ananda
>
> Ananda B Mudar, PMP
> Senior Technical Architect
> Wipro Technologies
> Ph: 972 765 8093 begin_of_the_skype_highlighting 972 765
8093 end_of_the_skype_highlighting
> Please do not print this email unless it is absolutely necessary.
>
> The information contained in this electronic message and any
attachments to this message are intended for the exclusive use of the
addressee(s) and may contain proprietary, confidential or privileged
information. If you are not the intended recipient, you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately and destroy all copies of this message and any attachments.
>
> WARNING: Computer viruses can be transmitted via email. The recipient
should
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
Jeff I am invoking MPI_Init_thread() with MPI_THREAD_MULTIPLE. If openib BTL is responsible for the warning and openib BTL is excluded in this case, then that explains the discrepancy. FYI, I invoked MPI_Init_thread() with MPI_THREAD_SERIALIZED and I got the warning message about the fork(). Thanks Ananda --- PREVIOUS MESSAGE - Subject: Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!! From: Jeff Squyres (jsquyres_at_[hidden]) List-Post: devel@lists.open-mpi.org Date: 2010-11-29 14:10:44 * Next message: N.M. Maclaren: "Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!" <http://www.open-mpi.org/community/lists/devel/2010/11/8731.php> * Previous message: George Bosilca: "Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!" <http://www.open-mpi.org/community/lists/devel/2010/11/8729.php> * In reply to: ananda.mudar_at_[hidden]: "[OMPI devel] Warning on fork() disappears if I use MPI threads!!" <http://www.open-mpi.org/community/lists/devel/2010/11/8698.php> On Nov 23, 2010, at 3:23 PM, wrote: > However this error message goes away, if I initialize MPI with threads ie; MPI_Init_thread(). Can anyone explain this discrepancy? What thread level are you invoking MPI_INIT_THREAD with? One possible reason this could be happening is that the openib BTL is excluding itself if you use MPI_THREAD_MULTIPLE. The openib BTL is the entity that is responsible for the fork warning -- if it's not being used, then no warning is issued because there is no problem with forking. -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
On Nov 29 2010, George Bosilca wrote: If your code doesn't exactly what is described in the code snippet attached to your previous email, then you can safely ignore the warning. In fact, any fork done prior to the communication is a non-issue, but it is difficult to identify. Therefore, we output the warning as soon as we detect a fork after MPI_Init. USUALLY a non-issue. You can find more information about the usage of fork in Open MPI at http://www.open-mpi.de/faq/?category=tuning#fork-warning Yes. It says about as much as can be said, except between consenting geeks in private! Well, I suppose that this mailing list counts :-) I have had trouble with systems that sent signals to process groups and/or all processes with a particular controlling terminal, and even operating system 'super groups', where the location of the fork won't save you. I can't see how to describe that issue in terms that make sense to most users. Regards, Nick Maclaren.
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
On Nov 23, 2010, at 3:23 PM, wrote: > However this error message goes away, if I initialize MPI with threads ie; > MPI_Init_thread(). Can anyone explain this discrepancy? What thread level are you invoking MPI_INIT_THREAD with? One possible reason this could be happening is that the openib BTL is excluding itself if you use MPI_THREAD_MULTIPLE. The openib BTL is the entity that is responsible for the fork warning -- if it's not being used, then no warning is issued because there is no problem with forking. -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
If your code doesn't exactly what is described in the code snippet attached to
your previous email, then you can safely ignore the warning. In fact, any fork
done prior to the communication is a non-issue, but it is difficult to
identify. Therefore, we output the warning as soon as we detect a fork after
MPI_Init.
You can find more information about the usage of fork in Open MPI at
http://www.open-mpi.de/faq/?category=tuning#fork-warning
george.
On Nov 29, 2010, at 12:12 , wrote:
> I am posting this question again as it was sent before the long weekend and
> didn’t see any responses so far. Can anyone please explain the discrepancy I
> am observing with the scenario explained in the post below?
>
> Thanks
> Ananda
> Sent: Tuesday, November 23, 2010 2:24 PM
> To: de...@open-mpi.org
> Subject: Warning on fork() disappears if I use MPI threads!!
>
> Hi
>
> I am running into a very wierd problem.
>
> If I initialize MPI normally ie; with MPI_Init(), and make one of the MPI
> process to do "popen()" call, I get the following warning/error message:
>
> == Message start ===
> An MPI process has executed an operation involving a call to the
> "fork()" system call to create a child process. Open MPI is currently
> operating in a condition that could result in memory corruption or
> other system errors; your MPI job may hang, crash, or produce silent
> data corruption. The use of fork() (or system() or other calls that
> create child processes) is strongly discouraged.
> == Message end
>
> However this error message goes away, if I initialize MPI with threads ie;
> MPI_Init_thread(). Can anyone explain this discrepancy?
>
> I am giving a snippet of the program that causes this problem:
>
> == Code snippet start ==
> if ( rank == 0) {
> output = popen("ls -l", "r");
> while((c=getc(output))!=EOF)
> printf("%c",c);
> pclose(output);
> }
> == Code snippet end ==
>
> If this is a design constraint, how can I overcome this problem.
>
> Thanks
> Ananda
>
> Ananda B Mudar, PMP
> Senior Technical Architect
> Wipro Technologies
> Ph: 972 765 8093
> Please do not print this email unless it is absolutely necessary.
>
> The information contained in this electronic message and any attachments to
> this message are intended for the exclusive use of the addressee(s) and may
> contain proprietary, confidential or privileged information. If you are not
> the intended recipient, you should not disseminate, distribute or copy this
> e-mail. Please notify the sender immediately and destroy all copies of this
> message and any attachments.
>
> WARNING: Computer viruses can be transmitted via email. The recipient should
> check this email and any attachments for the presence of viruses. The company
> accepts no liability for any damage caused by any virus transmitted by this
> email.
>
> www.wipro.com
>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] Warning on fork() disappears if I use MPI threads!!
I am posting this question again as it was sent before the long weekend
and didn't see any responses so far. Can anyone please explain the
discrepancy I am observing with the scenario explained in the post
below?
Thanks
Ananda
Sent: Tuesday, November 23, 2010 2:24 PM
To: de...@open-mpi.org
Subject: Warning on fork() disappears if I use MPI threads!!
Hi
I am running into a very wierd problem.
If I initialize MPI normally ie; with MPI_Init(), and make one of the
MPI process to do "popen()" call, I get the following warning/error
message:
== Message start ===
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process. Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption. The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.
== Message end
However this error message goes away, if I initialize MPI with threads
ie; MPI_Init_thread(). Can anyone explain this discrepancy?
I am giving a snippet of the program that causes this problem:
== Code snippet start ==
if ( rank == 0) {
output = popen("ls -l", "r");
while((c=getc(output))!=EOF)
printf("%c",c);
pclose(output);
}
== Code snippet end ==
If this is a design constraint, how can I overcome this problem.
Thanks
Ananda
Ananda B Mudar, PMP
Senior Technical Architect
Wipro Technologies
Ph: 972 765 8093
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to
this message are intended for the exclusive use of the addressee(s) and may
contain proprietary, confidential or privileged information. If you are not the
intended recipient, you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately and destroy all copies of this message and
any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should
check this email and any attachments for the presence of viruses. The company
accepts no liability for any damage caused by any virus transmitted by this
email.
www.wipro.com
[OMPI devel] Warning on fork() disappears if I use MPI threads!!
Hi
I am running into a very wierd problem.
If I initialize MPI normally ie; with MPI_Init(), and make one of the MPI
process to do "popen()" call, I get the following warning/error message:
== Message start ===
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process. Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption. The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.
== Message end
However this error message goes away, if I initialize MPI with threads ie;
MPI_Init_thread(). Can anyone explain this discrepancy?
I am giving a snippet of the program that causes this problem:
== Code snippet start ==
if ( rank == 0) {
output = popen("ls -l", "r");
while((c=getc(output))!=EOF)
printf("%c",c);
pclose(output);
}
== Code snippet end ==
If this is a design constraint, how can I overcome this problem.
Thanks
Ananda
Ananda B Mudar, PMP
Senior Technical Architect
Wipro Technologies
Ph: 972 765 8093
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to
this message are intended for the exclusive use of the addressee(s) and may
contain proprietary, confidential or privileged information. If you are not the
intended recipient, you should not disseminate, distribute or copy this e-mail.
Please notify the sender immediately and destroy all copies of this message and
any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should
check this email and any attachments for the presence of viruses. The company
accepts no liability for any damage caused by any virus transmitted by this
email.
www.wipro.com
Re: [OMPI devel] [WARNING : A/V UNSCANNABLE] [PMX:VIRUS] RE: issue with --without-tm in configure?
On Oct 19, 2007, at 4:27 PM, Jennis Pruett wrote: Here are the needed output files ompi124...gz is the output from the configure and make, This file was ok. You didn't include the output from "ompi_info -- all", but I think that I can infer most of what it would have said (but not all -- see below). the imb..tar contains output from the trial run. This file appeared to be corrupted. But don't worry about it -- the first file may have been enough for this issue. The main focus is the same error as listed here below with tm. From your build output, it looks like Open MPI did not configure, build, or install the TM support as you requested. Is there a problem with the TM support that you don't want to use it? Regardless, what I'm guessing you're seeing is remnants of a prior Open MPI installation that is causing a problem. You typically see the warning message that you're seeing when there is an incompatible plugin in the directory where Open MPI stores its components (e.g., left over from a prior build/install). In this case, I'm guessing there's a TM plugin in the installation there that is refusing to open for some reason (e.g., built for a different architecture, a shared library dependency can't be found, etc.). To confirm this you can probably run: ls /lib/openmpi/*tm* where is wherever you installed OMPI (possibly /usr/projects/ hpctools/jennyp/openmpi124flash/openmpi-1.2.4?). If you see any *tm* files in that directory, there's a good chance that they're left over from a prior install. The easiest thing to do is probably to wipe out the entire installation directory and install again. I can suggest less drastic things to do if you care, but I'm guessing this simple solution will be good enough. The system is gm, but I can't get the version right away. I don't have permissions. It is coming as soon as the systems people tell me what it is. This issue doesn't seem to be gm related. I can't help you with gm networks... Others may be able to. But I agree that it doesn't look like a gm issue; more of a startup issue. Thanks, Jenny -- - Jennis (Jenny) Los Alamos National Laboratory 505-667-1955 -Original Message- From: devel-boun...@open-mpi.org [mailto:devel-boun...@open-mpi.org] On Behalf Of Jennis Pruett Sent: Wednesday, October 17, 2007 1:49 PM To: de...@open-mpi.org Subject: [OMPI devel] issue with --without-tm in configure? Hello, All, I'm new to this forum, but I'm told this is where I'm to ask questions. I have a linux cluster, bproc, gm, and am trying to configure version 1.2.4. This is my configure command line: ./configure --prefix=/usr/projects/hpctools/jennyp/openmpi124flash/openmpi-1.2.4 --libdir=/usr/projects/hpctools/jennyp/openmpi124flash/openmpi -1.2.4/lib64 --with-bproc --without-tm --enable-shared --with-io_romio_flags=--with-file-system=ufs+nfs+panfs --without-pty_support ( I threw in the --without-pty-support just to see if it would make a difference.) The make and install seem to proced ok. The test I have is an IMB-MPI1 test, and I am getting these warnings, no matter what compiler I m using.: mpirun -n 6 ./IMB-MPI1 [n110:26208] mca: base: component_find: unable to open ras tm: file not found (ignored) [n110:26208] mca: base: component_find: unable to open pls tm: file not found (ignored) [n111:26215] mca: base: component_find: unable to open ras tm: file not found (ignored) [n111:26212] mca: base: component_find: unable to open ras tm: file not found (ignored) [n112:26216] mca: base: component_find: unable to open ras tm: file not found (ignored) [n112:26213] mca: base: component_find: unable to open ras tm: file not found (ignored) [n110:26211] mca: base: component_find: unable to open ras tm: file not found (ignored) [n110:26214] mca: base: component_find: unable to open ras tm: file not found (ignored) [n111:26212] mca: base: component_find: unable to open pls tm: file not found (ignored) [n111:26215] mca: base: component_find: unable to open pls tm: file not found (ignored) [n112:26213] mca: base: component_find: unable to open pls tm: file not found (ignored) [n110:26211] mca: base: component_find: unable to open pls tm: file not found (ignored) [n110:26214] mca: base: component_find: unable to open pls tm: file not found (ignored) [n112:26216] mca: base: component_find: unable to open pls tm: file not found (ignored) I was told today that the --without-tm parameter is being ignored in the configure statement. Anyone know what is going on? Thanks, -- - Jennis (Jenny) Los Alamos National Laboratory 505-667-1955 ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel WARNING: The virus scanner was unable to scan the next attachment. This att
