Gilles,
The design of the BTL move was to let the opal_process_name_t be agnostic to
what is stored inside, and all accesses should be done through the provided
accessors. Thus, big endian or little endian doesn’t make a difference, as long
as everything goes through the accessors.
I’m skeptical about the support of heterogeneous environments in the current
code, so I didn’t pay much attention to handling the case in the TCP BTL. But
in case we do care it is enough to make the 2 macros point to something
meaningful instead of being empty (bswap_64 or something).
George.
On Aug 1, 2014, at 06:52 , Gilles Gouaillardet <gilles.gouaillar...@iferc.org>
wrote:
> George and Ralph,
>
> i am very confused whether there is an issue or not.
>
>
> anyway, today Paul and i ran basic tests on big endian machines and did not
> face any issue related to big endianness.
>
> so i made my homework, digged into the code, and basically,
> opal_process_name_t is used as an orte_process_name_t.
> for example, in ompi_proc_init :
>
> OMPI_CAST_ORTE_NAME(&proc->super.proc_name)->jobid = OMPI_PROC_MY_NAME->jobid;
> OMPI_CAST_ORTE_NAME(&proc->super.proc_name)->vpid = i;
>
> and with
>
> #define OMPI_CAST_ORTE_NAME(a) ((orte_process_name_t*)(a))
>
> so as long as an opal_process_name_t is used as an orte_process_name_t, there
> is no problem,
> regardless the endianness of the homogenous cluster we are running on.
>
> for the sake of readability (and for being pedantic too ;-) ) in r32357,
> &proc_temp->super.proc_name
> could be replaced with
> OMPI_CAST_ORTE_NAME(&proc_temp->super.proc_name)
>
>
>
> That being said, in btl/tcp, i noticed :
>
> in mca_btl_tcp_component_recv_handler :
>
> opal_process_name_t guid;
> [...]
> /* recv the process identifier */
> retval = recv(sd, (char *)&guid, sizeof(guid), 0);
> if(retval != sizeof(guid)) {
> CLOSE_THE_SOCKET(sd);
> return;
> }
> OPAL_PROCESS_NAME_NTOH(guid);
>
> and in mca_btl_tcp_endpoint_send_connect_ack :
>
> /* send process identifier to remote endpoint */
> opal_process_name_t guid = btl_proc->proc_opal->proc_name;
>
> OPAL_PROCESS_NAME_HTON(guid);
> if(mca_btl_tcp_endpoint_send_blocking(btl_endpoint, &guid, sizeof(guid))
> !=
>
> and with
>
> #define OPAL_PROCESS_NAME_NTOH(guid)
> #define OPAL_PROCESS_NAME_HTON(guid)
>
>
> i had no time yet to test yet, but for now, i can only suspect :
> - there will be an issue with the tcp btl on an heterogeneous cluster
> - for this case, the fix is to have a different version of the
> OPAL_PROCESS_NAME_xTOy
> on little endian arch if heterogeneous mode is supported.
>
>
>
> does that make sense ?
>
> Cheers,
>
> Gilles
>
>
> On 2014/07/31 1:29, George Bosilca wrote:
>> The underlying structure changed, so a little bit of fiddling is normal.
>> Instead of using a field in the ompi_proc_t you are now using a field down
>> in opal_proc_t, a field that simply cannot have the same type as before
>> (orte_process_name_t).
>>
>> George.
>>
>>
>>
>> On Wed, Jul 30, 2014 at 12:19 PM, Ralph Castain <r...@open-mpi.org> wrote:
>>
>>> George - my point was that we regularly tested using the method in that
>>> routine, and now we have to do something a little different. So it is an
>>> "issue" in that we have to make changes across the code base to ensure we
>>> do things the "new" way, that's all
>>>
>>> On Jul 30, 2014, at 9:17 AM, George Bosilca <bosi...@icl.utk.edu> wrote:
>>>
>>> No, this is not going to be an issue if the opal_identifier_t is used
>>> correctly (aka only via the exposed accessors).
>>>
>>> George.
>>>
>>>
>>>
>>> On Wed, Jul 30, 2014 at 12:09 PM, Ralph Castain <r...@open-mpi.org> wrote:
>>>
>>>> Yeah, my fix won't work for big endian machines - this is going to be an
>>>> issue across the code base now, so we'll have to troll and fix it. I was
>>>> doing the minimal change required to fix the trunk in the meantime.
>>>>
>>>> On Jul 30, 2014, at 9:06 AM, George Bosilca <bosi...@icl.utk.edu> wrote:
>>>>
>>>> Yes. opal_process_name_t has basically no meaning by itself, it is a 64
>>>> bits storage location used by the upper layer to save some local key that
>>>> can be later used to extract information. Calling the OPAL level compare
>>>> function might be a better fit there.
>>>>
>>>> George.
>>>>
>>>>
>>>>
>>>> On Wed, Jul 30, 2014 at 11:50 AM, Gilles Gouaillardet <
>>>> gilles.gouaillar...@gmail.com> wrote:
>>>>
>>>>> Ralph,
>>>>>
>>>>> was it really that simple ?
>>>>>
>>>>> proc_temp->super.proc_name has type opal_process_name_t :
>>>>> typedef opal_identifier_t opal_process_name_t;
>>>>> typedef uint64_t opal_identifier_t;
>>>>>
>>>>> *but*
>>>>>
>>>>> item_ptr->peer has type orte_process_name_t :
>>>>> struct orte_process_name_t {
>>>>> orte_jobid_t jobid;
>>>>> orte_vpid_t vpid;
>>>>> };
>>>>>
>>>>> bottom line, is r32357 still valid on a big endian arch ?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Gilles
>>>>>
>>>>>
>>>>> On Wed, Jul 30, 2014 at 11:49 PM, Ralph Castain <r...@open-mpi.org>
>>>>> wrote:
>>>>>
>>>>>> I just fixed this one - all that was required was an ampersand as the
>>>>>> name was being passed into the function instead of a pointer to the name
>>>>>>
>>>>>> r32357
>>>>>>
>>>>>> On Jul 30, 2014, at 7:43 AM, Gilles GOUAILLARDET <
>>>>>> gilles.gouaillar...@gmail.com> wrote:
>>>>>>
>>>>>> Rolf,
>>>>>>
>>>>>> r32353 can be seen as a suspect...
>>>>>> Even if it is correct, it might have exposed the bug discussed in #4815
>>>>>> even more (e.g. we hit the bug 100% after the fix)
>>>>>>
>>>>>> does the attached patch to #4815 fixes the problem ?
>>>>>>
>>>>>> If yes, and if you see this issue as a showstopper, feel free to commit
>>>>>> it and drop a note to #4815
>>>>>> ( I am afk until tomorrow)
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Gilles
>>>>>>
>>>>>> Rolf vandeVaart <rvandeva...@nvidia.com> wrote:
>>>>>>
>>>>>> Just an FYI that my trunk version (r32355) does not work at all anymore
>>>>>> if I do not include "--mca coll ^ml". Here is a stack trace from the
>>>>>> ibm/pt2pt/send test running on a single node.
>>>>>>
>>>>>>
>>>>>>
>>>>>> (gdb) where
>>>>>>
>>>>>> #0 0x00007f6c0d1321d0 in ?? ()
>>>>>>
>>>>>> #1 <signal handler called>
>>>>>>
>>>>>> #2 0x00007f6c183abd52 in orte_util_compare_name_fields (fields=15
>>>>>> '\017', name1=0x192350001, name2=0xbaf76c) at
>>>>>> ../../orte/util/name_fns.c:522
>>>>>>
>>>>>> #3 0x00007f6c0bea17be in bcol_basesmuma_smcm_allgather_connection
>>>>>> (sm_bcol_module=0x7f6bf3b68040, module=0xb3d200,
>>>>>> peer_list=0x7f6c0c0a6748,
>>>>>> back_files=0x7f6bf3ffd6c8,
>>>>>>
>>>>>> comm=0x6037a0, input=..., base_fname=0x7f6c0bea2606
>>>>>> "sm_payload_mem_", map_all=false) at
>>>>>> ../../../../../ompi/mca/bcol/basesmuma/bcol_basesmuma_smcm.c:237
>>>>>>
>>>>>> #4 0x00007f6c0be98307 in bcol_basesmuma_bank_init_opti
>>>>>> (payload_block=0xbc0f60, data_offset=64, bcol_module=0x7f6bf3b68040,
>>>>>> reg_data=0xba28c0)
>>>>>>
>>>>>> at
>>>>>> ../../../../../ompi/mca/bcol/basesmuma/bcol_basesmuma_buf_mgmt.c:302
>>>>>>
>>>>>> #5 0x00007f6c0cced386 in mca_coll_ml_register_bcols
>>>>>> (ml_module=0xba5c40) at
>>>>>> ../../../../../ompi/mca/coll/ml/coll_ml_module.c:510
>>>>>>
>>>>>> #6 0x00007f6c0cced68f in ml_module_memory_initialization
>>>>>> (ml_module=0xba5c40) at
>>>>>> ../../../../../ompi/mca/coll/ml/coll_ml_module.c:558
>>>>>>
>>>>>> #7 0x00007f6c0ccf06b1 in ml_discover_hierarchy (ml_module=0xba5c40) at
>>>>>> ../../../../../ompi/mca/coll/ml/coll_ml_module.c:1539
>>>>>>
>>>>>> #8 0x00007f6c0ccf4e0b in mca_coll_ml_comm_query (comm=0x6037a0,
>>>>>> priority=0x7fffe7991b58) at
>>>>>> ../../../../../ompi/mca/coll/ml/coll_ml_module.c:2963
>>>>>>
>>>>>> #9 0x00007f6c18cc5b09 in query_2_0_0 (component=0x7f6c0cf50940,
>>>>>> comm=0x6037a0, priority=0x7fffe7991b58, module=0x7fffe7991b90)
>>>>>>
>>>>>> at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:372
>>>>>>
>>>>>> #10 0x00007f6c18cc5ac8 in query (component=0x7f6c0cf50940,
>>>>>> comm=0x6037a0, priority=0x7fffe7991b58, module=0x7fffe7991b90)
>>>>>>
>>>>>> at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:355
>>>>>>
>>>>>> #11 0x00007f6c18cc59d2 in check_one_component (comm=0x6037a0,
>>>>>> component=0x7f6c0cf50940, module=0x7fffe7991b90)
>>>>>>
>>>>>> at ../../../../ompi/mca/coll/base/coll_base_comm_select.c:317
>>>>>>
>>>>>> #12 0x00007f6c18cc5818 in check_components (components=0x7f6c18f46ef0,
>>>>>> comm=0x6037a0) at
>>>>>> ../../../../ompi/mca/coll/base/coll_base_comm_select.c:281
>>>>>>
>>>>>> #13 0x00007f6c18cbe3c9 in mca_coll_base_comm_select (comm=0x6037a0) at
>>>>>> ../../../../ompi/mca/coll/base/coll_base_comm_select.c:117
>>>>>>
>>>>>> #14 0x00007f6c18c52301 in ompi_mpi_init (argc=1, argv=0x7fffe79924c8,
>>>>>> requested=0, provided=0x7fffe79922e8) at
>>>>>> ../../ompi/runtime/ompi_mpi_init.c:918
>>>>>>
>>>>>> #15 0x00007f6c18c86e92 in PMPI_Init (argc=0x7fffe799234c,
>>>>>> argv=0x7fffe7992340) at pinit.c:84
>>>>>>
>>>>>> #16 0x0000000000401056 in main (argc=1, argv=0x7fffe79924c8) at
>>>>>> send.c:32
>>>>>>
>>>>>> (gdb) up
>>>>>>
>>>>>> #1 <signal handler called>
>>>>>>
>>>>>> (gdb) up
>>>>>>
>>>>>> #2 0x00007f6c183abd52 in orte_util_compare_name_fields (fields=15
>>>>>> '\017', name1=0x192350001, name2=0xbaf76c) at
>>>>>> ../../orte/util/name_fns.c:522
>>>>>>
>>>>>> 522 if (name1->jobid < name2->jobid) {
>>>>>>
>>>>>> (gdb) print name1
>>>>>>
>>>>>> $1 = (const orte_process_name_t *) 0x192350001
>>>>>>
>>>>>> (gdb) print *name1
>>>>>>
>>>>>> Cannot access memory at address 0x192350001
>>>>>>
>>>>>> (gdb) print name2
>>>>>>
>>>>>> $2 = (const orte_process_name_t *) 0xbaf76c
>>>>>>
>>>>>> (gdb) print *name2
>>>>>>
>>>>>> $3 = {jobid = 2452946945, vpid = 1}
>>>>>>
>>>>>> (gdb)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: devel [mailto:devel-boun...@open-mpi.org
>>>>>> <devel-boun...@open-mpi.org>] On Behalf Of Gilles
>>>>>>
>>>>>>> Gouaillardet
>>>>>>> Sent: Wednesday, July 30, 2014 2:16 AM
>>>>>>> To: Open MPI Developers
>>>>>>> Subject: Re: [OMPI devel] trunk compilation errors in jenkins
>>>>>>> George,
>>>>>>> #4815 is indirectly related to the move :
>>>>>>> in bcol/basesmuma, we used to compare ompi_process_name_t, and now
>>>>>>> we (try to) compare an ompi_process_name_t and an opal_process_name_t
>>>>>>> (which causes a glory SIGSEGV)
>>>>>>> i proposed a temporary patch which is both broken and unelegant, could
>>>>>> you
>>>>>>
>>>>>>> please advise a correct solution ?
>>>>>>> Cheers,
>>>>>>> Gilles
>>>>>>> On 2014/07/27 7:37, George Bosilca wrote:
>>>>>>>> If you have any issue with the move, I’ll be happy to help and/or
>>>>>> support
>>>>>>
>>>>>>> you on your last move toward a completely generic BTL. To facilitate
>>>>>> your
>>>>>>
>>>>>>> work I exposed a minimalistic set of OMPI information at the OPAL
>>>>>> level. Take
>>>>>>
>>>>>>> a look at opal/util/proc.h for more info, but please try not to expose
>>>>>> more.
>>>>>>
>>>>>>> _______________________________________________
>>>>>>> devel mailing list
>>>>>>> de...@open-mpi.org
>>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>>>>> Link to this post: http://www.open-
>>>>>> <http://www.open-mpi.org/community/lists/devel/2014/07/15348.php>
>>>>>>
>>>>>>> mpi.org/community/lists/devel/2014/07/15348.php
>>>>>> <http://www.open-mpi.org/community/lists/devel/2014/07/15348.php>
>>>>>> ------------------------------
>>>>>> This email message is for the sole use of the intended recipient(s)
>>>>>> and may contain confidential information. Any unauthorized review, use,
>>>>>> disclosure or distribution is prohibited. If you are not the intended
>>>>>> recipient, please contact the sender by reply email and destroy all
>>>>>> copies
>>>>>> of the original message.
>>>>>> ------------------------------
>>>>>> _______________________________________________
>>>>>> devel mailing list
>>>>>> de...@open-mpi.org
>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>>>> Link to this post:
>>>>>> http://www.open-mpi.org/community/lists/devel/2014/07/15355.php
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> devel mailing list
>>>>>> de...@open-mpi.org
>>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>>>> Link to this post:
>>>>>> http://www.open-mpi.org/community/lists/devel/2014/07/15356.php
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> devel mailing list
>>>>> de...@open-mpi.org
>>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>>> Link to this post:
>>>>> http://www.open-mpi.org/community/lists/devel/2014/07/15363.php
>>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> de...@open-mpi.org
>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>> Link to this post:
>>>> http://www.open-mpi.org/community/lists/devel/2014/07/15364.php
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> de...@open-mpi.org
>>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>> Link to this post:
>>>> http://www.open-mpi.org/community/lists/devel/2014/07/15365.php
>>>>
>>> _______________________________________________
>>> devel mailing list
>>> de...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/devel/2014/07/15366.php
>>>
>>>
>>>
>>> _______________________________________________
>>> devel mailing list
>>> de...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>> Link to this post:
>>> http://www.open-mpi.org/community/lists/devel/2014/07/15367.php
>>>
>>
>>
>> _______________________________________________
>> devel mailing list
>> de...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post:
>> http://www.open-mpi.org/community/lists/devel/2014/07/15368.php
>
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/08/15446.php
