Re: [OMPI devel] "Open MPI"-based MPI library used by K computer

[Kawashima, Takahiro]

Hi,

Sorry for not replying sooner.
I'm taliking with the authors (they are not in this list) and
will request linking the PDF soon if they allowed.

Takahiro Kawashima,
MPI development team,
Fujitsu

> Our policy so far was that adding a paper to the list of publication on the 
> Open MPI website was a discretionary action at the authors' request. I don't 
> see any compelling reason to change. Moreover, Fujitsu being a contributor of 
> the Open MPI community, there is no obstacle of adding a link to their paper 
> -- at their request.
> 
>   George.
> 
> On Jan 10, 2013, at 00:15 , Rayson Ho  wrote:
> 
> > Hi Ralph,
> > 
> > Since the whole journal is available online, and is reachable by
> > Google, I don't believe we can get into copyright issues by providing
> > a link to it (but then, I also know that there are countries that have
> > more crazy web page linking rules!).
> > 
> > http://www.fujitsu.com/global/news/publications/periodicals/fstj/archives/vol48-3.html
> > 
> > Rayson
> > 
> > ==
> > Open Grid Scheduler - The Official Open Source Grid Engine
> > http://gridscheduler.sourceforge.net/
> > 
> > Scalable Cloud HPC: 10,000-node OGS/GE Amazon EC2 cluster
> > http://blogs.scalablelogic.com/2012/11/running-1-node-grid-engine-cluster.html
> > 
> > 
> > On Thu, Sep 20, 2012 at 6:46 AM, Ralph Castain  wrote:
> >> I'm unaware of any formal criteria. The papers currently located there are 
> >> those written by members of the OMPI community, but we can certainly link 
> >> to something written by someone else, so long as we don't get into 
> >> copyright issues.
> >> 
> >> On Sep 19, 2012, at 11:57 PM, Rayson Ho  wrote:
> >> 
> >>> I found this paper recently, "MPI Library and Low-Level Communication
> >>> on the K computer", available at:
> >>> 
> >>> http://www.fujitsu.com/downloads/MAG/vol48-3/paper11.pdf
> >>> 
> >>> What are the criteria for adding papers to the "Open MPI Publications" 
> >>> page?
> >>> 
> >>> Rayson
> >>> 
> >>> ==
> >>> Open Grid Scheduler - The Official Open Source Grid Engine
> >>> http://gridscheduler.sourceforge.net/
> >>> 
> >>> 
> >>> On Fri, Nov 18, 2011 at 5:32 AM, George Bosilca  
> >>> wrote:
>  Dear Yuki and Takahiro,
>  
>  Thanks for the bug report and for the patch. I pushed a [nearly 
>  identical] patch in the trunk in 
>  https://svn.open-mpi.org/trac/ompi/changeset/25488. A special version 
>  for the 1.4 has been prepared and has been attached to the ticket #2916 
>  (https://svn.open-mpi.org/trac/ompi/ticket/2916).
>  
>  Thanks,
>  george.
>  
>  
>  On Nov 14, 2011, at 02:27 , Y.MATSUMOTO wrote:
>  
> > Dear Open MPI community,
> > 
> > I'm a member of MPI library development team in Fujitsu,
> > Takahiro Kawashima, who sent mail before, is my colleague.
> > We start to feed back.
> > 
> > First, we fixed about MPI_LB/MPI_UB and data packing problem.
> > 
> > Program crashes when it meets all of the following conditions:
> > a: The type of sending data is contiguous and derived type.
> > b: Either or both of MPI_LB and MPI_UB is used in the data type.
> > c: The size of sending data is smaller than extent(Data type has gap).
> > d: Send-count is bigger than 1.
> > e: Total size of data is bigger than "eager limit"
> > 
> > This problem occurs in attachment C program.
> > 
> > An incorrect-address accessing occurs
> > because an unintended value of "done" inputs and
> > the value of "max_allowd" becomes minus
> > in the following place in "ompi/datatype/datatype_pack.c(in version 
> > 1.4.3)".
> > 
> > 
> > (ompi/datatype/datatype_pack.c)
> > 188 packed_buffer = (unsigned char *) 
> > iov[iov_count].iov_base;
> > 189 done = pConv->bConverted - i * pData->size;  /* partial 
> > data from last pack */
> > 190 if( done != 0 ) {  /* still some data to copy from the 
> > last time */
> > 191 done = pData->size - done;
> > 192 OMPI_DDT_SAFEGUARD_POINTER( user_memory, done, 
> > pConv->pBaseBuf, pData, pConv->count );
> > 193 MEMCPY_CSUM( packed_buffer, user_memory, done, 
> > pConv );
> > 194 packed_buffer += done;
> > 195 max_allowed -= done;
> > 196 total_bytes_converted += done;
> > 197 user_memory += (extent - pData->size + done);
> > 198 }
> > 
> > This program assumes "done" as the size of partial data from last pack.
> > However, when the program crashes, "done" equals the sum of all 
> > transmitted data size.
> > It makes "max_allowed" to be a negative value.
> > 
> 

Re: [OMPI devel] Build open MPI

[Jeff Squyres (jsquyres)]

Check the HACKING file in the top-level directory if you need some assistance 
on how to upgrade your Autoconf/Automake/Libtool.


On Jan 9, 2013, at 9:27 PM, Ralph Castain 
 wrote:

> I'm pretty sure we are at autoconf 2.69 now. You might want to upgrade it, 
> and ensure your m4 is correspondingly updated. Also, automake should probably 
> be at 1.12.x (avoid 1.13,x as it has bugs). I think libtool looks pretty old 
> too.
> 
> Sent from my iPad
> 
> On Jan 9, 2013, at 5:37 PM, "Ding, Boxiong"  wrote:
> 
>> Hi,
>> 
>> I am trying to build the code that Ralph has put here: 
>> https://boxd...@bitbucket.org/rhc/hdmon, but failed. It is a modified open 
>> MPI code. Can someone help?
>> 
>> [root@aesaroyp1d1c hdmon]# cat /etc/redhat-release 
>> Red Hat Enterprise Linux Server release 6.1 (Santiago)
>> 
>> I have manually installed m4/autoconf/automake/libtool on my local directory 
>> and the versions match those specified in HACKING.
>> [root@aesaroyp1d1c lib]# pwd
>> /root/local/lib
>> [root@aesaroyp1d1c lib]# ls
>> autoconf-2.68  automake-1.11.1  libtool-2.2.8  m4-1.4.13
>> 
>> [root@aesaroyp1d1c lib]# which m4
>> /root/local/lib/m4-1.4.13/bin/m4
>> [root@aesaroyp1d1c lib]# which autoconf
>> /root/local/lib/autoconf-2.68/bin/autoconf
>> [root@aesaroyp1d1c lib]# which automake
>> /root/local/lib/automake-1.11.1/bin/automake
>> [root@aesaroyp1d1c lib]# which libtool
>> /root/local/lib/libtool-2.2.8/bin/libtool
>> 
>> When I run autogen.pl I got the following error:
>> 
>> 6. Processing autogen.subdirs directories
>> 
>> === Processing subdir: 
>> /root/workspace/hdmon/opal/mca/event/libevent2019/libevent
>> --- Found autogen.sh; running...
>> Running: ./autogen.sh
>> autoreconf: Entering directory `.'
>> autoreconf: configure.in: not using Gettext
>> autoreconf: running: aclocal -I .. --force -I m4
>> autoreconf: configure.in: tracing
>> autoreconf: configure.in: not using Libtool
>> autoreconf: running: /root/local/lib/autoconf-2.68/bin/autoconf --include=.. 
>> --force
>> configure.in:146: error: possibly undefined macro: AC_PROG_LIBTOOL
>>   If this token and others are legitimate, please use m4_pattern_allow.
>>   See the Autoconf documentation.
>> autoreconf: /root/local/lib/autoconf-2.68/bin/autoconf failed with exit 
>> status: 1
>> Command failed: ./autogen.sh
>> 
>> 
>> Thanks,
>> Boxiong
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] [OMPI users] Backward Compatibility of MPI Java Binding

[Yoshiki SATO]

> Yes -- IIRC, the Java interface isn't really dependent upon anything specific 
> in the back-end C implementation of Open MPI.  So I'm guessing/assuming that 
> if you can build it, it should work against the 1.6 OMPI C engine just fine.

Sounds good :-)  I'm going to try to build it anyway, and let you know if 
stucked.

> Is MPI-for-Java of interest to you?  I ask because Ralph and I have been 
> trying to figure out how to get the cycles to do more with the Java interface 
> (e.g., make it more like a 1:1 set of bindings).  Is this something you'd be 
> willing to work on / contribute, perchance?

Yes.  This is because one of our roles is enabling Java applications to run on 
top of the K computer (Fujitsu), which prevents user processes from TCP/IP 
communications directly.  It only allows communications via customized openmpi 
(scheduler) between inter node communications.  BTW, I don't fully understand 
what a 1:1 set of bindings mean.  I believe that the interfaces defined in 
mpiJava was carefully designed to match the MPI spec, and then should be a 1:1 
set of bindings.  Otherwise, I'm willing to contribute something to make it 
better ;-)

Thanks,
--yoshiki


On 2013/01/10, at 20:45, Jeff Squyres (jsquyres) wrote:

> On Jan 9, 2013, at 10:30 PM, Yoshiki SATO 
> wrote:
> 
>> The 1.7's Java implementation under ompi/mpi/java seem to be able to build 
>> up independently.  Do you think we can build just them and run it (via 
>> prunjava?) with our custom OpenMPI build based on 1.6?
> 
> 
> Yes -- IIRC, the Java interface isn't really dependent upon anything specific 
> in the back-end C implementation of Open MPI.  So I'm guessing/assuming that 
> if you can build it, it should work against the 1.6 OMPI C engine just fine.
> 
> Is MPI-for-Java of interest to you?  I ask because Ralph and I have been 
> trying to figure out how to get the cycles to do more with the Java interface 
> (e.g., make it more like a 1:1 set of bindings).  Is this something you'd be 
> willing to work on / contribute, perchance?
> 
> -- 
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to: 
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [hwloc-devel] [RFC] deprecating multiple load() support

[Jeff Squyres (jsquyres)]

+1

This sounds fine to me.


On Jan 3, 2013, at 2:46 PM, Brice Goglin  wrote:

> Hello,
> 
> hwloc users are supposed to configure/build topologies like this:
>  hwloc_topology_init()
>  /* all configuration calls */
>  hwloc_topology_set_xml()
>  hwloc_topology_set_flags()
>  /* end of all configuration calls */
>  hwloc_topology_load() /* load from XML with flags enabled */
> 
> We actually doing things like:
>  hwloc_topology_init()
>  hwloc_topology_set_xml()
>  hwloc_topology_load() /* load from XML */
>  hwloc_topology_set_flags()
>  hwloc_topology_load() /* reload from XML with flags enabled */
> 
> That's convenient in some corner cases, but it is causing some headache
> for maintenance. And it's not documented as far as I looked. So I am
> willing to remove this feature. Users will only be allowed to do a
> single load() per init(), and do set_foo() only between init() and this
> single load(). That's what basically everybody does, and what the
> documentation says.
> 
> Of course, using different topologies will still be allowed.
> 
> 
> As a side-effect, tools such as hwloc-calc and hwloc-bind will no longer
> let you mix options and locations in the command-line. You will have to
> specify the input topology and flags at the beginning of the
> command-line and then specify all locations within that topology.
> 
> 
> If anybody cares, please speakup now! The plan is to remove these
> "features" in v1.8 (could be released in 2013Q3). The delay isn't long
> but nobody is likely using these features anyway. In the meantime
> (starting in v1.6.1), we'll issue a warning if somebody ever uses them.
> 
> Brice
> 
> ___
> hwloc-devel mailing list
> hwloc-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] use of stat() during malloc initialization

[Jeff Squyres (jsquyres)]

Committed in https://svn.open-mpi.org/trac/ompi/changeset/27785, and I filed 
CMRs to get this fix in 1.6.4 and 1.7.


On Jan 10, 2013, at 9:23 AM, Phil Carns 
 wrote:

> Thanks Jeff.  I tested the patch just now using Open MPI SVN trunk revision 
> 27784.  I was able to instrument an application without any trouble at all, 
> and the patch looks great.
> 
> I definitely understand the memory registration cache pain.  I've dabbled in 
> network abstractions for file systems in the past, and it's disappointing but 
> not terribly surprising that this is still the state of affairs :)  
> 
> Thanks for addressing this so quickly.  This will definitely make life easier 
> for some Darshan and Open MPI users in the future.
> 
> -Phil
> 
> On 01/09/2013 04:24 PM, Jeff Squyres (jsquyres) wrote:
>> Greetings Phil.  Good analysis.
>> 
>> You can thank OFED for this horribleness, BTW.  :-)  Since OFED hardware 
>> requires memory registration, and since that registration is expensive, MPI 
>> implementations cache registered memory to alleviate the re-registration 
>> costs for repeated memory usage.  But MPI doesn't allocate user buffers, so 
>> MPI doesn't get notified when users free their buffers, meaning that MPI's 
>> internal cache gets out of sync with reality.  Hence, MPI implementations 
>> are forced to do horrid workaround like you found to find out when 
>> applications free buffers that may be cached.  Ugh.  Go knock your local 
>> OFED developer and tell them to give us a notification mechanism so that we 
>> don't have to do these horrid workarounds.  :-)
>> 
>> Regardless, I think your suggestion is fine (replace stat with access).
>> 
>> Can you confirm that the attached patch works for you?
>> 
>> 
>> On Jan 9, 2013, at 10:49 AM, Phil Carns 
>> 
>> 
>>  wrote:
>> 
>> 
>>> Hi,
>>> 
>>> I am a developer on the Darshan project (
>>> http://www.mcs.anl.gov/darshan
>>> ), which provides a set of lightweight wrappers to characterize the I/O 
>>> access patterns of MPI applications.  Darshan can operate on static or 
>>> dynamic executables.  As you might expect, it uses the LD_PRELOAD mechanism 
>>> to intercept I/O calls like open(), read(), write() and stat() on dynamic 
>>> executables.
>>> 
>>> We recently received an unusual bug report (courtesy of Myriam Botalla) 
>>> when Darshan is used in LD_PRELOAD mode with Open MPI 1.6.3, however. When 
>>> Darshan intercepts a function call via LD_PRELOAD, it must use dlsym() to 
>>> locate the "real" underlying function to invoke.  dlsym() in turn uses the 
>>> calloc() function internally.  In most cases this is fine, but Open MPI 
>>> actually makes its first stat() call within the malloc initialization hook 
>>> (opal_memory_linux_malloc_init_hook()) before the malloc() and its related 
>>> functions have been configured.  Darshan therefore (indirectly) triggers a 
>>> segfault because it intercepts those stat() calls but can't find the real 
>>> stat() function without using malloc.
>>> 
>>> There is some more detailed information about this issue, including a stack 
>>> trace, in this mailing list thread:
>>> 
>>> 
>>> http://lists.mcs.anl.gov/pipermail/darshan-users/2013-January/000131.html
>>> 
>>> 
>>> Looking a little more closely at the opal_memory_linux_malloc_init_hook() 
>>> function, it looks like the struct stat output argument from stat() is 
>>> being ignored in all cases.  Open MPI is just checking the stat() return 
>>> code to determine if the files in question exist or not.  Taking that into 
>>> account, would it be possible to make a minor change in Open MPI to replace 
>>> these instances:
>>> 
>>>stat("some_filename", )
>>> 
>>> with:
>>> 
>>>access("some_filename", F_OK)
>>> 
>>> in the opal_memory_linux_malloc_init_hook() function?  There is a slight 
>>> technical advantage to the change in that access() is lighter weight than 
>>> stat() on some systems (and it might arguably make the intent  of the calls 
>>> a little clearer), but of course my main motivation here is to have Open 
>>> MPI use a function that is less likely to be intercepted by I/O tracing 
>>> tools before a malloc implementation has been initialized.  Technically it 
>>> is possible to work around this in Darshan itself by checking the arguments 
>>> passed in to stat() and using a workaround path for this case, but this 
>>> isn't a very safe solution in the long run.
>>> 
>>> Thanks in advance for your time and consideration,
>>> -Phil
>>> ___
>>> devel mailing list
>>> 
>>> de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> 
>> 
>> 
>> ___
>> devel mailing list
>> 
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff 

Re: [OMPI devel] use of stat() during malloc initialization

[Phil Carns]

Thanks Jeff.  I tested the patch just now using Open MPI SVN trunk 
revision 27784.  I was able to instrument an application without any 
trouble at all, and the patch looks great.


I definitely understand the memory registration cache pain.  I've 
dabbled in network abstractions for file systems in the past, and it's 
disappointing but not terribly surprising that this is still the state 
of affairs :)


Thanks for addressing this so quickly.  This will definitely make life 
easier for some Darshan and Open MPI users in the future.


-Phil

On 01/09/2013 04:24 PM, Jeff Squyres (jsquyres) wrote:

Greetings Phil.  Good analysis.

You can thank OFED for this horribleness, BTW.  :-)  Since OFED hardware 
requires memory registration, and since that registration is expensive, MPI 
implementations cache registered memory to alleviate the re-registration costs 
for repeated memory usage.  But MPI doesn't allocate user buffers, so MPI 
doesn't get notified when users free their buffers, meaning that MPI's internal 
cache gets out of sync with reality.  Hence, MPI implementations are forced to 
do horrid workaround like you found to find out when applications free buffers 
that may be cached.  Ugh.  Go knock your local OFED developer and tell them to 
give us a notification mechanism so that we don't have to do these horrid 
workarounds.  :-)

Regardless, I think your suggestion is fine (replace stat with access).

Can you confirm that the attached patch works for you?


On Jan 9, 2013, at 10:49 AM, Phil Carns 
  wrote:


Hi,

I am a developer on the Darshan project (http://www.mcs.anl.gov/darshan), which 
provides a set of lightweight wrappers to characterize the I/O access patterns 
of MPI applications.  Darshan can operate on static or dynamic executables.  As 
you might expect, it uses the LD_PRELOAD mechanism to intercept I/O calls like 
open(), read(), write() and stat() on dynamic executables.

We recently received an unusual bug report (courtesy of Myriam Botalla) when Darshan is 
used in LD_PRELOAD mode with Open MPI 1.6.3, however. When Darshan intercepts a function 
call via LD_PRELOAD, it must use dlsym() to locate the "real" underlying 
function to invoke.  dlsym() in turn uses the calloc() function internally.  In most 
cases this is fine, but Open MPI actually makes its first stat() call within the malloc 
initialization hook (opal_memory_linux_malloc_init_hook()) before the malloc() and its 
related functions have been configured.  Darshan therefore (indirectly) triggers a 
segfault because it intercepts those stat() calls but can't find the real stat() function 
without using malloc.

There is some more detailed information about this issue, including a stack 
trace, in this mailing list thread:

http://lists.mcs.anl.gov/pipermail/darshan-users/2013-January/000131.html

Looking a little more closely at the opal_memory_linux_malloc_init_hook() 
function, it looks like the struct stat output argument from stat() is being 
ignored in all cases.  Open MPI is just checking the stat() return code to 
determine if the files in question exist or not.  Taking that into account, 
would it be possible to make a minor change in Open MPI to replace these 
instances:

stat("some_filename", )

with:

access("some_filename", F_OK)

in the opal_memory_linux_malloc_init_hook() function?  There is a slight 
technical advantage to the change in that access() is lighter weight than 
stat() on some systems (and it might arguably make the intent  of the calls a 
little clearer), but of course my main motivation here is to have Open MPI use 
a function that is less likely to be intercepted by I/O tracing tools before a 
malloc implementation has been initialized.  Technically it is possible to work 
around this in Darshan itself by checking the arguments passed in to stat() and 
using a workaround path for this case, but this isn't a very safe solution in 
the long run.

Thanks in advance for your time and consideration,
-Phil
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] [OMPI users] Backward Compatibility of MPI Java Binding

[Jeff Squyres (jsquyres)]

On Jan 9, 2013, at 10:30 PM, Yoshiki SATO 
 wrote:

> The 1.7's Java implementation under ompi/mpi/java seem to be able to build up 
> independently.  Do you think we can build just them and run it (via 
> prunjava?) with our custom OpenMPI build based on 1.6?


Yes -- IIRC, the Java interface isn't really dependent upon anything specific 
in the back-end C implementation of Open MPI.  So I'm guessing/assuming that if 
you can build it, it should work against the 1.6 OMPI C engine just fine.

Is MPI-for-Java of interest to you?  I ask because Ralph and I have been trying 
to figure out how to get the cycles to do more with the Java interface (e.g., 
make it more like a 1:1 set of bindings).  Is this something you'd be willing 
to work on / contribute, perchance?

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI devel] "Open MPI"-based MPI library used by K computer

[George Bosilca]

Our policy so far was that adding a paper to the list of publication on the 
Open MPI website was a discretionary action at the authors' request. I don't 
see any compelling reason to change. Moreover, Fujitsu being a contributor of 
the Open MPI community, there is no obstacle of adding a link to their paper -- 
at their request.

  George.

On Jan 10, 2013, at 00:15 , Rayson Ho  wrote:

> Hi Ralph,
> 
> Since the whole journal is available online, and is reachable by
> Google, I don't believe we can get into copyright issues by providing
> a link to it (but then, I also know that there are countries that have
> more crazy web page linking rules!).
> 
> http://www.fujitsu.com/global/news/publications/periodicals/fstj/archives/vol48-3.html
> 
> Rayson
> 
> ==
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> 
> Scalable Cloud HPC: 10,000-node OGS/GE Amazon EC2 cluster
> http://blogs.scalablelogic.com/2012/11/running-1-node-grid-engine-cluster.html
> 
> 
> On Thu, Sep 20, 2012 at 6:46 AM, Ralph Castain  wrote:
>> I'm unaware of any formal criteria. The papers currently located there are 
>> those written by members of the OMPI community, but we can certainly link to 
>> something written by someone else, so long as we don't get into copyright 
>> issues.
>> 
>> On Sep 19, 2012, at 11:57 PM, Rayson Ho  wrote:
>> 
>>> I found this paper recently, "MPI Library and Low-Level Communication
>>> on the K computer", available at:
>>> 
>>> http://www.fujitsu.com/downloads/MAG/vol48-3/paper11.pdf
>>> 
>>> What are the criteria for adding papers to the "Open MPI Publications" page?
>>> 
>>> Rayson
>>> 
>>> ==
>>> Open Grid Scheduler - The Official Open Source Grid Engine
>>> http://gridscheduler.sourceforge.net/
>>> 
>>> 
>>> On Fri, Nov 18, 2011 at 5:32 AM, George Bosilca  
>>> wrote:
 Dear Yuki and Takahiro,
 
 Thanks for the bug report and for the patch. I pushed a [nearly identical] 
 patch in the trunk in https://svn.open-mpi.org/trac/ompi/changeset/25488. 
 A special version for the 1.4 has been prepared and has been attached to 
 the ticket #2916 (https://svn.open-mpi.org/trac/ompi/ticket/2916).
 
 Thanks,
 george.
 
 
 On Nov 14, 2011, at 02:27 , Y.MATSUMOTO wrote:
 
> Dear Open MPI community,
> 
> I'm a member of MPI library development team in Fujitsu,
> Takahiro Kawashima, who sent mail before, is my colleague.
> We start to feed back.
> 
> First, we fixed about MPI_LB/MPI_UB and data packing problem.
> 
> Program crashes when it meets all of the following conditions:
> a: The type of sending data is contiguous and derived type.
> b: Either or both of MPI_LB and MPI_UB is used in the data type.
> c: The size of sending data is smaller than extent(Data type has gap).
> d: Send-count is bigger than 1.
> e: Total size of data is bigger than "eager limit"
> 
> This problem occurs in attachment C program.
> 
> An incorrect-address accessing occurs
> because an unintended value of "done" inputs and
> the value of "max_allowd" becomes minus
> in the following place in "ompi/datatype/datatype_pack.c(in version 
> 1.4.3)".
> 
> 
> (ompi/datatype/datatype_pack.c)
> 188 packed_buffer = (unsigned char *) iov[iov_count].iov_base;
> 189 done = pConv->bConverted - i * pData->size;  /* partial 
> data from last pack */
> 190 if( done != 0 ) {  /* still some data to copy from the 
> last time */
> 191 done = pData->size - done;
> 192 OMPI_DDT_SAFEGUARD_POINTER( user_memory, done, 
> pConv->pBaseBuf, pData, pConv->count );
> 193 MEMCPY_CSUM( packed_buffer, user_memory, done, pConv 
> );
> 194 packed_buffer += done;
> 195 max_allowed -= done;
> 196 total_bytes_converted += done;
> 197 user_memory += (extent - pData->size + done);
> 198 }
> 
> This program assumes "done" as the size of partial data from last pack.
> However, when the program crashes, "done" equals the sum of all 
> transmitted data size.
> It makes "max_allowed" to be a negative value.
> 
> We modified the code as following and it passed our test suite.
> But we are not sure this fix is correct. Can anyone review this fix?
> Patch (against Open MPI 1.4 branch) is attached to this mail.
> 
> -if( done != 0 ) {  /* still some data to copy from the last 
> time */
> +if( (done + max_allowed) >= pData->size ) {  /* still some 
> data to copy from the last time */
> 
> Best