Hi Jeff, Ralph, and all, Thank you for your reply. RIKEN and Fujitsu will work toword 10Pflops with Open MPI continuously.
Here we can explain some parts of our MPI: As page 13 of Koh Hotta's presentation shows, we extended OMPI communication layers. > http://www.fujitsu.com/downloads/TC/sc10/programming-on-k-computer.pdf # Sorry, this figure is somewhat broken. Arrows point incorrect layers. First, we created a new BTL component, 'tofu BTL'. It's not so special one but dedicated to our Tofu interconnect. But its latency was not enough for us. So we created a new framework, 'LLP', and its component, 'tofu LLP'. It bypasses request object creation in PML and BML/BTL, and sends a message immediately if possible. Also, we modified tuned COLL to implement interconnect-and-topology- specific bcast/allgather/alltoall/allreduce algorithm. These algorithm implementations also bypass PML/BML/BTL to eliminate protocol and software overhead. To achieve above, we created 'tofu COMMON', like sm (ompi/mca/common/sm/). Is there interesting one? Though our BTL and COLL are quite interconnect-specific, LLP may be contributed in the future. Regards, Takahiro Kawashima, MPI development team, Fujitsu > I echo what Ralph said -- congratulations! > > Let us know when you'll be ready to contribute back what you can. > > Thanks! > > > On Jun 27, 2011, at 9:58 PM, Takahiro Kawashima wrote: > > > Dear Open MPI community, > > > > I'm a member of MPI library development team in Fujitsu. Shinji > > Sumimoto, whose name appears in Jeff's blog, is one of our bosses. > > > > As Rayson and Jeff noted, K computer, world's most powerful HPC system > > developed by RIKEN and Fujitsu, utilizes Open MPI as a base of its MPI > > library. We, Fujitsu, are pleased to announce that, and also have special > > thanks to Open MPI community. > > We are sorry to be late announce! > > > > Our MPI library is based on Open MPI 1.4 series, and has a new point- > > to-point component (BTL) and new topology-aware collective communication > > algorithms (COLL). Also, it is adapted to our runtime environment (ESS, > > PLM, GRPCOMM etc). > > > > K computer connects 68,544 nodes by our custom interconnect. > > Its runtime environment is our proprietary one. So we don't use orted. > > We cannot tell start-up time yet because of disclosure restriction, sorry. > > > > We are surprised by the extensibility of Open MPI, and have proved that > > Open MPI is scalable to 68,000 processes level! We feel pleasure to > > utilize such a great open-source software. > > > > We cannot tell detail of our technology yet because of our contract > > with RIKEN AICS, however, we will plan to feedback of our improvements > > and bug fixes. We can contribute some bug fixes soon, however, for > > contribution of our improvements will be next year with Open MPI > > agreement. > > > > Best regards, > > > > MPI development team, > > Fujitsu > > > > > >> I got more information: > >> > >> http://blogs.cisco.com/performance/open-mpi-powers-8-petaflops/ > >> > >> Short version: yes, Open MPI is used on K and was used to power the 8PF > >> runs. > >> > >> w00t! > >> > >> > >> > >> On Jun 24, 2011, at 7:16 PM, Jeff Squyres wrote: > >> > >>> w00t! > >>> > >>> OMPI powers 8 petaflops! > >>> (at least I'm guessing that -- does anyone know if that's true?) > >>> > >>> > >>> On Jun 24, 2011, at 7:03 PM, Rayson Ho wrote: > >>> > >>>> Interesting... page 11: > >>>> > >>>> http://www.fujitsu.com/downloads/TC/sc10/programming-on-k-computer.pdf > >>>> > >>>> Open MPI based: > >>>> > >>>> * Open Standard, Open Source, Multi-Platform including PC Cluster. > >>>> * Adding extension to Open MPI for "Tofu" interconnect > >>>> > >>>> Rayson > > _______________________________________________ > > devel mailing list > > de...@open-mpi.org > > http://www.open-mpi.org/mailman/listinfo.cgi/devel > > > -- > Jeff Squyres > jsquy...@cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/
