Re: [OMPI users] MPI is still dominantparadigm?

[Oddo Da via users]

On Sat, Aug 8, 2020 at 2:33 PM Supun Kamburugamuve 
wrote:

> Hi Oddo,
>
> If you are interested, check out https://github.com/DSC-SPIDAL/twister2.
> It provides a Java-based API similar to other big data tools and uses
> OpenMPI underneath. It can work with other frameworks (UCX, Kubernetes) but
> OpenMPI is one of the major supported ways it can communicate and be
> deployed. I think this shows the versatility of the OpenMPI with Java
> bindings.
>

Nice! Thank you.

Re: [OMPI users] MPI is still dominantparadigm?

[Supun Kamburugamuve via users]

Hi Oddo,

If you are interested, check out https://github.com/DSC-SPIDAL/twister2. It
provides a Java-based API similar to other big data tools and uses OpenMPI
underneath. It can work with other frameworks (UCX, Kubernetes) but OpenMPI
is one of the major supported ways it can communicate and be deployed. I
think this shows the versatility of the OpenMPI with Java bindings.

Best,
Supun..

On Sat, Aug 8, 2020 at 8:23 AM Oddo Da via users 
wrote:

> On Sat, Aug 8, 2020 at 6:51 AM Benson Muite via users <
> users@lists.open-mpi.org> wrote:
>
>> Some Ping pong tests using the Java interface, JDK 8
>> https://github.com/mboysan/ping-pong-mpi-tcp
>>
>> Perhaps there might be interest in porting the other OSU or Intel MPI
>> benchmarks to this?
>>
>> There are also some parallel Java PGAS related projects that might
>> possibly use at some point some of the OpenMPI components:
>> https://pcj.icm.edu.pl/
>> https://wiki.rice.edu/confluence/display/PARPROG/HJ+Library
>> http://mpj-express.org/
>>
>
>  Thank you, this is very useful!
>


-- 
Supun Kamburugamuve, PhD
Digital Science Center, Indiana University
Member, Apache Software Foundation; http://www.apache.org
E-mail: supun@apache.o rg;  Mobile: +1 812 219 2563

Re: [OMPI users] MPI is still dominantparadigm?

[Oddo Da via users]

On Sat, Aug 8, 2020 at 6:51 AM Benson Muite via users <
users@lists.open-mpi.org> wrote:

> Some Ping pong tests using the Java interface, JDK 8
> https://github.com/mboysan/ping-pong-mpi-tcp
>
> Perhaps there might be interest in porting the other OSU or Intel MPI
> benchmarks to this?
>
> There are also some parallel Java PGAS related projects that might
> possibly use at some point some of the OpenMPI components:
> https://pcj.icm.edu.pl/
> https://wiki.rice.edu/confluence/display/PARPROG/HJ+Library
> http://mpj-express.org/
>

 Thank you, this is very useful!

Re: [OMPI users] MPI is still dominantparadigm?

[Benson Muite via users]

On 8/8/20 1:19 PM, Jeff Squyres (jsquyres) via users wrote:
On Aug 7, 2020, at 12:52 PM, Oddo Da via users > wrote:


The Java bindings support "recent" JDK, and if you face an issue,
please report a bug (either here or on github)


Starting with Java 8, the language went into a much different 
direction - functional programming, lambdas, streams, concurrency. It 
is not at all the language we may remember from the early 2000s. At 
JDK 9 or 10+ I think it supports actor based concurrency. So, the 
"recent" part is rather vague - I was hoping for a particular JDK 
minimum :-)


As Ralph mentioned, the Java bindings were added quite a while ago.  I'm 
afraid I don't know which JDK is the minimum level, but they probably 
support fairly old versions (probably JDK 7? Maybe 6? I don't know 
offhand).


Specifically, there were a few Java MPI experiments in the 90's/early 
2000's.  Those kinda fizzled out.  The Open MPI Java bindings was an 
attempt by some Open MPI community members to revamp/modernize the 
earlier Java MPI bindings, and specifically to bridge some of the 
performance gap that the earlier work saw (by using the Java bridge to 
the underlying C bindings, which didn't exist at the time of the first 
Java experiments).  It was largely successful within its defined goals.


I am not a Java expert, but I understand that the authors also 
modernized the interface a bit, too.


That being said, as you mentioned, JDK 9/10 have changed many, many 
things.  The Java MPI bindings in Open MPI have not been following the 
evolution of the JDK, so there is no usage of modern JDK 
abstractions/language features.


That being said, Open MPI is quite definitely an open source project. 
  If you have some thoughts about how to improve the existing Java 
bindings, that would be great!  The Open MPI Java bindings are *not* 
recognized by the MPI Forum, however -- they are a cool side project and 
I think at least a small number of apps used them.  But they never 
caught wide-spread usage or popularity.  That being said, if there's 
some interesting, publish-worthy stuff to pursue here, well, that's part 
of why the Open MPI project exists.  :-)


Some Ping pong tests using the Java interface, JDK 8
https://github.com/mboysan/ping-pong-mpi-tcp

Perhaps there might be interest in porting the other OSU or Intel MPI 
benchmarks to this?


There are also some parallel Java PGAS related projects that might 
possibly use at some point some of the OpenMPI components:

https://pcj.icm.edu.pl/
https://wiki.rice.edu/confluence/display/PARPROG/HJ+Library
http://mpj-express.org/

Re: [OMPI users] MPI is still dominantparadigm?

[Jeff Squyres (jsquyres) via users]

On Aug 7, 2020, at 12:52 PM, Oddo Da via users 
mailto:users@lists.open-mpi.org>> wrote:

The Java bindings support "recent" JDK, and if you face an issue, please report 
a bug (either here or on github)

Starting with Java 8, the language went into a much different direction - 
functional programming, lambdas, streams, concurrency. It is not at all the 
language we may remember from the early 2000s. At JDK 9 or 10+ I think it 
supports actor based concurrency. So, the "recent" part is rather vague - I was 
hoping for a particular JDK minimum :-)

As Ralph mentioned, the Java bindings were added quite a while ago.  I'm afraid 
I don't know which JDK is the minimum level, but they probably support fairly 
old versions (probably JDK 7? Maybe 6? I don't know offhand).

Specifically, there were a few Java MPI experiments in the 90's/early 2000's.  
Those kinda fizzled out.  The Open MPI Java bindings was an attempt by some 
Open MPI community members to revamp/modernize the earlier Java MPI bindings, 
and specifically to bridge some of the performance gap that the earlier work 
saw (by using the Java bridge to the underlying C bindings, which didn't exist 
at the time of the first Java experiments).  It was largely successful within 
its defined goals.

I am not a Java expert, but I understand that the authors also modernized the 
interface a bit, too.

That being said, as you mentioned, JDK 9/10 have changed many, many things.  
The Java MPI bindings in Open MPI have not been following the evolution of the 
JDK, so there is no usage of modern JDK abstractions/language features.

That being said, Open MPI is quite definitely an open source project.  If you 
have some thoughts about how to improve the existing Java bindings, that would 
be great!  The Open MPI Java bindings are *not* recognized by the MPI Forum, 
however -- they are a cool side project and I think at least a small number of 
apps used them.  But they never caught wide-spread usage or popularity.  That 
being said, if there's some interesting, publish-worthy stuff to pursue here, 
well, that's part of why the Open MPI project exists.  :-)

--
Jeff Squyres
jsquy...@cisco.com

Re: [OMPI users] MPI is still dominantparadigm?

[Oddo Da via users]

On Fri, Aug 7, 2020 at 11:41 AM Gilles Gouaillardet via users <
users@lists.open-mpi.org> wrote:

> The Java bindings support "recent" JDK, and if you face an issue, please
> report a bug (either here or on github)


Starting with Java 8, the language went into a much different direction -
functional programming, lambdas, streams, concurrency. It is not at all the
language we may remember from the early 2000s. At JDK 9 or 10+ I think it
supports actor based concurrency. So, the "recent" part is rather vague - I
was hoping for a particular JDK minimum :-)

Thanks!

Re: [OMPI users] MPI is still dominantparadigm?

[Gilles Gouaillardet via users]

 The goal of Open MPI is to provide a high quality of the MPI standard,

and the goal of this mailing list is to discuss Open MPI (and not the 
MPI standard)



The Java bindings support "recent" JDK, and if you face an issue, please 
report a bug (either here or on github)



Cheers,



Gilles

- Original Message -

Hello,

This may be a bit of a longer post and I am not sure if it is even 
appropriate here but I figured I ask. There are no hidden agendas in it, 
so please treat it as "asking for opinions/advice", as opposed to 
judging or provoking.

For the period between 2010 to 2017 I used to work in (buzzword alert!) 
"big data" (meaning Spark, HDFS, reactive stuff like Akka) but way 
before that in the early 2000s I used to write basic multithreaded C and 
some MPI code. I came back to HPC/academia two years ago and what struck 
me was that (for lack of better word) the field is still "stuck" (again, 
for lack of better word) on MPI. This itself may seem negative in this 
context, however, I am just stating my observation, which may be wrong.

I like low level programming and I like being in control of what is 
going on but having had the experience in Spark and Akka, I kind of got 
spoiled. Yes, I understand that the latter has fault-tolerance (which is 
nice) and MPI doesn't (or at least, didn't when I played with in 1999-
2005) but I always felt like MPI needed higher level abstractions as a 
CHOICE (not _only_ choice) laid over the bare metal offerings. The whole 
world has moved onto programming in patterns and higher level 
abstractions, why is the academic/HPC world stuck on bare metal, still? 
Yes, I understand that performance often matters and the higher up you 
go, the more performance loss you incur, however, there is also 
something to be said about developer time and ease of understanding/
abstracting etc. etc.

Be that as it may, I am working on a project now in the HPC world and I 
noticed that Open MPI has Java bindings (or should I say "interface"?). 
What is the state of those? Which JDK do they support? Most importantly, 
would it be a HUGE pipe dream to think about building patterns a-la Akka 
(or even mixing actual Akka implementation) on top of OpenMPI via this 
Java bridge? What would be involved on the OpenMPI side? I have time/
interest in going this route if there would be any hope of coming up 
with something that would make my life (and future people coming into 
HPC/MPI) easier in terms of building applications. I am not saying MPI 
in C/C++/Fortran should go away, however, sometimes we don't need the 
low-level stuff to express a concept :-). It may also open a whole new 
world for people on large clusters...

Thank you!