RE: green thread support

Mon, 02 May 2005 09:52:43 -0700

Lee,
 
Thanks for the additional description of the type of work you are doing.  It appears that the way I am using the java green threads may be different from what you are wanting to do.  I will describe in more detail the way I have been able to use the java green threads on openMosix, and then pose some additional questions.
 
The application I am running is an "off the shelf" version of the GE PSLF Positive Sequence Load Flow Program, ver. 14.2 for linux for modeling and simulation load flow and stability on high-voltage electric utiliity power grids.  http://www.gepower.com/prod_serv/products/utility_software/en/ge_pslf/index.htm 
 
The program is written mostly in java with a graphical user interface based on the latest Java 1.4.2 libraries.  Some parts are written in C, to the best of my knowledge.  However, the program can also be run in a "batch mode" and successfully make use of the older Java 1.3.1 with the green threads capability.  The green threads allow multiple instances of the PSLF program to migrate automatically over the entire cluster [39-dual 2.4-Ghz P4-XEON nodes running Redhat 9 with the openMosix 2.4.24-2 SMP kernel, with XEON hyperthreading enabled]. 
 
PSLF is installed such that is can be run with either version of the java libaries, 1.4.2 or 1.3.1.  Java 1.4.2 is required for the graphical interactive sessions and 1.3.1 is used for batch job sessions to take advantage of the openMosix environment.  [I wish green threads support would be added back in to the later versions of Java!! --  that would be a big help for this particular application].
 
PSLF Batch Mode:  The openMosix SMP kernel running on the hyperthreading-enabled compute nodes shows up as an openMosix cluster with 156-CPU's.   As an example, I use a bash script to submit 300-simulations to be run as 300 separate instances of PSLF.  Each simulation models up to 30-seconds of real-time operation of the Western US Electric Power Grid.  The 300 instances automatically migrate out onto the 156 CPU's of the cluster.  The entire combined run completes in approximately 1-hour.  To run the 300 simulations on a single node of the cluster takes approx. 1-full calendar week of time.
 
The executable script that starts the Batch-Mode PSLF session contains these commands:
 
PATH=/usr/local/j2sdk1.3.1/jre/bin:$PATH
export PSLFDIR=~/upslf142
export CLASSPATH=~/upslf142/classes
export LD_LIBRARY_PATH=~/upslf142/classes
which java
java -version
java -green -jar ~/upslf142/classes/pslf.jar $1
 
 
For migration of individual threads within a single instance of a program, I am not sure if this would work the same way....?  I have run successfully a C/C++ test program in which individual threads do migrate on the openMosix cluster, by using the FSU-Pthreads library. 
 
 
This test program really works well!  I have pasted in some text from the test summary.
 
 
 

Cluster Test Program: BDIV -- search range of large integers for prime numbers using Computational Threads

Two Versions created:  to "parallelize" the computation, and parallelize CPU processes.

 

      ptDiv_STD -- compiled using Linux Threads, Standard, with Redhat 8.0/9.0 Linux Distribution

      ptDiv_FSU -- compiled using FSU-Posix Threads , installed as additional thread library.

 

Basic Program Command:  ./ptDiv_XXX -r 1048576 4194304 -t 40 -s -n

 

                        -r 1048576 4194304      Range to Seach, min. max.

                        -t 40                   Spawn "40" Threads

                        -s                      Force scheduler invocation for the test

                        -n                      Force "no output" mode

 

Then, I made three executable scripts:

 

      ptDiv_STD-40x5      -- runs "5" instances of the basic program command, above

      ptDiv_FSU-40x5      -- runs "5" instances of the basic program command, above

      ptDiv_FSU-40x50     -- runs "50" instances of the basic program command, above

 

-------------------------------------------------------------------------------------------------

 

Test System:      Linux Cluster, Redhat Linux 9.0, openMosix-2.4.22-1-SMP kernel, 39-nodes

                  Nodes: 1-Gb RAM, Dual-CPU Intel 2.4-Ghz XEON [P4] with hyperthreading enabled

 

-------------------------------------------------------------------------------------------------

Test Results:                                                    [# nodes automatically used]

 

LinuxThreads1:     ptDiv_STD-40x5  -- [1-instance]      Completion: 14:14-min./sec.     only 1

 

 

FSU-Threads1:      ptDiv_FSU-40x5  -- [1-instance]      Completion:  2:28-min./sec.     3

 

FSU-Threads2:      ptDiv_FSU-40x50 -- [2-instances]      Completion:  3:24-min./sec.    34   

 

FSU-Threads3:      ptDiv_FSU-40x50 -- [4-instances]      Completion:  4:36-min./sec.    all 39     

 

FSU-Threads4:      ptDiv_FSU-40x50 -- [8-instances]      Completion:  8:11-min./sec.    all 39

 

FSU-Threads5:      ptDiv_FSU-40x50 -- [16-instances]      Completion: 15:40-min./sec.   all 39

 

-------------------------------------------------------------------------------------------------

 

-------------------------------------------------------------------------------------------------

 

Analysis/Comparison of Results, based on equivalent "FSU-Threads5" Thread-Count/Timing:

 

 

FSU-Threads5:  [40x50x16] = 32,000-threads; [total solution time = 15:40-min-sec.] = 34.04 threads/sec.

 

LinuxThreads1: [40x5x1]   =    200-threads; [total solution time = 14:14-min-sec.] = 0.234 threads/sec.

 

               [40x50x16] = 32,000-threads; [equiv.solution time = 37:57:20-hours-min-sec.]

 

------------------------------------------------------------------------------------------------------

 

 

PSLF Graphic Interactive Mode:  The openMosix SMP kernel running on the hyperthreading-enabled compute nodes shows up as an openMosix cluster with 156-CPU's.   I can log in to multiple individual nodes and run up to 4 instances of PSLF graphically, and they will make use of the "4-cpu's per node" via the hyperthreading, but they will not migrate to other nodes using the Java 1.4.2 libraries, which do not support "green threads". 
 
The executable script that starts the Interactive PSLF session contains these commands:
 
PATH=~/upslf142/jre/bin:$PATH
export PSLFDIR=~/upslf142
export CLASSPATH=~/upslf142/classes
export LD_LIBRARY_PATH=~/upslf142/classes
which java
java -version
java -jar ~/upslf142/classes/pslf.jar $1
 
I hope that helps, Lee.  Let me know any additional questions you might have, and also if you can develop successful methods in doing what you are wanting to do.
 
Sincerely,
 
John
 
 
 
 
 
-----Original Message-----
From: Lee Rong [mailto:[EMAIL PROTECTED]
Sent: Monday, May 02, 2005 8:07 AM
To: Schaad,John G - TOC; [EMAIL PROTECTED]
Subject: RE: green thread support

Hi John,

 Thanks for the reply. My intention is indeed using blackdown JDK to run Java threads on OpenMosix cluster. Actually my program is a java P2P simulation and currently I am only able to run around 6000 threads (i.e., peers) on one machine. My goal is to run approximately 100,000 threads over a cluster. However my main concern is that since the threads in my application all communicate with each other (i.e., thread execute methods in other theads), does that mean it will generate too much network congestions if most of the threads trying to execute methods in other threads on different machines?

 

Thanks,

 

Lee

From: Schaad,John G - TOC [mailto:[EMAIL PROTECTED]
Sent: Tuesday, 3 May 2005 12:50 AM
To: Lee Rong
Subject: RE: green thread support

 

Hi, Lee,

 

The latest version that supports green threads is 1.3.1

 

 

I use this version of Java with Green Threads, with an application on an openMosix Cluster.  It really works well.

 

--John Schaad

-----Original Message-----
From: Lee Rong [mailto:[EMAIL PROTECTED]]
Sent: Monday, May 02, 2005 7:25 AM
To: [EMAIL PROTECTED]
Subject: green thread support

Hi,

 I was just wandering to what blackdown version JDK is green thread supported?

 

Thanks,

 

Lee

Reply via email to