JDK 1.1.6 Version 4a and VolanoMark 2.0

1998-09-23 Thread John Neffenger

Thanks for the great port in Version 4a!

It fixes that socket timeout bug I reported here back in October 1997
against JDK 1.1.3.  I was unable to reproduce the problem at all outside
of the VolanoMark benchmark or our VolanoChat product, but I'm grateful
someone figured it out.

Using the same VolanoMark 2.0.0 Build 137 that I used in my August
JavaWorld article, I get the new scores shown below -- a 37% performance
improvement over JDK 1.1.6 Version 2!  From the stress tests I've run,
it seems to be very stable as well.

  VolanoMark 2.0.0 Build 137 Loopback Test

  Java virtual machine Scores  Average (best 2 of 3)
    -  -
  Linux JDK 1.1.6 v2230, 234, 233   234
  Linux JDK 1.1.6 v4a   317, 321, 319   320  (+ 37%)

For the details on the test, see:

  http://www.javaworld.com/javaworld/jw-08-1998/jw-08-volanomark.html

If anyone on the Blackdown Java-Linux Porting Team wants a copy of
VolanoMark 2.0, please let me know.

Thanks again,
John Neffenger



Re: JDK 1.1.6 Version 4a and VolanoMark 2.0

1998-09-24 Thread John Neffenger

Hi Dan,

> Thanks John for updating us with the new results!
> Great, we're now neck and neck with that speed demon, MRJ 2.0 :-)

It looks as if FreeBSD's JDK 1.1.6 port is the same speed (for
VolanoMark) as their JDK 1.1.5, but they also fixed that nasty socket
timeout bug I was hitting on both Linux and FreeBSD.  Microsoft's and
Tower's latest releases are coming up a bit slower than before, but not
enough to matter.

Actually, the connection scalability on Linux -- not speed -- is the
bottleneck for now.  Even after I rebuilt my Red Hat Linux 4.2 kernel
for 1024 file descriptors per process, I was unable to get over about
250 connections to the VolanoChat server.  TowerJ 2.1.2 could get up to
about 350 connections on the same machine.

Is there a limit on file descriptors built into the Linux Java VM or its
libraries somewhere?  I followed the instructions at:

  http://www.volano.net/guide/linux.html

I haven't tried more than 200 connections on Red Hat Linux 5.1 yet.  The
default limit is increased to 1024 in the kernel for version 2.1, right?

> Does the the Volano benchmark run
> with the TYA JIT (http://www.tya.home.ml.org)?

I haven't tried it yet.

> Or for that matter with Kaffe (http://www.kaffe.org)?

Kaffe can't run VolanoMark (nor our product, VolanoChat), but Tim
Wilkinson at Transvirtual Technologies is working on it.

John Neffenger



JDK 1.2.2 RC2 VolanoMark results

1999-12-04 Thread John Neffenger

The JDK 1.2.2 RC2 release of the Java-Linux port does the best job yet
at running VolanoMark.  Here's what I get for the loopback performance
test in messages per second:

  Sun JDK 1.2.2 RC2 Linux

JITno JIT
  ---
  Native931  513
  Green 956  478

and here's what I get on the network scalability test in messages per
second for 1,000 through 4,000 simultaneous connections (using green
threads with the JIT and disabling the use of socket timeouts on the
server):

  Java Platform   1000  2000  3000  4000
  ==        
  Sun JDK 1.2.2 RC2 Linux 1166   690   463   (*)

  (*) The 4000-connection test failed with:
  java.lang.StackOverflowError
at java.security.AccessController.doPrivileged(Native Method)
at java.net.Socket.getOutputStream(Socket.java, Compiled Code)
at COM.volano.mby.(Unknown Source)
at COM.volano.Main.run(Unknown Source)
at java.lang.Thread.run(Thread.java, Compiled Code)

I needed to disable the use of socket timeouts on the server --
something our customers can't do without bad side effects -- in order to
work around this problem:

  Bug Id 1578, "java.net.SocketException: Interrupted system call"
  http://www.blackdown.org/cgi-bin/jdk/incoming?id=1578

That's a dramatic improvement over the network scalability of JDK 1.1.7
Version 3 on Linux, and far better than the current limit of 500
connections with IBM's JDK 1.1.8.  Better yet, I did not need to
recompile the Linux kernel nor rebuild the LinuxThreads library in order
to run the test, as is required for IBM's Java VM on Linux.  I just set
the per-process file descriptor limit to 4096 and ran!

For comparison scores, see:

  The Volano Report
  http://www.volano.com/report.html

Looking forward to RC3!

Thanks,
John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: Sun and Inprise Java 2 announcement

1999-12-07 Thread John Neffenger

I wrote directly to Sun about it (mailto:[EMAIL PROTECTED],
attached below).  I'll try to post the results of both JDK 1.2.2 ports
-- what I thought was Blackdown *and* Sun, and the new port from Sun --
to the Volano Report today or tomorrow.

John Neffenger

John Neffenger wrote:
> 
> Dear Sun,
> 
> I find it bizarre that you make no mention of Sun's and the Blackdown
> Organization's release today of JDK 1.2.2 Release Candidate 3 on Linux.
> It's one of the best Java-Linux ports so far -- even better than IBM's
> -- and includes support for native threads.  (I'll be comparing it with
> your new port today and posting the results.)  The people at the
> Blackdown Organization, including Randy Chapman and Steve Byrne, were
> the only ones keeping Java on Linux alive for the past three years.
> 
> It was disappointing to find no mention of that history in your press
> release.
> 
> Sincerely,
> John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: Native vs. green threads

1999-12-08 Thread John Neffenger

We don't have to argue about native vs. green threads since the
Blackdown JDK 1.2.2 release does both.  

I understand why people need native threads, but I need green threads on
Linux for handling a large number of dedicated connections to a pure
Java server (see
http://developer.java.sun.com/developer/bugParade/bugs/4075058.html).

The alternative with native-thread-only implementations on Linux, like
the one from IBM, is to rebuild the kernel and LinuxThreads library --
not fun (see "Linux 2.2 kernel file descriptors and tasks" and
"LinuxThreads 0.7 limits" at http://www.volano.com/linux.html).

I assume the Inprise Java VM will soon do both green and native threads
as well.

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



JDK 1.2.2 RC3 and RC1 VolanoMark results

1999-12-08 Thread John Neffenger

I updated the Volano Report to include Blackdown JDK 1.2.2 RC3 on Red
Hat 6.1 and Inprise JDK 1.2.2 RC1 on Red Hat 6.0:

  The Volano Report
  http://www.volano.com/report.html

Kevin Hendricks told me he already knows the fix for the SIGSEGV problem
at just over 1000 connections (see Table 2).  When I work around the
problem by not setting socket timeouts, the Blackdown VM goes right up
to 4000 connections.  Amazing.

The Inprise Java VM still hits the following bug, which Blackdown fixed
in Release Candidate 3:

  http://www.blackdown.org/cgi-bin/jdk/incoming?id=1578

Let me know if you find any errors in the updated Volano Report.

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Segmentation violation with 1020 timeout sockets

1999-12-08 Thread John Neffenger

I opened what I think is the last bug holding back the VolanoMark
network scalability tests under Blackdown JDK 1.2.2:

  Bug Id 1604, "Segmentation violation with 1020 timeout sockets"
  http://www.blackdown.org/cgi-bin/jdk/incoming?id=1604

I guess we report these bugs to Sun and Inprise through this page?

  http://java.sun.com/cgi-bin/bugreport.cgi

Sun and Inprise need to fix both Blackdown Bug Id 1604 and 1578 for me
to run the VolanoMark network test with their JDK 1.2.2:

  http://www.blackdown.org/cgi-bin/jdk/incoming?expression=neffenger

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: JDK 1.2.2 RC3 and RC1 VolanoMark results

1999-12-08 Thread John Neffenger

Hi Joseph,

> Is there any way I can get a hold of the benchmark software?  I'd like to
> try the tya compiler with blackdown's jdk and see how it stacks up.

See the section called "Download" at:

  http://www.volano.com/benchmarks.html

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: JDK 1.2.2 RC3 and RC1 VolanoMark results

1999-12-08 Thread John Neffenger

Hi Rachit,

> how does it compare to the other two java ports at high connections?
> your page i assume wasn't updated after you ran the 4000 connections
> test. just curious.

IBM JDK 1.1.8 on Linux can't get over 500 connections (a bug confirmed
by IBM).  Blackdown JDK 1.2.2 RC3 fails at just over 1,000 connections
(Bug Id 1604).  Inprise JDK 1.2.2 RC1 fails at more than one connection
over the network (Bug Id 1578).  TowerJ goes right up to 4,000
connections -- about the limit on my 256 MB box.

I ran a couple tests with Blackdown JDK 1.2.2 at high connection counts
after disabling the socket timeouts, but I'm waiting for the bug fixes
before being able to run the tests without modification for the Volano
Report.

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



New Volano Report

2001-04-14 Thread John Neffenger

I posted our latest test results yesterday:

  The Volano Report
  http://www.volano.com/report.html

Thank you, Blackdown, for keeping green threads alive and working
extraordinarily well on Linux!  Sun abandoned their "sunwjit" back in
1999:

  java.lang.StackOverflowError with sunwjit
  http://developer.java.sun.com/developer/bugParade/bugs/4299454.html
  "The sunwjit has been obsolete for long time.  We are not going
   to fix the problem."

and now they've abandoned the Classic VM:

  Classic VM goes into hard run at 1018 socket connections
  http://developer.java.sun.com/developer/bugParade/bugs/4427986.html
  "As to classic VM hang, it's a known problem.  Since we are near
   ladybird FCS and merlin does not include classic VM.  This bug will
   not be fixed."

Blackdown's Java VM using green threads on Linux is the only hope for
pure Java servers with lots of connections -- at least while we're
waiting for the Java 1.4 "new I/O" (or a different Linux threading
model).

By the way, I tested several just-in-time compilers with Blackdown on
Linux, and I'm very impressed with OpenJIT:

  OpenJIT: A Reflective JIT Compiler for Java
  http://www.openjit.org/

I tried OpenJIT, javacomp (Borland), shuJIT, sunwjit (Sun), and tya. 
OpenJIT was the fastest in my VolanoMark tests and seemed to be the most
stable.

Thanks,
John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]




Re: New Volano Report

2001-04-17 Thread John Neffenger

Hi Matt,

I did some processor scalability tests with VolanoMark back in August,
1998, testing Microsoft's and Sun's Java virtual machines on one through
four processors (both Intel and SPARC).  The results are under the
section called "The hardware solution" in my JavaWorld article:

  "Which Java VM scales best?"
  http://www.javaworld.com/javaworld/jw-08-1998/jw-08-volanomark_p.html

I don't test processor scalability anymore because, for me, it's not the
problem.  (It used to be a problem back in 1997 when Sun's Java VM went
two times *slower* on Solaris when you added a second processor.)  These
days, for pure Java servers with lots of dedicated long-lasting socket
connections, the problem is network scalability:

  VolanoMark 2.1.2 Network Scalability (Line Plot)
  http://www.volano.com/report/network2.gif

Note that the limits on the number of concurrent connection exist no
matter how many processors or how much memory you throw at the problem. 
Only Sun's 1.3.0 HotSpot Server VM on Solaris had problems under 6,000
connections because of real memory (320 MB).  All of the other problems
are due to architectural limits or bugs.  These limits used to be due to
file descriptors and thread handling, but now it's just the thread
handling.  The details of each failure are shown if you click on the
chart to get this page:

  VolanoMark 2.1.2 Network Scalability (Details)
  http://www.volano.com/report/network.txt

The Windows VMs using a one-to-one mapping of Java to native threads
allocate a stack reserve of 256 KB per thread and fail (with an
OutOfMemoryError) when the reserved storage exhausts the 2 GB address
space.  The Linux VMs using a one-to-one mapping fail when they hit the
maximum number of Linux tasks or limits in the Linux threads library. 
The other failures are due to bugs.

There are, of course, applications which need good processor
scalability.  But as far as VolanoMark and our VolanoChat product are
concerned, what good is processor scalability, or even superb
performance in general, when you can only connect 422 people?

John Neffenger

Matt Peterson wrote:
> 
> John,
> 
> Thanks for sending the results of the Volano benchmark to the list.  It
> was interesting to see the performance comparisons.  However, I was
> disappointed to see that SMP hardware was not used in the test.  Is
> there a chance that we will see published numbers that show results on
> SMP hardware?  It would definitely be interesting to those that are
> interested in seeing how native thread Java scales in comparison to
> those JVMs that do not use kernel (system mode) thread libraries for
> their thread implementations.  It would also suggest the suitability of
> various platforms (Linux vs Windows vs Solaris) for implementation of a
> fast scalable JVM.
> 
> --
> Matthew Peterson
> Sr. Software Engineer
> Caldera Systems, Inc
> [EMAIL PROTECTED]


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]




Re: New Volano Report

2001-04-17 Thread John Neffenger

Hi Kazuyuki,

> I remember Hewlett Packard was working on the problem and they
> introduced a select()-like API into their class library.

Sun also provided a Java wrapper for its Solaris /dev/poll interface
with the Solaris JDK Production Release.  I'm not sure whether it's
still in the Solaris J2SE SDK.

> But the HP does not participate with the expert group for the JSR #51.
> I feel it is strange.

I was pleased to see that Dan Kegel is on the group, though, since he
gives the most thorough discussion of the problem:

  The C10K Problem
  http://www.kegel.com/c10k.html

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]




Segmentation violation (even with ulimit -s 2048)

2002-11-05 Thread John Neffenger
I had been running Blackdown 1.3.1-FCS (green threads, nojit) without 
any problems for over a year on a system with Linux Kernel version 
2.2.16 and Glibc 2.2.4 on a Red Hat Linux 7.0 base.  I've been using the 
"ulimit -s 2048" setting to avoid the problem that caused segmentation 
violations earlier in 2001 (Sun Bug Id 4466587).

Now I've upgraded to all the very latest Red Hat Linux 7.0 updates:

  Blackdown 1.3.1-02b-FCS (green theads, nojit)
  kernel-2.2.22-7.0.2
  glibc-2.2.4-18.7.0.7

The version strings for Blackdown and the kernel are:

  Classic VM (build Blackdown-1.3.1-02b-FCS, green threads, nojit)
  Linux www.volano.com 2.2.22-7.0.2 #1 Mon Sep 23 06:55:40 EDT 2002 i686

As before, I'm still setting:

  ulimit -s 2048
  ulimit -n 1024

and I still start up Blackdown with the options "-green -Xmx256m 
-Xss64k".  But now I'm getting daily segmentation violations on our 
server -- one or two per day:

  SIGSEGV   11*  segmentation violation
si_signo [11]: SIGSEGV   11*  segmentation violation
si_errno [0]: Success
si_code [0]: SI_USER [pid: 0, uid: 0]
stackpointer=0x523f0528

I get either a nice Java thread stack trace to standard error or a 
300-megabyte core dump with no messages.  I've been unable to reproduce 
the problem in any simplified test cases.  So far, it happens only 
online with real client connections.  Any tips on how I might get useful 
information from the core dumps?

Is anyone successfully using Blackdown 1.3.1-02b-FCS (green threads, 
nojit) with the latest Red Hat Linux 7.0 Kernel and Glibc packages 
(Kernel 2.2.22, Glibc 2.2.4)?

Would I be better off switching to the combination of Linux Kernel 
2.4.18 and Glibc 2.2.4, as in Red Hat Linux 7.2?  I guess I'm curious 
which kernel and library versions are getting the most testing by 
Blackdown developers among all these combinations, shown with their 
latest updated version numbers:

  Red Hat  Kernel  Glibc
  ---  --  --
6.22.2.22  2.1.3
7.02.2.22  2.2.4
7.12.4.18  2.2.4
7.22.4.18  2.2.4
7.3    2.4.18  2.2.5
8.02.4.18  2.2.93

Thanks,
John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Solved - Re: Segmentation violation (even with ulimit -s 2048)

2002-12-02 Thread John Neffenger
It turns out the segmentation violations were due to a bug in the Linux 
Glibc library.  We applied the following patch on November 6, and our 
pure Java server has not crashed since:

  Updated glibc packages fix vulnerabilities in resolver
  https://rhn.redhat.com/errata/RHSA-2002-197.html

We started hitting the problem when we switched DNS servers from our 
provider's name servers to a local BIND Version 9.2.1 server (Red Hat's 
bind-9.2.1-0.70.2 package).

The tip-off, thanks to Juergen Kreileder, was finding a running Java 
thread in the following location on the occasions where the system would 
give us a full Java thread dump:

"ThreadName" (TID:0x43576778, sys_thread_t:0x83ca620, state:R) prio=5
at java.net.InetAddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress.getAddressFromNameService(InetAddress.java:619)
at java.net.InetAddress.getAllByName0(InetAddress.java:568)
at java.net.InetAddress.getAllByName0(InetAddress.java:540)
at java.net.InetAddress.getByName(InetAddress.java:467)
at java.net.InetAddress.getAllByName(InetAddress.java:530)

If you're running Blackdown on Linux and doing DNS requests from your 
application, you may want the Glibc fix as well.

John Neffenger

John Neffenger wrote:
I had been running Blackdown 1.3.1-FCS (green threads, nojit) without 
any problems for over a year on a system with Linux Kernel version 
2.2.16 and Glibc 2.2.4 on a Red Hat Linux 7.0 base.  I've been using the 
"ulimit -s 2048" setting to avoid the problem that caused segmentation 
violations earlier in 2001 (Sun Bug Id 4466587).

Now I've upgraded to all the very latest Red Hat Linux 7.0 updates:

  Blackdown 1.3.1-02b-FCS (green theads, nojit)
  kernel-2.2.22-7.0.2
  glibc-2.2.4-18.7.0.7

The version strings for Blackdown and the kernel are:

  Classic VM (build Blackdown-1.3.1-02b-FCS, green threads, nojit)
  Linux www.volano.com 2.2.22-7.0.2 #1 Mon Sep 23 06:55:40 EDT 2002 i686

As before, I'm still setting:

  ulimit -s 2048
  ulimit -n 1024

and I still start up Blackdown with the options "-green -Xmx256m 
-Xss64k".  But now I'm getting daily segmentation violations on our 
server -- one or two per day:

  SIGSEGV   11*  segmentation violation
si_signo [11]: SIGSEGV   11*  segmentation violation
si_errno [0]: Success
si_code [0]: SI_USER [pid: 0, uid: 0]
stackpointer=0x523f0528

I get either a nice Java thread stack trace to standard error or a 
300-megabyte core dump with no messages.  I've been unable to reproduce 
the problem in any simplified test cases.  So far, it happens only 
online with real client connections.  Any tips on how I might get useful 
information from the core dumps?

Is anyone successfully using Blackdown 1.3.1-02b-FCS (green threads, 
nojit) with the latest Red Hat Linux 7.0 Kernel and Glibc packages 
(Kernel 2.2.22, Glibc 2.2.4)?

Would I be better off switching to the combination of Linux Kernel 
2.4.18 and Glibc 2.2.4, as in Red Hat Linux 7.2?  I guess I'm curious 
which kernel and library versions are getting the most testing by 
Blackdown developers among all these combinations, shown with their 
latest updated version numbers:

  Red Hat  Kernel  Glibc
  ---  --  --
6.22.2.22  2.1.3
7.02.2.22  2.2.4
7.12.4.18  2.2.4
7.2    2.4.18  2.2.4
7.32.4.18  2.2.5
8.02.4.18  2.2.93

Thanks,
John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]




Re: per jvm thread limit

2003-02-13 Thread John Neffenger
Hi Joseph,

Here's a good explanation about the space Java reserves for each thread 
stack.  It's written about Windows, but applies to Solaris, Linux and 
FreeBSD as well:

Connection scaling in Java
http://www.javaworld.com/javaworld/jw-03-1999/jw-03-volanomark_p.html#sidebar

It's just a matter of dividing the virtual address space (2, 4, or more 
gigabytes) by the reserved stack size, and then leaving some extra room 
for the program, the Java heap, and other things that have to fit as well.

The memory is not actually allocated, but it does get reserved, and you 
eventually run out of room.  That's why I appreciate the "-green" option 
on the Blackdown Java VM.  With user-level threads, you can pretty much 
have all the threads you want (or, on Java 1.3 without the new I/O, all 
the threads you need).

John Neffenger

Joseph Shraibman wrote:
I'm running:
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

... on redhat 8.0.  I've discovered that I get get somewhere between 
3600 and 3700 threads in a jvm without any arguments.  If I pass in 
-Xss100k I can get that up to 8190 threads, but I'm not sure why.  How 
can I maximize the number of threads per jvm?


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]




Re: per jvm thread limit

2003-02-14 Thread John Neffenger
Hi Joseph,


(See also http://www.kegel.com/c10k.html )


That points to http://www.volano.com/linux.html which doesn't exist 
anymore. The wayback machine has a verion from Oct 31 2001 that doesn't 
say how to recompile the kernel to increase the limit.  It does point me 
to /etc/security/limits.conf, but I don't have anything set there.

I removed that page since I think it's obsolete.  Fortunately, I haven't 
had to go recompile the Kernel or the Glibc library in years.

Using the following options with Blackdown 1.3.1-02b-FCS on Red Hat 
Linux 8.0, I got 10,000 active socket connections (with 20,000 threads) 
on a 500-MHz Intel Pentium III with 384 megabytes of RAM:

  /usr/local/j2sdk1.3.1/bin/java -green -Xmx256m -Xss64k ...

All I had to do was increase the per-process file descriptor limits for 
all the socket connections, by doing the following.

Add to /etc/pam.d/login:

  sessionrequired /lib/security/pam_limits.so

Add to /etc/security/limits.conf:

  *   softnofile  1024
  *   hardnofile  10240

Don't forget to add a "ulimit -n 10240" to your startup script.

John Neffenger



--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Re: per jvm thread limit

2003-02-14 Thread John Neffenger
Hi Dan,


Note to readers: he's using a smaller thread stack than the
previous poster (64k vs. 100k).  At 1GB address space limit
(or 2^30 bytes), that's an upper limit of 2^(30-16 = 14) = 16000
or so threads.


Further note to readers:  I got past this 16,000 native thread limit by 
specifying the "-green" user-level thread option to the Blackdown 
virtual machine.  So my 20,000 threads were all mapped onto one Linux 
process (one Linux native thread).

I'm eager to find out what kind of socket and thread limits we'll hit 
when we test the Java 1.4 new I/O implementations on Windows, Solaris, 
Linux, and FreeBSD.

Our next Volano Report will have results up to 10,000 connections using 
the "old I/O" -- a place where, so far in our tests, only Blackdown and 
BEA JRockit can go.  There may be some options, that I just recently 
discovered, which we can use to get Sun's HotSpot Server VM on Solaris 
there, too:

  http://java.sun.com/docs/hotspot/PerformanceFAQ.html#5

  Java and Solaris Threading
  http://java.sun.com/docs/hotspot/threads/threads.html

John Neffenger


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Volano Report update

2003-06-01 Thread John Neffenger
I updated our VolanoMark benchmark test results yesterday, comparing 17 
Java virtual machines and four operating systems on identical hardware:

  The Volano Report
  http://www.volano.com/report/index.html
It's a new, faster version of VolanoMark running on faster hardware, 
updated operating systems, and the latest Java 1.3 and 1.4 virtual 
machines from BEA, Blackdown, IBM, and Sun.  I tested up to 10,000 
connections this time.  Back in 1998 I could test to only 900 
connections, and most of the Java platforms couldn't even make it past 
500 connections, so we've come a long way!  Let me know if you see any 
tricks or command-line options I missed that would be helpful.

Now it's time to drop Java 1.3 and see how the "new I/O" implementations 
perform way beyond the 10,000 connection level on the modest Dell 
OptiPlex hardware we use.  I'm curious to compare the message throughput 
at 50,000 connections on a new Hyper-Threading Pentium 4 with a gigabyte 
of dual-channel memory, for example.

John Neffenger

--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]


Re: Volano Report update

2003-06-04 Thread John Neffenger
Ciao Marco,

- you tested Blackdown-1.3.1 using green threads, but Sun JVM used
native threads. In Linux this makes a huge difference in terms of thread
scalability with one CPU. It should be useful to show the Linux Sun
1.3.1 JVM results with -green option.
That's an easy test result to add. :)

  [EMAIL PROTECTED] john]$ /usr/java/jdk1.3.1_07/bin/java -green -version
  Green threads support not available
  Could not create the Java virtual machine.
Even if it could run with green threads, Sun never fixed the following 
bug and therefore could never handle more than 1,018 concurrent 
connections with the Classic VM:

  Bug Id 4427986,
  "Classic VM goes into hard run at 1018 socket connections"
  http://developer.java.sun.com/developer/bugParade/bugs/4427986.html
- It should be better to use Solaris 9 instead of Solaris 8, since the
threading model has changed.
I'm going to upgrade to Solaris 9 soon, but according to Sun's 
documention we should expect results similar to the "/usr/lib/lwp" rows 
in this table:

  Table 6: Performance comparison of Solaris threading options
  http://www.volano.com/report/index.html#sunthreads
The alternative thread library (in /usr/lib/lwp) gives you the same 
one-to-one threading model on Solaris 8 that you get by default with 
Solaris 9.  See the section called "Combinatorial review" here:

  Java and Solaris Threading
  http://java.sun.com/docs/hotspot/threads/threads.html
Actually, we're going to make all the following upgrades soon:

  FreeBSD 4.6 -> FreeBSD 5.0
  Red Hat Linux 8.0 -> Red Hat Enterprise Linux AS 2.1
  Solaris 8 -> Solaris 9
  Windows 2000 Advanced Server -> Windows Server 2003
Between 17 Java VMs and 4 operating systems, it's really hard to get the 
results published and have the very latest of everything.  We also try 
to hold back a little bit on the upgrades so that we are still using 
what our customers might have.

- It could be interesting to show the results using Sparc HW, in order
to see how things change from Solaris 9-x86 to Solaris9-Sparc with the
64-bit JVM.
I think I can compare 32-bit and 64-bit on the same machine if I go the 
AMD route, right?  Intel is going to do something similar by emulating 
the 32-bit along with 64-bit, right?  I prefer testing everything on 
identical hardware.

I tested on Intel and SPARC for an old Java World magazine article, but 
it doesn't help much.  Once you start changing the hardware, the results 
no longer tell you which *Java VM* is the best choice.  We run the 
identical application on identical hardware so it's always an "apples to 
apples" comparison.  The SPEC benchmarks are just hardware tests now 
because of the way they allow the results to be published.  Try to pick 
the best Java virtual machine from this set of results, for example:

  Second Quarter 2003 SPEC JBB2000 Results
  http://www.spec.org/jbb2000/results/res2003q2/
And this doesn't help much either:

  Configurable Request
  http://www.spec.org/cgi-bin/osgresults?conf=jbb2000;op=form
John Neffenger

--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]


Re: Volano Report update

2003-06-04 Thread John Neffenger
Hi Bill,

Also, 2.5.x Linux has changes to their 1:1 model that greatly increases
scalability for dealing with a large number of threads.
I was under the impression that Java application developers will have to 
wait for the Java virtual machines to take advantage of those threading 
changes before we see any benefits.  I read that IBM needs to make 
changes in their Java VM on Linux to support it, for example.  Is that 
correct?

John Neffenger

--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]


Re: Volano Report update

2003-06-04 Thread John Neffenger
Hi Juergen,

I'm not if anyone working on the blackdown project is planning to
get support for NPTL in ahead of Sun or IBM.
It already works.
Do you mean that the Sun and Blackdown native-thread HotSpot Server VMs 
will be able to go right up to 10,000 connections (about 20,000 threads) 
right out of the box on Red Hat Linux 9?  (On Red Hat Linux 8.0, they 
both fail miserably just over 1,000 connections.)  I've downloaded the 
Red Hat installation CD-ROMs, but I didn't think it was worth even 
trying it yet, figuring I had to wait for Java 1.5 or later for the 
vendors to support it.

If Java and NPTL are working together already, that changes everything. 
 Just when I thought I was done testing for a while ...

John Neffenger

--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]