On Oct 6, 2007, at 12:03 PM, Chris Chen wrote:

> Rather than doing all the profiling like you (which I would have
> liked to but i was too lazy), I just did a thread dump.  It appeared
> to be occurring within the VFS package.  Resin 3.1.2 under Linux
> CentOS 4.x was spending FOREVER inside the Jar loading code.
> Anyways, try installing the newest snapshot version (not the stable
> 3.1.2 release, which apparently was too stable for my own good) and
> see if it solves your problem.

That's true.  The new release has major performance fixes to the jar  
dependency checking.

-- Scott

> Thanks,
> Chris
> On Oct 6, 2007, at 4:03 AM, Mattias Jiderhamn wrote:
>> Thanks for all the suggestions.
>> Serge wrote:
>>> I've seen something like this, and we were able to diagnose it on
>>> a Unix
>>> environment by using a program called truss
>>> (http://www.scit.wlv.ac.uk/cgi-bin/mansec?1+truss) which allows
>>> you to
>>> watch every file that is getting read.  I don't know of a Windows
>>> equivalent.  Most Java profilers I know of do not help you find IO
>>> related issues very well, so you might want to look for a lower
>>> level OS
>>> tool to see what's changing.
>>> What we found was that the application was looking for a class
>>> that did
>>> not exist.  Java does not hold a cache entry for "class not
>>> found", so
>>> it kept opening every jar in the entire classpath to find said  
>>> class,
>>> numerous times for a single http request because of where this
>>> library
>>> was used.  You might want to check all the resin/lib and WEB-INF/
>>> lib and
>>> whatever other classes to see if a non-critical class is missing but
>>> being searched for over and over.
>> I have watched file access using FileMon on Windows. Unfortunately, I
>> don't get enough information to see if I have the same problem you
>> had.
>> But since the inital start of the application is also slow, I lean
>> towards there is a generic problem with class loading and/or
>> dependency
>> checking.
>> There seem to occur several depenency checks during a single request,
>> which is why a simple page can take 40 seconds. If I increase the
>> dependency check intervall, so that there is only one check during  
>> the
>> request monitored by FileMon, it will finish much faster.
>> Scott wrote:
>>> Do you see a difference if you remove the resin.dll? (Which would
>>> disable the JNI.)
>> Not much.
>>> Also, what is the stack trace for the slow read() call?
>> With JNI, 57% of the time for loading a single page (total 78 s) is
>> spent in
>> com.caucho.vfs.JniStream.read cased by (only!!!) 22 calls from
>>   com.caucho.vfs.ReadStream.readBuffer
>>   com.caucho.vfs.ReadStream.waitForRead
>>   com.caucho.server.port.TcpConnection.run
>>   ...
>> Without JNI 43% of the time (total 33 s) is spent in
>> java.io.File.lastModified caused by 28000 calls (depency checking  
>> over
>> and over) from
>>   com.caucho.vfs.FilePath.getLastModified
>>   com.caucho.vfs.Depend.isModified
>>   ...
>> and 41% of the time is spent in
>> java.io.File.length called 28000 times from
>>   com.caucho.vfs.FilePath.getLength
>>   com.caucho.vfs.Depend.isModified
>>   ...
>> I created a simple test page, calling File.length and
>> File.lastModified
>> 100'000 times and measured the time. I then ran the page in another
>> webapp, that does not seem to be affected (on the same computer
>> though).
>> The results were the same (about 0.2 ms / operation - not sure if  
>> this
>> is normal though; I'll have to compare to the office computer next
>> week).
>> Anyway, this could indicate the problem is not with the I/O calls
>> themselves, but the number of I/O calls... (Which makes Serges tip
>> more
>> interesting.) Again, I will have to profile at the other computer  
>> next
>> week and compare.
>> Tom Hintz wrote:
>>> I’ve seen small TCP/IP MTU sizes cause similar behavior.  We had VPN
>>> software that set the MTU to less than the IP default, which I think
>>> should have been 1500(?).  Result was packet fragmentation that
>>> caused
>>> huge delays in the Microsoft TCP implementation.  I think the MTU
>>> setting, in this case, was in the VPN software.
>> I have not run any VPN software other than the built in PPTP
>> client, and
>> have not changed the MTU of the NICs. (MTU isn't fetched via DHCP
>> is it?)
>> I thought maybe there was a problem fetching external DTDs when
>> reading
>> Hibernate mappings or whatever (lots of profiling time spent
>> parsing doc
>> types and my ISP has had some issues lately). However, inactivating
>> the
>> NIC altogether makes no difference.
>>> By the way, the CPU usage is peaking all the way through. Around
>>> 49% on
>>> my dual core system.
>>> (And there is even more memory on the computer with the problem
>>> than the
>>> other one)
>>>> -----Original Message-----
>>>> [mailto:[EMAIL PROTECTED] Behalf Of Mattias
>>>> Jiderhamn
>>>> Sent: Friday 05 October 2007 11:20
>>>> To: Resin
>>>> Subject: [Resin-interest] Slooow file reads (really weird!)
>>>> Hi list.
>>>> I have my J2EE webapp on an external hard drive, which I carry
>>>> between
>>>> my office and my home computer.
>>>> On each computer - running Windows XP and Java 1.5 - I have a Resin
>>>> (3.0.22) installation and a shortcut to start Resin with the
>>>> server root
>>>> on the external drive.
>>>> This has worked flawlessly for over a year.
>>>> Now suddenly (ok, after returning from vacation), the  
>>>> application is
>>>> immensely slow - on one of the computers!
>>>> Starting Resin and the application now takes anywhere from 3 to 5
>>>> minutes on my home computer, compared to the usual 30 or so  
>>>> seconds.
>>>> Loading a simple page can take 40 seconds. It seems that most of  
>>>> the
>>>> time is spent inside the disk access of the dependency checking.
>>>> (Turning dependency checking off was much faster, but still
>>>> slower than
>>>> normal).
>>>> I tried to copy the project to the internal drive to see if there
>>>> was
>>>> some interface hardware issue - no difference.
>>>> I profiled the application with JProfiler, and it thinks there is a
>>>> hotspot in com.caucho.vsf.JniStream.read(). Why...?
>>>> I am running out of ideas on what could be wrong and how to track
>>>> it down.
>>>> Any tips would be much welcome!
>>>>   /Mattias
>> _______________________________________________
>> resin-interest mailing list
>> resin-interest@caucho.com
>> http://maillist.caucho.com/mailman/listinfo/resin-interest
> _______________________________________________
> resin-interest mailing list
> resin-interest@caucho.com
> http://maillist.caucho.com/mailman/listinfo/resin-interest

resin-interest mailing list

Reply via email to