Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Some additional information: I tried installing 4.0.5 on my better server (Resin-4.0.0, Nexenta/Open Solaris, recent, lots of RAM, faster CPUs), and got similar behavior. I got it up and running, noticed some oddness at first (java link errors running my app, slow page loads the first few times), then it seemed to get normal. Then about 5 minutes later, it stopped responding. I didn't investigate, but rather just switched back to using resin-4.0.0. No idea if it was the same issue. In the logs, I saw it restarting, but it never came back. -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] configure not finding 64-bit java
I only have one JDK installed. On Mar 18, 2010, at 21:32:14, Tyson Weihs wrote: > Are you pointing ./configure to the right JDK installation via > "--with-java-home=DIR"? > > On Thu, Mar 18, 2010 at 11:28 PM, Rick Mann wrote: > Hmm, configure still reports > >checking if Java is 64-bit... no > > > On Mar 18, 2010, at 20:40:38, Tyson Weihs wrote: > > > Add "--enable-64bit" when calling ./configure. > > > > -tyson > > > > On Thu, Mar 18, 2010 at 10:33 PM, Rick Mann wrote: > > Hi. I'm pretty sure I have a 64-bit java running on my Nxenta/Open Solaris > > box. > > > >SunOS -- 5.11 NexentaOS_20081207 i86pc i386 i86pc Solaris > > > > However, when preparing to install resin 4.0.5 on it, I notice the > > configure script didn't find 64-bit java. However: > > > > $ java -version > > java version "1.6.0_12" > > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > > Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode) > > > > $ java -d64 -version > > java version "1.6.0_12" > > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > > Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) > > > > > > Do I need to install a different JVM, or should configure add -d64 when > > checking? > > > > TIA, > > Rick > > > > > > > > ___ > > resin-interest mailing list > > resin-interest@caucho.com > > http://maillist.caucho.com/mailman/listinfo/resin-interest > > > > ___ > > resin-interest mailing list > > resin-interest@caucho.com > > http://maillist.caucho.com/mailman/listinfo/resin-interest > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] configure not finding 64-bit java
Are you pointing ./configure to the right JDK installation via "--with-java-home=DIR"? On Thu, Mar 18, 2010 at 11:28 PM, Rick Mann wrote: > Hmm, configure still reports > >checking if Java is 64-bit... no > > > On Mar 18, 2010, at 20:40:38, Tyson Weihs wrote: > > > Add "--enable-64bit" when calling ./configure. > > > > -tyson > > > > On Thu, Mar 18, 2010 at 10:33 PM, Rick Mann > wrote: > > Hi. I'm pretty sure I have a 64-bit java running on my Nxenta/Open > Solaris box. > > > >SunOS -- 5.11 NexentaOS_20081207 i86pc i386 i86pc Solaris > > > > However, when preparing to install resin 4.0.5 on it, I notice the > configure script didn't find 64-bit java. However: > > > > $ java -version > > java version "1.6.0_12" > > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > > Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode) > > > > $ java -d64 -version > > java version "1.6.0_12" > > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > > Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) > > > > > > Do I need to install a different JVM, or should configure add -d64 when > checking? > > > > TIA, > > Rick > > > > > > > > ___ > > resin-interest mailing list > > resin-interest@caucho.com > > http://maillist.caucho.com/mailman/listinfo/resin-interest > > > > ___ > > resin-interest mailing list > > resin-interest@caucho.com > > http://maillist.caucho.com/mailman/listinfo/resin-interest > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] configure not finding 64-bit java
Hmm, configure still reports checking if Java is 64-bit... no On Mar 18, 2010, at 20:40:38, Tyson Weihs wrote: > Add "--enable-64bit" when calling ./configure. > > -tyson > > On Thu, Mar 18, 2010 at 10:33 PM, Rick Mann wrote: > Hi. I'm pretty sure I have a 64-bit java running on my Nxenta/Open Solaris > box. > >SunOS -- 5.11 NexentaOS_20081207 i86pc i386 i86pc Solaris > > However, when preparing to install resin 4.0.5 on it, I notice the configure > script didn't find 64-bit java. However: > > $ java -version > java version "1.6.0_12" > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode) > > $ java -d64 -version > java version "1.6.0_12" > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) > > > Do I need to install a different JVM, or should configure add -d64 when > checking? > > TIA, > Rick > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] configure not finding 64-bit java
Ah! Thank you! On Mar 18, 2010, at 20:40:38, Tyson Weihs wrote: > Add "--enable-64bit" when calling ./configure. > > -tyson > > On Thu, Mar 18, 2010 at 10:33 PM, Rick Mann wrote: > Hi. I'm pretty sure I have a 64-bit java running on my Nxenta/Open Solaris > box. > >SunOS -- 5.11 NexentaOS_20081207 i86pc i386 i86pc Solaris > > However, when preparing to install resin 4.0.5 on it, I notice the configure > script didn't find 64-bit java. However: > > $ java -version > java version "1.6.0_12" > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode) > > $ java -d64 -version > java version "1.6.0_12" > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) > > > Do I need to install a different JVM, or should configure add -d64 when > checking? > > TIA, > Rick > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] configure not finding 64-bit java
Add "--enable-64bit" when calling ./configure. -tyson On Thu, Mar 18, 2010 at 10:33 PM, Rick Mann wrote: > Hi. I'm pretty sure I have a 64-bit java running on my Nxenta/Open Solaris > box. > >SunOS -- 5.11 NexentaOS_20081207 i86pc i386 i86pc Solaris > > However, when preparing to install resin 4.0.5 on it, I notice the > configure script didn't find 64-bit java. However: > > $ java -version > java version "1.6.0_12" > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode) > > $ java -d64 -version > java version "1.6.0_12" > Java(TM) SE Runtime Environment (build 1.6.0_12-b04) > Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) > > > Do I need to install a different JVM, or should configure add -d64 when > checking? > > TIA, > Rick > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] configure not finding 64-bit java
Hi. I'm pretty sure I have a 64-bit java running on my Nxenta/Open Solaris box. SunOS -- 5.11 NexentaOS_20081207 i86pc i386 i86pc Solaris However, when preparing to install resin 4.0.5 on it, I notice the configure script didn't find 64-bit java. However: $ java -version java version "1.6.0_12" Java(TM) SE Runtime Environment (build 1.6.0_12-b04) Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode) $ java -d64 -version java version "1.6.0_12" Java(TM) SE Runtime Environment (build 1.6.0_12-b04) Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode) Do I need to install a different JVM, or should configure add -d64 when checking? TIA, Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Resin keeps restarting...
I have a server that Resin keeps restarting every 15-20 minutes. Where should we start to look to see what is going on. I just enabled full debug logging so starting to gather that information. I'd like to know any pointers we should start looking at once some data is captured. -- Peter Amiri Founder | Alurium Hosting 949-338-3862 | pe...@alurium.com | http://www.alurium.com LinkedIn: http://www.linkedin.com/in/peteramiri | IM bpamiri (AIM/MSN/Y!/GTLK/SKYPE) ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
On Mar 18, 2010, at 16:30:02, Scott Ferguson wrote: > And it may be better to set the for your older OS to > something smaller, like 10 or 15, because that OS may not be as good > with lots of threads. 10 seems to have no effect on the overall stability. -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
On Mar 18, 2010, at 16:30:02, Scott Ferguson wrote: > That trace looks normal, assuming the *.216 and *.217 are your two HTTP > ports. I assume that's the connection-refused state? Each one has > several threads in the nativeAccept() state. The situation that would be > a problem is if there are no threads for the port waiting in accept(). > > When you're in a connection-refused, can you try both addresses, just to > make sure? (It's possible, too, that netstat might show something in > that state.) I got to that connection-refused state again, and tried connecting to each IP address separately. They both behaved the same way, although apparently the server isn't completely dead. Every now and again, it will respond. But attempting to load a web page, it's obvious that many of the connections are being refused (no CSS, images, etc). netstat -an output in that state (sorry, I might've forgotten an option on that command): http://pastie.org/private/bttcmdyixr023xasvvsdzq -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
On Mar 18, 2010, at 16:30:02, Scott Ferguson wrote: > Rick Mann wrote: >> Hmm, I can't seem to post longer email to the list. Here's a link to my last >> post: >> >> http://pastie.org/private/2c4lrjuj4oybdtzmj4gbzw >> > Thanks. There is a size limit for the list. > > That trace looks normal, assuming the *.216 and *.217 are your two HTTP > ports. I assume that's the connection-refused state? Each one has > several threads in the nativeAccept() state. The situation that would be > a problem is if there are no threads for the port waiting in accept(). > > When you're in a connection-refused, can you try both addresses, just to > make sure? (It's possible, too, that netstat might show something in > that state.) Will do. It's currently responding, so I'll have to wait. > The JNI is used in Resin-OS for ports/sockets because we needed the JNI > capability to handle the setuid (and we decided setuid really should be > in Resin-OS.) That's great! Thank you for that. > And it may be better to set the for your older OS to > something smaller, like 10 or 15, because that OS may not be as good > with lots of threads. In the past, there have always been a lot of threads spawned. I wanted to limit the number of threads here to see, but I always have trouble finding the right configuration stuff in the website docs. I wanted to search for "thread" because I've seen that for older versions, but my experience is that links are broken (i.e. http://caucho.com/resin-4.0/admin/index-tags.xtp, http://caucho.com/resin-4.0/admin/cluster-tags), or the index doesn't fully reflect the available tags, etc. I do wish there was a more comprehensive document describing configuration. > > -- Sctt >> >> On Mar 18, 2010, at 15:33:08, Rick Mann wrote: >> >> >>> I replied with the thread dump, but don't see it appearing in the list. >>> Seems like the list server is flaky. >>> >>> On Mar 18, 2010, at 14:37:25, Scott Ferguson wrote: >>> >>> Rick Mann wrote: > Slower is subjective. Unfortunately, I can't currently run the old server > to actually measure times, but it's just slow. I can see individual > requests (images, CSS, etc) taking a long time to be fulfilled, whereas > before, the pages would just come up fully rendered. Time to first render > is longer, too. > > "finer" logging might show the issue, but there's a good chance it's related to the connection issue. > I left the server running late last night, tried to make requests of it > this morning, but I just got "server not responding" errors in the > browser. Mind you, these were not timeouts, but actual connection > refusals. > > Thanks. Refusing connections is a more helpful description. If possible, can you get a thread dump in that situation? There are a couple of possible scenarios: 1. After a while, Resin no longer has a thread listening for an accept(). (for example, if it doesn't properly spawn a new thread.) 2. The JVM is truly frozen. It is possible, by the way, that the older Linux is having trouble with the threading. They made huge improvements in the kernel's threading after the version you're using. > logs show nothing with this config: > > > > > > Ok, although, the "warning" level would only show pretty serious issues. > So, that's an interesting and scary thought. However, I'm not using Resin > Professional, so I thought I didn't get native sockes. > > Ok. That's one less variable. The main issue is the connection failures. That could cause your slowness as well. For example, if only one thread is listening for connections for some reason, the server would appear to be very slow. -- Scott >>> root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 >>> /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf >>> /var/resin/resin.xml -log-directory /var/logs/resin -root-directory >>> /var/resin/root -J-verbosegc console >>> >>> root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 >>> /usr/local/java-versions/jdk1.6.0_07/bin/java -server >>> -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 >>> -Djava.util.logging.manager=com.caucho.log.LogManagerImpl >>> -Djava.system.class.loader=com.caucho.loader.SystemClassLoader >>> -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl >>> -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m >>> -Xmx256m -verbosegc -verbosegc com.caucho.server.resin.Resin >>> --root-directory /var/resin/root -conf /var/resin/resin.xml -socketwait >>> 34280 -log-directory /var/logs/resin -root-directory /var/resin/root >>> console >>> >>> >>> I can't find where the -Xss1m and -
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Rick Mann wrote: > Hmm, I can't seem to post longer email to the list. Here's a link to my last > post: > > http://pastie.org/private/2c4lrjuj4oybdtzmj4gbzw > Thanks. There is a size limit for the list. That trace looks normal, assuming the *.216 and *.217 are your two HTTP ports. I assume that's the connection-refused state? Each one has several threads in the nativeAccept() state. The situation that would be a problem is if there are no threads for the port waiting in accept(). When you're in a connection-refused, can you try both addresses, just to make sure? (It's possible, too, that netstat might show something in that state.) The JNI is used in Resin-OS for ports/sockets because we needed the JNI capability to handle the setuid (and we decided setuid really should be in Resin-OS.) And it may be better to set the for your older OS to something smaller, like 10 or 15, because that OS may not be as good with lots of threads. -- Sctt > > On Mar 18, 2010, at 15:33:08, Rick Mann wrote: > > >> I replied with the thread dump, but don't see it appearing in the list. >> Seems like the list server is flaky. >> >> On Mar 18, 2010, at 14:37:25, Scott Ferguson wrote: >> >> >>> Rick Mann wrote: >>> Slower is subjective. Unfortunately, I can't currently run the old server to actually measure times, but it's just slow. I can see individual requests (images, CSS, etc) taking a long time to be fulfilled, whereas before, the pages would just come up fully rendered. Time to first render is longer, too. >>> "finer" logging might show the issue, but there's a good chance it's >>> related to the connection issue. >>> I left the server running late last night, tried to make requests of it this morning, but I just got "server not responding" errors in the browser. Mind you, these were not timeouts, but actual connection refusals. >>> Thanks. Refusing connections is a more helpful description. >>> >>> If possible, can you get a thread dump in that situation? There are a >>> couple of possible scenarios: >>> >>> 1. After a while, Resin no longer has a thread listening for an >>> accept(). (for example, if it doesn't properly spawn a new thread.) >>> 2. The JVM is truly frozen. >>> >>> It is possible, by the way, that the older Linux is having trouble with >>> the threading. They made huge improvements in the kernel's threading >>> after the version you're using. >>> logs show nothing with this config: >>> Ok, although, the "warning" level would only show pretty serious issues. >>> So, that's an interesting and scary thought. However, I'm not using Resin Professional, so I thought I didn't get native sockes. >>> Ok. That's one less variable. The main issue is the connection failures. >>> That could cause your slowness as well. For example, if only one thread >>> is listening for connections for some reason, the server would appear to >>> be very slow. >>> >>> -- Scott >>> >> root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 >> /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf >> /var/resin/resin.xml -log-directory /var/logs/resin -root-directory >> /var/resin/root -J-verbosegc console >> >> root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 >> /usr/local/java-versions/jdk1.6.0_07/bin/java -server >> -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 >> -Djava.util.logging.manager=com.caucho.log.LogManagerImpl >> -Djava.system.class.loader=com.caucho.loader.SystemClassLoader >> -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl >> -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m >> -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory >> /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 >> -log-directory /var/logs/resin -root-directory /var/resin/root console >> >> >> I can't find where the -Xss1m and -Xmx256m are being set. According to >> the admin manual, it should default to -Xss2m. >> >> >> > You can change that in the resin.xml with a -Xss1m. > > I used to have exactly those values specified in resin.xml, and thought perhaps that was part of the problem, so I commented them out (to let the JVM and resin pick defaults). I was still seeing those in the command, though, and didn't know where they were coming from. That was something I had not been specifying in my 3.0.23 installation, and so I thought they were causing more problems than they were solving. ___ resin-interest mailing list resin-interest@caucho.com >
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Hmm, I can't seem to post longer email to the list. Here's a link to my last post: http://pastie.org/private/2c4lrjuj4oybdtzmj4gbzw On Mar 18, 2010, at 15:33:08, Rick Mann wrote: > I replied with the thread dump, but don't see it appearing in the list. Seems > like the list server is flaky. > > On Mar 18, 2010, at 14:37:25, Scott Ferguson wrote: > >> Rick Mann wrote: >>> Slower is subjective. Unfortunately, I can't currently run the old server >>> to actually measure times, but it's just slow. I can see individual >>> requests (images, CSS, etc) taking a long time to be fulfilled, whereas >>> before, the pages would just come up fully rendered. Time to first render >>> is longer, too. >>> >> "finer" logging might show the issue, but there's a good chance it's >> related to the connection issue. >>> I left the server running late last night, tried to make requests of it >>> this morning, but I just got "server not responding" errors in the browser. >>> Mind you, these were not timeouts, but actual connection refusals. >>> >> Thanks. Refusing connections is a more helpful description. >> >> If possible, can you get a thread dump in that situation? There are a >> couple of possible scenarios: >> >> 1. After a while, Resin no longer has a thread listening for an >> accept(). (for example, if it doesn't properly spawn a new thread.) >> 2. The JVM is truly frozen. >> >> It is possible, by the way, that the older Linux is having trouble with >> the threading. They made huge improvements in the kernel's threading >> after the version you're using. >>> logs show nothing with this config: >>> >>> >>> >>> >>> >> Ok, although, the "warning" level would only show pretty serious issues. >>> So, that's an interesting and scary thought. However, I'm not using Resin >>> Professional, so I thought I didn't get native sockes. >>> >> Ok. That's one less variable. The main issue is the connection failures. >> That could cause your slowness as well. For example, if only one thread >> is listening for connections for some reason, the server would appear to >> be very slow. >> >> -- Scott >>> > root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 > /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf > /var/resin/resin.xml -log-directory /var/logs/resin -root-directory > /var/resin/root -J-verbosegc console > > root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 > /usr/local/java-versions/jdk1.6.0_07/bin/java -server > -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 > -Djava.util.logging.manager=com.caucho.log.LogManagerImpl > -Djava.system.class.loader=com.caucho.loader.SystemClassLoader > -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl > -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m > -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory > /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 > -log-directory /var/logs/resin -root-directory /var/resin/root console > > > I can't find where the -Xss1m and -Xmx256m are being set. According to > the admin manual, it should default to -Xss2m. > > You can change that in the resin.xml with a -Xss1m. >>> >>> I used to have exactly those values specified in resin.xml, and thought >>> perhaps that was part of the problem, so I commented them out (to let the >>> JVM and resin pick defaults). I was still seeing those in the command, >>> though, and didn't know where they were coming from. That was something I >>> had not been specifying in my 3.0.23 installation, and so I thought they >>> were causing more problems than they were solving. >>> >>> >>> >>> >>> >>> ___ >>> resin-interest mailing list >>> resin-interest@caucho.com >>> http://maillist.caucho.com/mailman/listinfo/resin-interest >>> >>> >> >> >> >> ___ >> resin-interest mailing list >> resin-interest@caucho.com >> http://maillist.caucho.com/mailman/listinfo/resin-interest > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
I replied with the thread dump, but don't see it appearing in the list. Seems like the list server is flaky. On Mar 18, 2010, at 14:37:25, Scott Ferguson wrote: > Rick Mann wrote: >> Slower is subjective. Unfortunately, I can't currently run the old server to >> actually measure times, but it's just slow. I can see individual >> requests (images, CSS, etc) taking a long time to be fulfilled, whereas >> before, the pages would just come up fully rendered. Time to first render is >> longer, too. >> > "finer" logging might show the issue, but there's a good chance it's > related to the connection issue. >> I left the server running late last night, tried to make requests of it this >> morning, but I just got "server not responding" errors in the browser. Mind >> you, these were not timeouts, but actual connection refusals. >> > Thanks. Refusing connections is a more helpful description. > > If possible, can you get a thread dump in that situation? There are a > couple of possible scenarios: > > 1. After a while, Resin no longer has a thread listening for an > accept(). (for example, if it doesn't properly spawn a new thread.) > 2. The JVM is truly frozen. > > It is possible, by the way, that the older Linux is having trouble with > the threading. They made huge improvements in the kernel's threading > after the version you're using. >> logs show nothing with this config: >> >> >> >> >> > Ok, although, the "warning" level would only show pretty serious issues. >> So, that's an interesting and scary thought. However, I'm not using Resin >> Professional, so I thought I didn't get native sockes. >> > Ok. That's one less variable. The main issue is the connection failures. > That could cause your slowness as well. For example, if only one thread > is listening for connections for some reason, the server would appear to > be very slow. > > -- Scott >> root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf /var/resin/resin.xml -log-directory /var/logs/resin -root-directory /var/resin/root -J-verbosegc console root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 /usr/local/java-versions/jdk1.6.0_07/bin/java -server -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 -Djava.util.logging.manager=com.caucho.log.LogManagerImpl -Djava.system.class.loader=com.caucho.loader.SystemClassLoader -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 -log-directory /var/logs/resin -root-directory /var/resin/root console I can't find where the -Xss1m and -Xmx256m are being set. According to the admin manual, it should default to -Xss2m. >>> You can change that in the resin.xml with a -Xss1m. >>> >> >> I used to have exactly those values specified in resin.xml, and thought >> perhaps that was part of the problem, so I commented them out (to let the >> JVM and resin pick defaults). I was still seeing those in the command, >> though, and didn't know where they were coming from. That was something I >> had not been specifying in my 3.0.23 installation, and so I thought they >> were causing more problems than they were solving. >> >> >> >> >> >> ___ >> resin-interest mailing list >> resin-interest@caucho.com >> http://maillist.caucho.com/mailman/listinfo/resin-interest >> >> > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Rick Mann wrote: > Slower is subjective. Unfortunately, I can't currently run the old server to > actually measure times, but it's just slow. I can see individual requests > (images, CSS, etc) taking a long time to be fulfilled, whereas before, the > pages would just come up fully rendered. Time to first render is longer, too. > "finer" logging might show the issue, but there's a good chance it's related to the connection issue. > I left the server running late last night, tried to make requests of it this > morning, but I just got "server not responding" errors in the browser. Mind > you, these were not timeouts, but actual connection refusals. > Thanks. Refusing connections is a more helpful description. If possible, can you get a thread dump in that situation? There are a couple of possible scenarios: 1. After a while, Resin no longer has a thread listening for an accept(). (for example, if it doesn't properly spawn a new thread.) 2. The JVM is truly frozen. It is possible, by the way, that the older Linux is having trouble with the threading. They made huge improvements in the kernel's threading after the version you're using. > logs show nothing with this config: > > > > > Ok, although, the "warning" level would only show pretty serious issues. > So, that's an interesting and scary thought. However, I'm not using Resin > Professional, so I thought I didn't get native sockes. > Ok. That's one less variable. The main issue is the connection failures. That could cause your slowness as well. For example, if only one thread is listening for connections for some reason, the server would appear to be very slow. -- Scott > >>> root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 >>> /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf >>> /var/resin/resin.xml -log-directory /var/logs/resin -root-directory >>> /var/resin/root -J-verbosegc console >>> >>> root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 >>> /usr/local/java-versions/jdk1.6.0_07/bin/java -server >>> -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 >>> -Djava.util.logging.manager=com.caucho.log.LogManagerImpl >>> -Djava.system.class.loader=com.caucho.loader.SystemClassLoader >>> -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl >>> -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m >>> -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory >>> /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 -log-directory >>> /var/logs/resin -root-directory /var/resin/root console >>> >>> >>> I can't find where the -Xss1m and -Xmx256m are being set. According to the >>> admin manual, it should default to -Xss2m. >>> >>> >> You can change that in the resin.xml with a -Xss1m. >> > > I used to have exactly those values specified in resin.xml, and thought > perhaps that was part of the problem, so I commented them out (to let the JVM > and resin pick defaults). I was still seeing those in the command, though, > and didn't know where they were coming from. That was something I had not > been specifying in my 3.0.23 installation, and so I thought they were causing > more problems than they were solving. > > > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > > ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
By the way, the server-dying thing seems to happen within just a couple of hours after launch. I launched it before lunch, and checked it just now, and it was dead again. ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Thanks for getting back to me, Scott! Responses below On Mar 18, 2010, at 12:37:21, Scott Ferguson wrote: > Rick Mann wrote: >> My 4.0.5 install is really very slow. It just won't stay running (although >> ps shows many resin threads), and it's much, much slower than 3.0.23 was. >> I'm actually running fewer webapps currently, and my server is always idle. >> > How are you measuring the "slower"? (ab? latency? startup? some > particular page size?) If the server is idle, how is it slow? Slower is subjective. Unfortunately, I can't currently run the old server to actually measure times, but it's just slow. I can see individual requests (images, CSS, etc) taking a long time to be fulfilled, whereas before, the pages would just come up fully rendered. Time to first render is longer, too. > Also, I'm not sure I understand you mean by "not-running" if there are > Resin threads. Do the two JVMs exist? What do the logs show? I left the server running late last night, tried to make requests of it this morning, but I just got "server not responding" errors in the browser. Mind you, these were not timeouts, but actual connection refusals. logs show nothing with this config: I'm not sure how to verify that the JVMs exist, other than the ps command outputs many lines for java (as described before). It's not that the server died, it just stopped listening. >> Configuration is slightly different than it used to be, a necessity of >> updating the config file(s), but I don't think it's radically different from >> what I had. >> >> ps -axwwu | grep resin on redhat 7.2 (I know, it's old) gives me many lines >> like this: >> > Hmm. If it's showing separate ps lines for each thread, that's very old. > Some of those very old linux versions only partially implemented some of > the system calls we use, like epoll(). (So the configure script would > discover epoll(), but it didn't actually work as advertised.) So, that's an interesting and scary thought. However, I'm not using Resin Professional, so I thought I didn't get native sockes. >> root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 >> /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf >> /var/resin/resin.xml -log-directory /var/logs/resin -root-directory >> /var/resin/root -J-verbosegc console >> >> root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 >> /usr/local/java-versions/jdk1.6.0_07/bin/java -server >> -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 >> -Djava.util.logging.manager=com.caucho.log.LogManagerImpl >> -Djava.system.class.loader=com.caucho.loader.SystemClassLoader >> -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl >> -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m >> -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory >> /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 -log-directory >> /var/logs/resin -root-directory /var/resin/root console >> >> >> I can't find where the -Xss1m and -Xmx256m are being set. According to the >> admin manual, it should default to -Xss2m. >> > You can change that in the resin.xml with a -Xss1m. I used to have exactly those values specified in resin.xml, and thought perhaps that was part of the problem, so I commented them out (to let the JVM and resin pick defaults). I was still seeing those in the command, though, and didn't know where they were coming from. That was something I had not been specifying in my 3.0.23 installation, and so I thought they were causing more problems than they were solving. ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Rick Mann wrote: > My 4.0.5 install is really very slow. It just won't stay running (although ps > shows many resin threads), and it's much, much slower than 3.0.23 was. I'm > actually running fewer webapps currently, and my server is always idle. > How are you measuring the "slower"? (ab? latency? startup? some particular page size?) If the server is idle, how is it slow? Also, I'm not sure I understand you mean by "not-running" if there are Resin threads. Do the two JVMs exist? What do the logs show? > Configuration is slightly different than it used to be, a necessity of > updating the config file(s), but I don't think it's radically different from > what I had. > > ps -axwwu | grep resin on redhat 7.2 (I know, it's old) gives me many lines > like this: > Hmm. If it's showing separate ps lines for each thread, that's very old. Some of those very old linux versions only partially implemented some of the system calls we use, like epoll(). (So the configure script would discover epoll(), but it didn't actually work as advertised.) > root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 > /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf > /var/resin/resin.xml -log-directory /var/logs/resin -root-directory > /var/resin/root -J-verbosegc console > > root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 > /usr/local/java-versions/jdk1.6.0_07/bin/java -server > -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 > -Djava.util.logging.manager=com.caucho.log.LogManagerImpl > -Djava.system.class.loader=com.caucho.loader.SystemClassLoader > -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl > -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m > -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory > /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 -log-directory > /var/logs/resin -root-directory /var/resin/root console > > > I can't find where the -Xss1m and -Xmx256m are being set. According to the > admin manual, it should default to -Xss2m. > You can change that in the resin.xml with a -Xss1m. -- Scott > The verbose GC output seems reasonable. Every few seconds, a line like this. > I'm pretty sure my app(s) aren't leaking, since they've run for months before > under 3.0.23: > > [GC 41277K->35464K(63664K), 0.0226660 secs] > > Sigh. > > On Mar 18, 2010, at 00:47:39, Rick Mann wrote: > > >> Hi. My new insatll of 4.0.5 is running very badly. My server is basically >> unloaded, except for the requests I make, but connections are constantly >> dropped. I was running 3.0.23 without any real problems before this. Also, I >> don't see anything in any of the logs when this happens. >> >> Any suggestions? >> >> Thanks! >> -- >> Rick >> >> >> >> ___ >> resin-interest mailing list >> resin-interest@caucho.com >> http://maillist.caucho.com/mailman/listinfo/resin-interest >> > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest > > ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
My 4.0.5 install is really very slow. It just won't stay running (although ps shows many resin threads), and it's much, much slower than 3.0.23 was. I'm actually running fewer webapps currently, and my server is always idle. Configuration is slightly different than it used to be, a necessity of updating the config file(s), but I don't think it's radically different from what I had. ps -axwwu | grep resin on redhat 7.2 (I know, it's old) gives me many lines like this: root 20511 0.0 5.1 226964 26396 pts/2 S10:54 0:00 /usr/java/bin/java -jar /usr/local/resin/lib/resin.jar -conf /var/resin/resin.xml -log-directory /var/logs/resin -root-directory /var/resin/root -J-verbosegc console root 20517 0.0 6.8 450916 35108 pts/2 S10:54 0:00 /usr/local/java-versions/jdk1.6.0_07/bin/java -server -Djava.awt.headless=true -Dfile.encoding=utf-8 -Dresin.server=1 -Djava.util.logging.manager=com.caucho.log.LogManagerImpl -Djava.system.class.loader=com.caucho.loader.SystemClassLoader -Djavax.management.builder.initial=com.caucho.jmx.MBeanServerBuilderImpl -Djava.awt.headless=true -Dresin.home=/usr/local/resin/ -Xss1m -Xmx256m -verbosegc -verbosegc com.caucho.server.resin.Resin --root-directory /var/resin/root -conf /var/resin/resin.xml -socketwait 34280 -log-directory /var/logs/resin -root-directory /var/resin/root console I can't find where the -Xss1m and -Xmx256m are being set. According to the admin manual, it should default to -Xss2m. The verbose GC output seems reasonable. Every few seconds, a line like this. I'm pretty sure my app(s) aren't leaking, since they've run for months before under 3.0.23: [GC 41277K->35464K(63664K), 0.0226660 secs] Sigh. On Mar 18, 2010, at 00:47:39, Rick Mann wrote: > Hi. My new insatll of 4.0.5 is running very badly. My server is basically > unloaded, except for the requests I make, but connections are constantly > dropped. I was running 3.0.23 without any real problems before this. Also, I > don't see anything in any of the logs when this happens. > > Any suggestions? > > Thanks! > -- > Rick > > > > ___ > resin-interest mailing list > resin-interest@caucho.com > http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] New 4.0.5 install constantly dropping connections on unloaded server
Hi. My new insatll of 4.0.5 is running very badly. My server is basically unloaded, except for the requests I make, but connections are constantly dropped. I was running 3.0.23 without any real problems before this. Also, I don't see anything in any of the logs when this happens. Any suggestions? Thanks! -- Rick ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
