[Resin-interest] Limiting access to caucho-status
Hi, Is there a way to limit access to "/caucho-status" URL either per IP or per user/password? I read that it's suggested to disable it in production servers, but it has good information for monitoring servers. I think it would be nice to just limit accesses. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Apache issue
Hi, I have a server with apache+resin-3.1.6 installed. Once in a while it have some performance issues. I have noted on that times: - theres is one (or two) httpd process consuming 200% (each process) from CPU - after some time (few hours) application gets almost unresponsive (most of request hangs for so long in the client's browser) - apache-only requests (static HTML, images) seems to be processed normally on that time - if I restart resin (java process) on that time the problem persists - if I "killall -1 httpd", problem is solved - I have found no log messages about that As a workaround I have written a script to run a "killall -1 httpd" every time it finds a httpd process consuming 200% from CPU It got the application much more stable, but once in a while (once in week for example) I have the same problem: server gets almost unresponsive. So as a workaround it run a "killall -1 httpd" each hour. It got the server much more stable, but it isn't the ideal behavior. Do you guys have some trouble like that? What do you suggest? The same application in another application server runs without this issue. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Apache issue
Scott, Scott Ferguson escreveu: > I have an open report on that issue, but haven't been able to > reproduce it here under stress testing. Do you have any information > in the logs suggesting something that might trigger this? (e.g. some > connection issue on the Resin side.) > It's really too weird. I have found no information about that in the logs (apache/resin/os), even in the exact moment of the problem. I´ve noted when there is a httpd process consuming 200% from cpu, system load (from OS) gets higher and dynamic pages get slower. Usually we have tens of process like that per day, even in not so overloaded times. But I haven't seen such a process in lighter times. I have monitored httpd process when it occurs to see what's running on that moment, but it seems to be normal requests. I didn't see anything wrong on that process, any suspicious request or URL. I even didn't see any relation like the same process present on every overloaded httpd process. It seems there is lack in communication between apache and resin, and this problem doesn't seem to be related to the 200% httpd process. Even I restarting httpd when it find and 200% httpd process, after some days the system got unresponsive. That's why I schedule an httpd restart once in a hour. I'm sorry for not supply more specific informations about that, but I didn't find it yet. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] cluster with different versions
Hi, Is it possible to have a cluster with different nodes with different version? My plan is to install a higher version in only one node. Test it's stability and, after everything OK and tested (few days), upgrade the other ones. Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Apache issue
David, david day escreveu: I have a server with apache+resin-3.1.6 installed. Are you running caucho_module? Have you used netstat command during troubleshooting? We had our thread-max fairly high and I was seeing mass quantities of connections to the srun port. Shutting down the JVM would clear the connections, but it could take a long time for the count to drop. Bouncing Apache at the same time had me going quickly. Dropping thread-max helped. Thanks for answering. Well, I did. I'm not an expert in Java Application Servers, so it's possible my answer isn't 100% correct. This issue is really too weird for me. Actually, it seem we have more than one issue. Some times if a only reinit apache it solves the problem, other times not. I do need to restart resin, too. Usually (including times when the problem occurs) netstat doesn't shows a mass quantity of connections. Dropping thread-max took the system more instable. Best number for us seems to be 256. I'm sorry I don't have enough informations about this problem. Perhaps because my inexpertness with Java Application Servers. Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Limiting access to caucho-status
David, david day escreveu: Is there a way to limit access to "/caucho-status" URL either per IP or per user/password? Have you checked out directive? This works nicely. SetHandler default-handler Order Deny,Allow Deny from all Allow from 192.168. I did it. I helped, but caucho-status still answers when accessed via hostname:resin's port It helps but it isn't 100%. Thanks again, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] caucho-status
Hi, What exactly mean the numbers inside "Pooled" column in a caucho-status? When it should be 0/1/2? Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] caucho-status
Thank you Scott, Understood, Ronan Scott Ferguson escreveu: On Mar 12, 2009, at 12:06 PM, Ronan Lucio wrote: Hi, What exactly mean the numbers inside "Pooled" column in a caucho- status? When it should be 0/1/2? It's the number of idle connections from mod_caucho to Resin. You'll generally only see 0/1 unless you're using the worker model or running under windows. -- Scott Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Why Apache?
there's more than one reason. Serving different sites/services on the same server, not all can be served by resin for various reasons: These are the typical reasons. If you're running a specific Apache module (like mod_perl) or running several different kinds of backends, or prefer the logging, or mod_rewrite works better for you than Resin's rewrite-dispatch, then Apache can be a good choice. I didn't hear anybody talking about mod_evasive. How do you care about DoS attacks? Of course mod_evasive isn't a solution for that, but it helps a little. Helps to identify a suspicious attack when you see one or more IPs sending a huge amount of requests. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Perf Issues
Hi, We have had a perf issue. Our servers have 4Gb RAM. It has just Resin and Apache installed. The problem is, when I start Resin, the whole server uses about 2.5 Gb RAM After that memory usage keep growing til it reachs the 4Gb RAM, use swap and so on. After few hours the application start getting slow. Analyzing the server sources, it's using so low CPU, load about 1... I see none overload evidence, except for RAM memory. So I just restart Resin and/or Apache and application gets fast again, but few ours later it will raise the same issue. This problem usually happens on peaks hours. So we upgraded RAM memory to 8Gb with a PAE kernel. Although it doesn't reach the 8G RAM, slow moments gots for frequent. It seems to work worse that way (8Gb + PAE kernel). The question is: Is there everyone having the same issue with Resin (3.1.6)? My doubt if such problem resides either on Resin or on my application. Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Perf Issues
Scott, Scott Ferguson escreveu: > The /resin-admin has a heap dump which will give you a general idea of > what's taking the memory (you will need to add a -agentlib:resin to > enable it. > > That heap information is very important because it will very quickly > let you focus on the real issue. Otherwise, it's almost impossible to > guess what the underlying problem is. > Thank you for the answer, I'll do that. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Perf Issues
Hi Bill, I have already done it some weeks ago, but it doesn't seem the problem. Thanks, Ronan Bill Au escreveu: I would also enable garbage collection logging (-Xloggc:) to see if your application is running slow because the JVM is spending a lot of time doing garbage collection. If you are running SUN Java 6, you can use the jmap command to do a heap dump also. Jmap is available in Java 5 but it doesn't always produce a consistent heap dump. Bill On Fri, Mar 20, 2009 at 1:12 PM, Ronan Lucio <mailto:lis...@tiper.com.br>> wrote: Scott, Scott Ferguson escreveu: > The /resin-admin has a heap dump which will give you a general idea of > what's taking the memory (you will need to add a -agentlib:resin to > enable it. > > That heap information is very important because it will very quickly > let you focus on the real issue. Otherwise, it's almost impossible to > guess what the underlying problem is. > Thank you for the answer, I'll do that. Ronan ___ resin-interest mailing list resin-interest@caucho.com <mailto:resin-interest@caucho.com> http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Perf Issues
Stargazer, Stargazer escreveu: I have been suffering from exactly those symptoms for years. Do your httpd processes consume all the swap, with top showing some at 450Mb? (default httpd.conf values) I.e does restarting apache alone, and not resin, cause the swap to drop back down to normal until the next slow growth starts it all again? Yes, it does. Exactly the same behavior reported by you... :-/ Most of the times restarting only the httpd process back it to normal. Other times I need to restart resin, too. That is my pattern. So when I profile resin theres no unusual growth even through the whole server is effectiviley dead until restart. Given that, whos to blame? Could mod_caucho somehow be at fault even though resin itself is ok? Same thoughts. Thank you, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Perf Issues
Scott, Scott Ferguson escreveu: The /resin-admin has a heap dump which will give you a general idea of what's taking the memory (you will need to add a -agentlib:resin to enable it. That heap information is very important because it will very quickly let you focus on the real issue. Otherwise, it's almost impossible to guess what the underlying problem is. -- Scott Do you if it's normal? rankself+desc selfdesccount class 1. 121.487M121.487M0.000M 943239 char[] 0.8121 98.662M 98.662M 0.000M 78702 byte[] Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Perf Issues
Scott, Scott Ferguson escreveu: It depends on the caching in your system. What do the next few entries show? The difficulty with char[] and byte[] by themselves is that they're generally held by other objects like String or a cache. If you have a big cache, then that size could be perfectly normal. If they're held by non-caching objects, then it's likely a leak. So it seems to be cache: rankself+desc selfdesccount class 1. 123.367M123.367M0.000M 965651 char[] 0.8045 99.249M 99.249M 0.000M 78774 byte[] 0.7537 92.977M 20.117M 72.860M 878934 java.lang.String 0.5117 63.124M 0.046M 63.078M 1009 com.caucho.db.store.ReadBlock 0.3217 39.690M 6.001M 33.688M 65552 com.caucho.server.dispatch.Invocation 0.2474 30.527M 7.157M 23.370M 91701 java.lang.Object[] 0.2155 26.582M 3.574M 23.009M 156144 com.caucho.util.CharBuffer 0.1693 20.885M 0.566M 20.318M 14849 java.util.HashMap 0.1569 19.353M 3.071M 16.283M 134164 java.util.HashMap$Entry 0.1110 13.697M 0.014M 13.684M 253 railo.runtime.writer.CFMLWriterImpl 0.1110 13.688M 0.005M 13.683M 310 java.lang.StringBuffer I got this dump in a slow moment, but that slow moment seemed to be an apache issue because in that moment it was too slow to open images, too. Any way, thank you for the help. I'll try to get more data. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] watchdog stop authentication failure
Hi, Does anybody knows what can cause this error? --- [2009/05/06 16:18:16.552] watchdog stop authentication failure [2009/05/06 16:18:16.552] com.caucho.config.ConfigException: watchdog stop forbidden - authentication failed [2009/05/06 16:18:16.552] at com.caucho.boot.WatchdogServlet.stop(WatchdogServlet.java:92) [2009/05/06 16:18:16.552] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [2009/05/06 16:18:16.552] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [2009/05/06 16:18:16.552] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [2009/05/06 16:18:16.552] at java.lang.reflect.Method.invoke(Method.java:597) [2009/05/06 16:18:16.552] at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180) [2009/05/06 16:18:16.552] at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109) [2009/05/06 16:18:16.552] at com.caucho.hessian.server.HessianServlet.service(HessianServlet.java:396) [2009/05/06 16:18:16.552] at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103) [2009/05/06 16:18:16.552] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187) [2009/05/06 16:18:16.552] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266) [2009/05/06 16:18:16.552] at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:435) [2009/05/06 16:18:16.552] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:678) [2009/05/06 16:18:16.552] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721) [2009/05/06 16:18:16.552] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643) [2009/05/06 16:18:16.552] at java.lang.Thread.run(Thread.java:619) --- I'm running Resin-Pro 3.1.6. No lucky researching about that in google. Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] watchdog stop authentication failure
Hi Steffen, Thank you very much, I think it's my case, too. After changing resin.conf I executed a restart. It probably doesn't. Thank you, Ronan Steffen Busch escreveu: As far as I remember, I have experienced this once when changing the resin.conf and modifying (or adding) a to while Resin was started. ... ... The change in resin.conf caused Resin to restart but trying to stop it was unsuccessful - similar to your exception. You could try to kill the watchdog and resin processes and start both again afterwards - then 'stop' should be working again. Best Regards, Steffen 2009/5/6 Ronan Lucio mailto:lis...@tiper.com.br>> Hi, Does anybody knows what can cause this error? --- [2009/05/06 16:18:16.552] watchdog stop authentication failure [2009/05/06 16:18:16.552] com.caucho.config.ConfigException: watchdog stop forbidden - authentication failed [2009/05/06 16:18:16.552] at com.caucho.boot.WatchdogServlet.stop(WatchdogServlet.java:92) [2009/05/06 16:18:16.552] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [2009/05/06 16:18:16.552] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [2009/05/06 16:18:16.552] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [2009/05/06 16:18:16.552] at java.lang.reflect.Method.invoke(Method.java:597) [2009/05/06 16:18:16.552] at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:180) [2009/05/06 16:18:16.552] at com.caucho.hessian.server.HessianSkeleton.invoke(HessianSkeleton.java:109) [2009/05/06 16:18:16.552] at com.caucho.hessian.server.HessianServlet.service(HessianServlet.java:396) [2009/05/06 16:18:16.552] at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103) [2009/05/06 16:18:16.552] at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:187) [2009/05/06 16:18:16.552] at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266) [2009/05/06 16:18:16.552] at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:435) [2009/05/06 16:18:16.552] at com.caucho.server.port.TcpConnection.run(TcpConnection.java:678) [2009/05/06 16:18:16.552] at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:721) [2009/05/06 16:18:16.552] at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:643) [2009/05/06 16:18:16.552] at java.lang.Thread.run(Thread.java:619) --- I'm running Resin-Pro 3.1.6. No lucky researching about that in google. Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com <mailto:resin-interest@caucho.com> http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] lighttpd and nginx
Hi, Does Resin works with lighttpd or nginx? Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] lighttpd and nginx
Hi Steffen, For all I have understood it refers to Resin's-4.0, which is still beta. Anyway to use it with 3.1? Thanks, Ronan Steffen Busch escreveu: I don't know about lighttpd, but there was a blog post recently regarding nginx, check this: http://blog.caucho.com/?p=173 Regards, Steffen 2009/6/16 Ronan Lucio mailto:lis...@tiper.com.br>> Hi, Does Resin works with lighttpd or nginx? Thanks, Ronan ___ resin-interest mailing list resin-interest@caucho.com <mailto:resin-interest@caucho.com> http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] lighttpd and nginx
Hi Scott, Scott Ferguson escreveu: > If you're looking at nginx for performance, you should benchmark Resin > as a load-balancer as well, especially if you're using proxy caching. > For 4.0, the performance numbers were fairly close (nginx slightly > faster). Do you think Resin-4.0 is reliable for production? For all I have understood it seems it wont be release soon due so JEE-1.6 specifications. Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] lighttpd and nginx
Understood Scott, thank you, Ronan Scott Ferguson escreveu: On Jun 17, 2009, at 6:14 AM, Ronan Lucio wrote: Hi Scott, Scott Ferguson escreveu: If you're looking at nginx for performance, you should benchmark Resin as a load-balancer as well, especially if you're using proxy caching. For 4.0, the performance numbers were fairly close (nginx slightly faster). Do you think Resin-4.0 is reliable for production? Not yet. We did add the FastCGI specifically for nginx. For 3.1, you'd need to use the nginx http proxy. It should be pretty close to the fastcgi efficiency. For all I have understood it seems it wont be release soon due so JEE-1.6 specifications. That's correct. For example, the JSR-299 (Java Injection) spec has changed its internal SPI pretty radically, causing a major refactor on our part. -- Scott Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
[Resin-interest] Metrics
Hi, What metrics does Resin offer to measure system health? I'd like to store some data about system info for measurement and capacity planning. The problem is that the server with resin doesn't seem to show a real information. I used to see CPU as 3%, load as 0,8. How can I found out the possible bottlenecks in this server? Is there a way to get know many concurrent requests are running at a given time? PS: I know that we discover the requests ceiling with stress test tools, therefore I still like to measure system health. With actual system info it doesn't seem to correspond to the reality. Thank you, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
Re: [Resin-interest] Metrics
Hi Alex, Perfect. Thank you very much, Ronan Alex escreveu: Hi Ronan, Luckily Emil has recently published a blog entry right on the topic: http://blog.caucho.com/?p=219 --Alex On 20-Aug-09, at 1:41 PM, Ronan Lucio wrote: Hi, What metrics does Resin offer to measure system health? I'd like to store some data about system info for measurement and capacity planning. The problem is that the server with resin doesn't seem to show a real information. I used to see CPU as 3%, load as 0,8. How can I found out the possible bottlenecks in this server? Is there a way to get know many concurrent requests are running at a given time? PS: I know that we discover the requests ceiling with stress test tools, therefore I still like to measure system health. With actual system info it doesn't seem to correspond to the reality. Thank you, Ronan ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest ___ resin-interest mailing list resin-interest@caucho.com http://maillist.caucho.com/mailman/listinfo/resin-interest
