i just dont see what we can do about this... we are calling connection.getLastModified(), we are not creating a file, etc...
-igor On Wed, Oct 21, 2009 at 10:18 AM, adambender <[email protected]> wrote: > > This issue looks like it is directly related to > http://issues.apache.org/jira/browse/WICKET-438 and the access of the > lastModified header. Every time a URLResourceStream is constructed the > lastModified time is requested at line 85 and the number of file handles > goes up by one. The solution to the jira issue indicated that upgrading the > version of linux fixed the problem but it doesnt seem to be the case with OS > X or Red Hat Enterprise Linux... > > > adambender wrote: >> >> I was able to distill the problem down to what I think is the core issue >> here. >> >> When an AJAX Wicket page is rendered it contains a reference to two .js >> files as Igor had mentioned, the Web URLs of these files look like this: >> >> <App Context >> Root>/resources/org.apache.wicket.markup.html.WicketEventReference/wicket-event.js >> <App Context >> Root>/resources/org.apache.wicket.markup.html.WicketEventReference/wicket-ajax.js >> >> These requests are of course fielded by the Wicket filter which ends up >> trying to load the resources using the UrlResourceStream. In order to >> actually load these files Wicket gets the JAR URL for each file, converts >> them to a URIs and then passes them to the File constructor. The problem >> is that the URI that is created is opaque - >> http://www.docjar.com/docs/api/java/net/URI.html see here for a good >> explanation in the case of wicket-event.js the URI string is like the >> following : >> >> jar:file:/<absolute-path-tojar>/wicket-1.4.1.jar!/org/apache/wicket/markup/html/wicket-event.js >> >> It is similar for the case of wicket-ajax.js >> >> A quick glance at the File constructor shows that an opaque URL cannot be >> used to create a File as it is considered non-hierarchical. This of course >> leaves you with lots of requests to load this file, none of which can be >> completed and results in a lot of open URL connections which can never be >> closed. >> >> I guess my question is, should this loading mechanism be working because >> it doesnt seem like it could the way it is currently written? Is it >> possible to configure how Wicket finds these files embedded in the jar? >> Have I missed something? >> >> >> Adam Bender-2 wrote: >>> >>> Thanks for the explanation I think that helps shed some light... The >>> tests are actually JMeter tests driving load by emulating a web >>> browser - the application the are testing is running in Deployment >>> mode set up as though it were a production server. After a little more >>> digging I have been able to determine that is only a problem on pages >>> which use Ajax - and it looks like there is a related debug message >>> coming from an exception handler regarding the wicket-event.js file >>> not being accessible (URI is not hierarchical). This would actually >>> explain why the problem only manifests when the JMeter tests are set >>> to request embedded assets as wicket ajax pages embed requests for >>> additional pieces of javascript - which in the case of wicket-event.js >>> are causing exceptions to be thrown and the JVM bug you mentioned is >>> probably preventing these resources from being cleaned up properly. >>> >>> Does that sound right? >>> >>> Adam >>> >>> >>> On Oct 20, 2009, at 8:41 PM, Igor Vaynberg wrote: >>> >>>> when you are requesting an embedded resource wicket needs to stream >>>> the file out of a jar, the way the jvm handles that is by creating a >>>> jar url connection object that streams the file. unfortunately, there >>>> is a bug in the jdk where the url connection does not have a close >>>> method and so wicket or any other java app cannot release the >>>> connection. this is addressed, afair, in jdk 7. >>>> >>>> i have many apps deployed in production and have not managed yet to >>>> run out of the handles with the limit set about 4K. not sure why this >>>> is different in your case. you mentioned "tests", are those unit tests >>>> and is wicket there running in deployment mode? >>>> >>>> the resource watcher, which should be disabled, will cause the handles >>>> to run out because it continuously monitors markup files for changes >>>> (hot reloading in dev mode) and everytime it checks a markup file in a >>>> jar it creates the url connection and leaks it. by default it is >>>> disabled in deployment mode but if you manually set the poll frequency >>>> in your settings it will be reenabled. >>>> >>>> -igor >>>> >>>> >>>> On Tue, Oct 20, 2009 at 4:07 PM, Adam Bender <[email protected]> >>>> wrote: >>>>> We have run with a limit as high as 10,000 files and our tests can >>>>> bring it >>>>> to the limit in 20 minutes, but that still doesn't explain why so >>>>> many >>>>> copies of the jar are needed - and only when we are also requesting >>>>> embedded >>>>> assets... >>>>> >>>>> On Oct 20, 2009, at 5:04 PM, Major Péter wrote: >>>>> >>>>>> resolution -> solution... >>>>>> Just set the ulimit in the initscript and run it with start-stop- >>>>>> daemon, >>>>>> it will make the problem disappear (but still there would be many >>>>>> open >>>>>> files...) >>>>>> >>>>>> Peter >>>>>> >>>>>> 2009-10-21 00:38 keltezéssel, Major Péter írta: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> we also had this issue, because (I think) we are running two >>>>>>> different >>>>>>> wicket application in only one glassfish domain. Our resolution >>>>>>> was, >>>>>>> that we are running now the glassfish domains with custom init >>>>>>> scripts >>>>>>> with ulimit settings. Maybe this will help for you. >>>>>>> >>>>>>> Best Regards, >>>>>>> Peter >>>>>>> >>>>>>> 2009-10-21 00:14 keltezéssel, Martin Grigorov írta: >>>>>>>> >>>>>>>> Hi Adam, >>>>>>>> >>>>>>>> You may try to debug what is the problem with >>>>>>>> https://wiki.sdn.sap.com/wiki/display/Java/JPicus >>>>>>>> >>>>>>>> El mar, 20-10-2009 a las 15:39 -0600, Adam Bender escribió: >>>>>>>>> >>>>>>>>> Greetings all, >>>>>>>>> >>>>>>>>> Recently I have been performance testing a Wicket application >>>>>>>>> (1.4.1) >>>>>>>>> and I >>>>>>>>> am running into the dreaded Too Many Files Open issue. I have >>>>>>>>> searched >>>>>>>>> the >>>>>>>>> mailing lists and most of the trouble around this issue seems >>>>>>>>> to come >>>>>>>>> from >>>>>>>>> being in DEVELOPMENT mode so I made sure we were in DEPLOYMENT >>>>>>>>> mode. >>>>>>>>> When I >>>>>>>>> run 'lsof -p xxxx | grep wicket-1.4.1.jar' I see over 1000 >>>>>>>>> entries and >>>>>>>>> it's >>>>>>>>> growing monotonically. This seems really unusual - why would >>>>>>>>> wicket >>>>>>>>> need >>>>>>>>> 1000 copies of this jar open? An additional bit of weirdness >>>>>>>>> came up >>>>>>>>> when >>>>>>>>> our load tests stopped loading the embedded assets (css, js and >>>>>>>>> images) >>>>>>>>> in >>>>>>>>> each page - this seemed to cap the number of lsof entries at >>>>>>>>> 5... This >>>>>>>>> is >>>>>>>>> even weirder because our app serves these static items from httpd >>>>>>>>> without >>>>>>>>> tomcat ever knowing about them. How could our loading of >>>>>>>>> embedded items >>>>>>>>> really affect the number of file handles wicket needs? >>>>>>>>> >>>>>>>>> For completeness we are running this app on Red Hat Enterprise >>>>>>>>> Linx 5 >>>>>>>>> with >>>>>>>>> Tomcat 6.0.20 and Java 1.6 >>>>>>>>> >>>>>>>>> Adam >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> >> >> > > -- > View this message in context: > http://www.nabble.com/Too-Many-Files-Wicket-1.4.1-tp25983047p25996646.html > Sent from the Wicket - User mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
