Is it possible to use this http://www.mail-archive.com/wicket-u...@lists.sourceforge.net/msg20951.html workaround in the URLResourceStream constructor - similar to how it is done in URLResourceStream#lastModifiedTime()?
adambender wrote: > > For what it is worth I saw a file being created in the URLResourceStream > constructor at line 88 but it doesn't look like this is the source of any > problems, its just a way to figure out how to check the lastModified time > later on. > > While I can't claim any kind of expertise on the inner workings of Wicket > it seems that this problem would be solved if it wasn't required to check > the last modification time on those resources which are embedded in the > Wicket jar. Of course I have no idea what else this would affect so it may > not be so simple. It may also be possible to just cache the stream or the > actual .js string for use by later requests so that it is not necessary to > try and reload files like this from the jar every time they are requested. > > For our use what we are likely going to do is pull the .js files out and > place them where httpd can serve them and then use some url rewriting > magic to prevent Wicket from serving them. > > > Adam > > > igor.vaynberg wrote: >> >> i just dont see what we can do about this... we are calling >> connection.getLastModified(), we are not creating a file, etc... >> >> -igor >> >> On Wed, Oct 21, 2009 at 10:18 AM, adambender <adamben...@gmail.com> >> wrote: >>> >>> This issue looks like it is directly related to >>> http://issues.apache.org/jira/browse/WICKET-438 and the access of the >>> lastModified header. Every time a URLResourceStream is constructed the >>> lastModified time is requested at line 85 and the number of file handles >>> goes up by one. The solution to the jira issue indicated that upgrading >>> the >>> version of linux fixed the problem but it doesnt seem to be the case >>> with OS >>> X or Red Hat Enterprise Linux... >>> >>> >>> adambender wrote: >>>> >>>> I was able to distill the problem down to what I think is the core >>>> issue >>>> here. >>>> >>>> When an AJAX Wicket page is rendered it contains a reference to two .js >>>> files as Igor had mentioned, the Web URLs of these files look like >>>> this: >>>> >>>> <App Context >>>> Root>/resources/org.apache.wicket.markup.html.WicketEventReference/wicket-event.js >>>> <App Context >>>> Root>/resources/org.apache.wicket.markup.html.WicketEventReference/wicket-ajax.js >>>> >>>> These requests are of course fielded by the Wicket filter which ends up >>>> trying to load the resources using the UrlResourceStream. In order to >>>> actually load these files Wicket gets the JAR URL for each file, >>>> converts >>>> them to a URIs and then passes them to the File constructor. The >>>> problem >>>> is that the URI that is created is opaque - >>>> http://www.docjar.com/docs/api/java/net/URI.html see here for a good >>>> explanation in the case of wicket-event.js the URI string is like the >>>> following : >>>> >>>> jar:file:/<absolute-path-tojar>/wicket-1.4.1.jar!/org/apache/wicket/markup/html/wicket-event.js >>>> >>>> It is similar for the case of wicket-ajax.js >>>> >>>> A quick glance at the File constructor shows that an opaque URL cannot >>>> be >>>> used to create a File as it is considered non-hierarchical. This of >>>> course >>>> leaves you with lots of requests to load this file, none of which can >>>> be >>>> completed and results in a lot of open URL connections which can never >>>> be >>>> closed. >>>> >>>> I guess my question is, should this loading mechanism be working >>>> because >>>> it doesnt seem like it could the way it is currently written? Is it >>>> possible to configure how Wicket finds these files embedded in the jar? >>>> Have I missed something? >>>> >>>> >>>> Adam Bender-2 wrote: >>>>> >>>>> Thanks for the explanation I think that helps shed some light... The >>>>> tests are actually JMeter tests driving load by emulating a web >>>>> browser - the application the are testing is running in Deployment >>>>> mode set up as though it were a production server. After a little more >>>>> digging I have been able to determine that is only a problem on pages >>>>> which use Ajax - and it looks like there is a related debug message >>>>> coming from an exception handler regarding the wicket-event.js file >>>>> not being accessible (URI is not hierarchical). This would actually >>>>> explain why the problem only manifests when the JMeter tests are set >>>>> to request embedded assets as wicket ajax pages embed requests for >>>>> additional pieces of javascript - which in the case of wicket-event.js >>>>> are causing exceptions to be thrown and the JVM bug you mentioned is >>>>> probably preventing these resources from being cleaned up properly. >>>>> >>>>> Does that sound right? >>>>> >>>>> Adam >>>>> >>>>> >>>>> On Oct 20, 2009, at 8:41 PM, Igor Vaynberg wrote: >>>>> >>>>>> when you are requesting an embedded resource wicket needs to stream >>>>>> the file out of a jar, the way the jvm handles that is by creating a >>>>>> jar url connection object that streams the file. unfortunately, there >>>>>> is a bug in the jdk where the url connection does not have a close >>>>>> method and so wicket or any other java app cannot release the >>>>>> connection. this is addressed, afair, in jdk 7. >>>>>> >>>>>> i have many apps deployed in production and have not managed yet to >>>>>> run out of the handles with the limit set about 4K. not sure why this >>>>>> is different in your case. you mentioned "tests", are those unit >>>>>> tests >>>>>> and is wicket there running in deployment mode? >>>>>> >>>>>> the resource watcher, which should be disabled, will cause the >>>>>> handles >>>>>> to run out because it continuously monitors markup files for changes >>>>>> (hot reloading in dev mode) and everytime it checks a markup file in >>>>>> a >>>>>> jar it creates the url connection and leaks it. by default it is >>>>>> disabled in deployment mode but if you manually set the poll >>>>>> frequency >>>>>> in your settings it will be reenabled. >>>>>> >>>>>> -igor >>>>>> >>>>>> >>>>>> On Tue, Oct 20, 2009 at 4:07 PM, Adam Bender <a...@magpieti.com> >>>>>> wrote: >>>>>>> We have run with a limit as high as 10,000 files and our tests can >>>>>>> bring it >>>>>>> to the limit in 20 minutes, but that still doesn't explain why so >>>>>>> many >>>>>>> copies of the jar are needed - and only when we are also requesting >>>>>>> embedded >>>>>>> assets... >>>>>>> >>>>>>> On Oct 20, 2009, at 5:04 PM, Major Péter wrote: >>>>>>> >>>>>>>> resolution -> solution... >>>>>>>> Just set the ulimit in the initscript and run it with start-stop- >>>>>>>> daemon, >>>>>>>> it will make the problem disappear (but still there would be many >>>>>>>> open >>>>>>>> files...) >>>>>>>> >>>>>>>> Peter >>>>>>>> >>>>>>>> 2009-10-21 00:38 keltezéssel, Major Péter írta: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> we also had this issue, because (I think) we are running two >>>>>>>>> different >>>>>>>>> wicket application in only one glassfish domain. Our resolution >>>>>>>>> was, >>>>>>>>> that we are running now the glassfish domains with custom init >>>>>>>>> scripts >>>>>>>>> with ulimit settings. Maybe this will help for you. >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> Peter >>>>>>>>> >>>>>>>>> 2009-10-21 00:14 keltezéssel, Martin Grigorov írta: >>>>>>>>>> >>>>>>>>>> Hi Adam, >>>>>>>>>> >>>>>>>>>> You may try to debug what is the problem with >>>>>>>>>> https://wiki.sdn.sap.com/wiki/display/Java/JPicus >>>>>>>>>> >>>>>>>>>> El mar, 20-10-2009 a las 15:39 -0600, Adam Bender escribió: >>>>>>>>>>> >>>>>>>>>>> Greetings all, >>>>>>>>>>> >>>>>>>>>>> Recently I have been performance testing a Wicket application >>>>>>>>>>> (1.4.1) >>>>>>>>>>> and I >>>>>>>>>>> am running into the dreaded Too Many Files Open issue. I have >>>>>>>>>>> searched >>>>>>>>>>> the >>>>>>>>>>> mailing lists and most of the trouble around this issue seems >>>>>>>>>>> to come >>>>>>>>>>> from >>>>>>>>>>> being in DEVELOPMENT mode so I made sure we were in DEPLOYMENT >>>>>>>>>>> mode. >>>>>>>>>>> When I >>>>>>>>>>> run 'lsof -p xxxx | grep wicket-1.4.1.jar' I see over 1000 >>>>>>>>>>> entries and >>>>>>>>>>> it's >>>>>>>>>>> growing monotonically. This seems really unusual - why would >>>>>>>>>>> wicket >>>>>>>>>>> need >>>>>>>>>>> 1000 copies of this jar open? An additional bit of weirdness >>>>>>>>>>> came up >>>>>>>>>>> when >>>>>>>>>>> our load tests stopped loading the embedded assets (css, js and >>>>>>>>>>> images) >>>>>>>>>>> in >>>>>>>>>>> each page - this seemed to cap the number of lsof entries at >>>>>>>>>>> 5... This >>>>>>>>>>> is >>>>>>>>>>> even weirder because our app serves these static items from >>>>>>>>>>> httpd >>>>>>>>>>> without >>>>>>>>>>> tomcat ever knowing about them. How could our loading of >>>>>>>>>>> embedded items >>>>>>>>>>> really affect the number of file handles wicket needs? >>>>>>>>>>> >>>>>>>>>>> For completeness we are running this app on Red Hat Enterprise >>>>>>>>>>> Linx 5 >>>>>>>>>>> with >>>>>>>>>>> Tomcat 6.0.20 and Java 1.6 >>>>>>>>>>> >>>>>>>>>>> Adam >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >>>>>>>> For additional commands, e-mail: users-h...@wicket.apache.org >>>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >>>>>>> For additional commands, e-mail: users-h...@wicket.apache.org >>>>>>> >>>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >>>>>> For additional commands, e-mail: users-h...@wicket.apache.org >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >>>>> For additional commands, e-mail: users-h...@wicket.apache.org >>>>> >>>>> >>>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Too-Many-Files-Wicket-1.4.1-tp25983047p25996646.html >>> Sent from the Wicket - User mailing list archive at Nabble.com. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >>> For additional commands, e-mail: users-h...@wicket.apache.org >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org >> For additional commands, e-mail: users-h...@wicket.apache.org >> >> >> > > -- View this message in context: http://www.nabble.com/Too-Many-Files-Wicket-1.4.1-tp25983047p26001171.html Sent from the Wicket - User mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@wicket.apache.org For additional commands, e-mail: users-h...@wicket.apache.org
