the os should be able to handle at least a few thousand open file handles.
furthermore you can configure it to have more if you need to. only one
handle should be open per resource (no way to avoid that) in a jar, so a
spike of user activity shouldnt affect this at all as resources cache
globally.
but yes, unpacking should eliminate these problems. i wish there was an
option in tomcat to unpack the jars on deployment, it can already do this
for the war.
-igor
On 2/1/07, beboris <[EMAIL PROTECTED]> wrote:
It's hard for me to say, what the right way to fix this situation is. The
number of open files (that look to the OS as wicket .jar files) will go up
significantly and then go down (down - probably due to the garbage
collection). It is during those spikes caused by many users hitting wicket
pages around the same time that we got "too many open files" error. It
almost looks like the only way to really eliminate those unnecessary open
files to numerous Wicket resources (and to guarantee "too many open files"
will never hit us) is to unpack those resources from the wicket .jar-s and
put them into WEB-INF/classes directory instead. Once we did it, we didn't
see all those dozens (and even hundreds) open files ever again...
Bob.
Eelco Hillenius wrote:
>
> Too bad,
>
> Eelco
>
>
> On 2/1/07, Johan Compagner <[EMAIL PROTECTED]> wrote:
>> why would you cache? and which one? the url connection to an entry in a
>> jar
>> file (thats the JarUrlConnection)
>> or (i guess) the FileUrlConnection (to the jar file itself)
>>
>> both don't make much sense to cache
>> the first one we don't need to cache we only need to use it once by
>> really
>> loading the resource
>> and i guess when it is finalized it is cleaned up.
>> We already don't use it anymore for the last modified. Because there we
>> use
>> only the second one
>> So the fileUrlConnection to the jarFile itself thats is inside the
>> JarUrlConnection object.
>> on that one we call last modified everytime, But that will not cause
the
>> file to open. (because it doesn't have to read the file itself)
>>
>> And we can't construct JarUrlConnections (for reading the jar entries)
>> with
>> the same file url connection because there is
>> no way to initialize the jar url connection directly with the file url
>> connection so they all would use the same.
>>
>> johan
>>
>>
>>
>> On 2/2/07, Eelco Hillenius <[EMAIL PROTECTED]> wrote:
>> >
>> > Would it be possible and useful to cache the URL connection? Does it
>> > have a time out and/ or does it use an exclusive lock?
>> >
>> > Eelco
>> >
>> > On 2/1/07, Johan Compagner <[EMAIL PROTECTED] > wrote:
>> > > that is what you would think... But why generates a modification
>> check
>> one
>> > > file handle for every check in the file?
>> > >
>> > > because UrlConnection.connect() has again a JarUrlConnection
>> internally
>> that
>> > > makes a new connection to that jar file
>> > > and UrlConnection does have a connect() but not a disconnect() so
you
>> can't
>> > > clear it.
>> > >
>> > > johan
>> > >
>> > >
>> > >
>> > > On 2/2/07, Eelco Hillenius < [EMAIL PROTECTED]> wrote:
>> > > > Yeah, but that would be always one fd for a jar, no matter how
many
>> > > > files in it that have to be read, right?
>> > > >
>> > > > Eelco
>> > > >
>> > > >
>> > > > On 2/1/07, Johan Compagner <[EMAIL PROTECTED]> wrote:
>> > > > > yes the modification checker.
>> > > > > But we do need to really load the resource out of the jar file
>> once.
>> So
>> > > that
>> > > > > file handle will be used.
>> > > > >
>> > > > > johan
>> > > > >
>> > > > >
>> > > > >
>> > > > > On 2/1/07, Eelco Hillenius <[EMAIL PROTECTED]> wrote:
>> > > > > >
>> > > > > > Wasn't it optimized now so that it only hits jars once and
only
>> hits
>> > > > > > per resource where normal files are involved?
>> > > > > >
>> > > > > > Eelco
>> > > > > >
>> > > > > > On 2/1/07, Johan Compagner < [EMAIL PROTECTED] > wrote:
>> > > > > > > one per resource will i think not really change.
>> > > > > > >
>> > > > > > >
>> > > > > > > johan
>> > > > > > >
>> > > > > > >
>> > > > > > > On 2/1/07, beboris < [EMAIL PROTECTED] > wrote:
>> > > > > > > >
>> > > > > > > > We will, when we are on production. Now that we are are
>> still
>> in
>> > > alpha
>> > > > > we
>> > > > > > > > prefer "development" (hey, we wrote our first line of
>> wicket
>> code
>> > > 5-6
>> > > > > > > weeks
>> > > > > > > > ago!)
>> > > > > > > >
>> > > > > > > > Also, even in deployment mode 'lsof' still shows us a lot
>> of
>> open
>> > > > > files
>> > > > > > > (one
>> > > > > > > > per resource) if we don't unpack resources from the .jar
.
>> It
>> may
>> > > be
>> > > > > > > smaller
>> > > > > > > > number than in development mode, but still... I imagine
>> your
>> > > > > "workaround"
>> > > > > > > > would close those unnecessarily open files.
>> > > > > > > >
>> > > > > > > > If you can't backport it, please, tell me where it is in
>> SVN.
>> > > We'll
>> > > > > "hack"
>> > > > > > > > it in oursleves for now...
>> > > > > > > >
>> > > > > > > > Bob
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Johan Compagner wrote:
>> > > > > > > > >
>> > > > > > > > > can't you run in deployment mode instead of development
>> mode?
>> > > > > > > > > then that shouldn't happen.
>> > > > > > > > >
>> > > > > > > > > I can see if we can backport it.
>> > > > > > > > >
>> > > > > > > > > johan
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On 2/1/07, beboris < [EMAIL PROTECTED]> wrote:
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> We have found a pretty weird situation with "too many
>> open
>> > > files"
>> > > > > error
>> > > > > > > > >> on
>> > > > > > > > >> our alpha-testing site. Further analysis showed that
on
>> each
>> > > page
>> > > > > > > refresh
>> > > > > > > > >> the following resources get repeatedly obtained from
the
>> > > wicket's
>> > > > > .jar
>> > > > > > > > >> and
>> > > > > > > > >> add to the number of open files:
>> > > > > > > > >> 'wicket/ajax/wicket-ajax.js'
>> > > > > > > > >>
>> > > > > > >
>> > > > >
>> > >
>> 'wicket/extensions/ajax/markup/html/modal/res/modal.css'
>> > > > > > > > >>
>> > > > > > >
>> > > 'wicket/extensions/ajax/markup/html/modal/res/modal.js'
>> > > > > > > > >>
>> > > > > 'wicket/extensions/markup/html/tree/res/tree.css'
>> > > > > > > > >>
>> > > > > 'wicket/extensions/markup/html/tree/res/tree.js'
>> > > > > > > > >>
>> > > > > > > > >> Eventually (due to garbage collection?) the number of
>> open
>> > > files
>> > > > > goes
>> > > > > > > > >> down
>> > > > > > > > >> again. But we wanted to know why those files stayed
open
>> in
>> the
>> > > > > first
>> > > > > > > > >> place
>> > > > > > > > >> and were not closed upon retrieving a resource.
>> > > > > > > > >>
>> > > > > > > > >> It looks like switching to the "deployment" mode from
>> > > "development"
>> > > > > one
>> > > > > > > > >> would significantly reduce the peak numbers of the
open
>> > > > > files/streams
>> > > > > > > to
>> > > > > > > > >> wicket .jar-s, and extracting resources from the .jar
>> would
>> > > reduce
>> > > > > it
>> > > > > > > > >> even
>> > > > > > > > >> better. We were told that the original reason for
files
>> staying
>> > > > > open is
>> > > > > > > a
>> > > > > > > > >> Java bug (the fact that URLConnection doesnt have a
>> .close),
>> > > which
>> > > > > > > causes
>> > > > > > > > >> those nasty results when combined with development
mode
>> trying
>> > > to
>> > > > > > > monitor
>> > > > > > > > >> [resource] files for changes and reloading them.
>> > > > > > > > >>
>> > > > > > > > >> We were also told that there is a workaround for that
>> problem
>> > > in
>> > > > > SVN
>> > > > > > > > >> somewhere, but it's probably not backported to 1.2.5 .
I
>> have 2
>> > > > > > > questions
>> > > > > > > > >> in
>> > > > > > > > >> that regard:
>> > > > > > > > >> (1) Where can we find those workarounds in the code?
>> > > > > > > > >> (2) Is it too much work to backport them to 1.2.5 so
>> when
>> it's
>> > > > > released
>> > > > > > > > >> it
>> > > > > > > > >> doesn't contain the problem?
>> > > > > > > > >>
>> > > > > > > > >> Thanks,
>> > > > > > > > >> Bob.
>> > > > > > > > >> --
>> > > > > > > > >> View this message in context:
>> > > > > > > > >>
>> > > > > > >
>> > > > >
>> > >
>>
http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682
>> > > > > > > > >> Sent from the Wicket - User mailing list archive at
>> Nabble.com
>> > > .
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > >
>> > > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > > > > > >> Using Tomcat but need to do more? Need to support web
>> services,
>> > > > > > > security?
>> > > > > > > > >> Get stuff done quickly with pre-integrated technology
to
>> make
>> > > your
>> > > > > job
>> > > > > > > > >> easier.
>> > > > > > > > >> Download IBM WebSphere Application Server v.1.0.1based
>> on
>> > > Apache
>> > > > > > > > >> Geronimo
>> > > > > > > > >>
>> > > > > > >
>> > > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > > > > > >>
>> > > _______________________________________________
>> > > > > > > > >> Wicket-user mailing list
>> > > > > > > > >> Wicket-user@lists.sourceforge.net
>> > > > > > > > >>
>> > > > > > >
>> > > > >
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > > > > > > Using Tomcat but need to do more? Need to support web
>> services,
>> > > > > > > security?
>> > > > > > > > > Get stuff done quickly with pre-integrated technology
to
>> make
>> > > your
>> > > > > job
>> > > > > > > > > easier.
>> > > > > > > > > Download IBM WebSphere Application Server v.1.0.1 based
>> on
>> > > Apache
>> > > > > > > Geronimo
>> > > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > > > > > >
>> _______________________________________________
>> > > > > > > > > Wicket-user mailing list
>> > > > > > > > > Wicket-user@lists.sourceforge.net
>> > > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > View this message in context:
>> > > > > > >
>> > > > >
>> > >
>>
http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8751579
>> > > > > > > > Sent from the Wicket - User mailing list archive at
>> Nabble.com
>> .
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > > > > > Using Tomcat but need to do more? Need to support web
>> services,
>> > > > > security?
>> > > > > > > > Get stuff done quickly with pre-integrated technology to
>> make
>> your
>> > > job
>> > > > > > > easier.
>> > > > > > > > Download IBM WebSphere Application Server v.1.0.1 based
on
>> Apache
>> > > > > Geronimo
>> > > > > > > >
>> > > > > > >
>> > > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > > > > >
>> _______________________________________________
>> > > > > > > > Wicket-user mailing list
>> > > > > > > > Wicket-user@lists.sourceforge.net
>> > > > > > > >
>> > > > >
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > > > > Using Tomcat but need to do more? Need to support web
>> services,
>> > > > > security?
>> > > > > > > Get stuff done quickly with pre-integrated technology to
make
>> your
>> > > job
>> > > > > > > easier.
>> > > > > > > Download IBM WebSphere Application Server v.1.0.1 based on
>> Apache
>> > > > > Geronimo
>> > > > > > >
>> > > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > > > >
>> > > > > > > _______________________________________________
>> > > > > > > Wicket-user mailing list
>> > > > > > > Wicket-user@lists.sourceforge.net
>> > > > > > >
>> > > > >
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > > > Using Tomcat but need to do more? Need to support web
services,
>> > > security?
>> > > > > > Get stuff done quickly with pre-integrated technology to make
>> your
>> job
>> > > > > easier.
>> > > > > > Download IBM WebSphere Application Server v.1.0.1 based on
>> Apache
>> > > Geronimo
>> > > > > >
>> > > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > > > _______________________________________________
>> > > > > > Wicket-user mailing list
>> > > > > > Wicket-user@lists.sourceforge.net
>> > > > > >
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > > Using Tomcat but need to do more? Need to support web services,
>> > > security?
>> > > > > Get stuff done quickly with pre-integrated technology to make
>> your
>> job
>> > > > > easier.
>> > > > > Download IBM WebSphere Application Server v.1.0.1 based on
Apache
>> > > Geronimo
>> > > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > > _______________________________________________
>> > > > > Wicket-user mailing list
>> > > > > Wicket-user@lists.sourceforge.net
>> > > > >
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > > >
>> > > > >
>> > > >
>> > > >
>> > >
>>
-------------------------------------------------------------------------
>> > > > Using Tomcat but need to do more? Need to support web services,
>> security?
>> > > > Get stuff done quickly with pre-integrated technology to make
your
>> job
>> > > easier.
>> > > > Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> > > >
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > > _______________________________________________
>> > > > Wicket-user mailing list
>> > > > Wicket-user@lists.sourceforge.net
>> > > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > > >
>> > >
>> > >
>> > >
>>
-------------------------------------------------------------------------
>> > > Using Tomcat but need to do more? Need to support web services,
>> security?
>> > > Get stuff done quickly with pre-integrated technology to make your
>> job
>> > > easier.
>> > > Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> > >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > > _______________________________________________
>> > > Wicket-user mailing list
>> > > Wicket-user@lists.sourceforge.net
>> > >
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>> > >
>> > >
>> >
>> >
>>
-------------------------------------------------------------------------
>> > Using Tomcat but need to do more? Need to support web services,
>> security?
>> > Get stuff done quickly with pre-integrated technology to make your
job
>> easier.
>> > Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> >
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > _______________________________________________
>> > Wicket-user mailing list
>> > Wicket-user@lists.sourceforge.net
>> > https://lists.sourceforge.net/lists/listinfo/wicket-user
>> >
>>
>>
>>
-------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services,
security?
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier.
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>
>> _______________________________________________
>> Wicket-user mailing list
>> Wicket-user@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/wicket-user
>>
>>
>>
>
>
-------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services,
security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Wicket-user mailing list
> Wicket-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/wicket-user
>
>
--
View this message in context:
http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8760198
Sent from the Wicket - User mailing list archive at Nabble.com.
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Wicket-user mailing list
Wicket-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wicket-user
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Wicket-user mailing list
Wicket-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wicket-user
