Re: [Wicket-develop] Memory profiling

Sun, 21 Aug 2005 16:02:06 -0700
Right... I made sure to force garbage collection before taking this snapshot. I'm using JProfiler and it doesn't have such a great capability of taking two snapshots and comparing them. It lets you compare the number of allocated objects between two snapshots but not the actual objects that differ, etc. 


	Still, I looked at the Tomcat code so I know for sure that it does not 
release its resources. Very annoying that :( What makes it worse is that 
Remy, who is in charge of Tomcat BugZilla, closes everyone's bugs as 
WON'T FIX or INVALID all the time even when they make perfect sense. 
Like for example, Tomcat has a cacheMaxSize and cacheTTL setting, but 
WebappClassLoader ignores them. Clearly this is a bug, but he closed it 
as INVALID without an explanation.


Gili

Johan Compagner wrote:

ehmmm
i could read this wrong of course. But this is a allocation hotspot view..

So course it is allocating more and more. What i see there is who is 
making Objects in memory

And yes resource loaders are making objects. But they will be cleaned up.


The question is what is being held in mem so you should compare 2 memory 
snapshots.


One where every thing is warmed up. All images are served once
and then do a refresh of those pages
anohter memory snapshot

and compare those. It shouldn't have grown much...



Gili wrote:




    Upon further investigation I think Tomcat is at fault and there is 
nothing I can do to fix it :( See the attached file...



    I see a constant pattern whereby ClassLoader.getResource() keeps 
on allocating more and more memory over the lifetime of my webapp 
(even after I force garbage collection) and all the calls are coming 
out of Wicket. Now, if I understand this correctly... when we use 
ClassLoader.getResource() it returns a URL but Tomcat also caches the 
bytes behind this resource indefinately inside a 
org.apache.catalina.loader.ResourceEntry which it stores inside 
WebappClassLoader.



    It never flushes these entries which is why they keep on building 
indefinately. So coming back to the original point, I'm no longer sure 
whether resolving the different locales to a single PackageResource 
would make a difference or not. But I *do* keep on seeing this 
allocation grow (to over 10MB after 30 minutes) so it's worrying me.


Gili

Johan Compagner wrote:


Those resourceStreams of a PackageResource aren't holding the 
complete size of youre stream?
It is just a wrapper around a UrlResourceStream where the real 
resource is closed after streaming it.

So we arenot storing an image in mem. We are just streaming it.


What you keep in mem for youre DynamicImages (in there byte buffer) 
is up to you


johan




Gili wrote:


Hi,


    I'm doing memory profiling on my webapp and it looks to me like 
we allocate a new ResourceStream per locale of the same packaged 
resource. The problem is that if I have a larged image exposed 
through a PackageResource and it is hit from 5 different locales, we 
store the exact same image in memory 5 times.



    Isn't there a way for us to resolve these requests to the same 
image? They are the same file in the underlying file-system. Taking 
this one step further, there should be a way for me to signal to 
Wicket that even my DynamicImages are locale-independant. Again, at 
500k-1MB per image, that's a lot of wasted memory (and I get hits 
from many different locales on my website). Can anyone help me with 
this optimization?


Thanks,
Gili





-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO

September 19-22, 2005 * San Francisco, CA * Development Lifecycle 
Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing 
& QA
Security * Process Improvement & Measurement * 
http://www.sqe.com/bsce5sf

_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop




------------------------------------------------------------------------


    Allocation hot spots

*Session:*     Apache Tomcat 5.x (with tomcat5.exe) on localhost
*Time of export:*     Sunday, August 21, 2005 3:03:54 PM EDT
*JVM time:*     16:17
       *Garbage collector mode: *     Live objects
*Allocations of: *     All classes
*Filtered classes: *     show separately

------------------------------------------------------------------------

      Hot spot     Allocated memory     Allocations

    org.apache.velocity.app.VelocityEngine.init      8,260 kB (18 
%)     123,791

    java.lang.ClassLoader.getResource      5,798 kB (12 %)     1,380

  12.2% - 5,430 kB - 795 alloc. 
wicket.util.resource.locator.ClassLoaderResourceStreamLocator.locate 
(line: 109)
  12.2% - 5,430 kB - 795 alloc. 
wicket.util.resource.locator.AbstractResourceStreamLocator.locate 
(line: 53)
  12.2% - 5,430 kB - 795 alloc. 
wicket.util.resource.locator.DefaultResourceStreamLocator$1.locate 
(line: 124)
  12.1% - 5,399 kB - 638 alloc. 
wicket.util.resource.locator.ResourceStreamLocator.locate (line: 180)
  12.1% - 5,399 kB - 638 alloc. 
wicket.markup.html.PackageResource.getResourceStream (line: 337)
  10.2% - 4,566 kB - 602 alloc. 
wicket.protocol.http.WicketServlet.getLastModified(javax.servlet.http.HttpServletRequest) 

  1.5% - 648 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/Wonderment_by_dilekt_en_US.jpg
  1.1% - 502 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/Ethereal_Bloom_by_sumopiggy_en_US.jpg
  1.1% - 482 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Ashundar_Mystical_Canyon_Final_by_merlin-tbd_en_US.jpg 

  1.0% - 447 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/The_Silver_Star_by_Skybase_en_US.jpg
  1.0% - 447 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Outburst_by_dilekt_en_US.jpg
  1.0% - 435 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Chasm_by_alyn_en_US.jpg
  1.0% - 434 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/final_by_active-x_en_US.jpg
  0.9% - 383 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Eternal_Origin_by_j4m3sb0nd_en_US.jpg
  0.8% - 346 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/Glacial_Haven_by_dilekt_en_US.jpg
  0.6% - 281 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Blue_Mist_by_dilekt_en_US.jpg
  0.1% - 30 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/The_Silver_Star_by_Skybase_en_US.jpg 


  0.0% - 18 kB - 12 alloc. URL: /Main/resources/shared/css/header.png

  0.0% - 8136 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Outburst_by_dilekt_en_US.jpg
  0.0% - 7344 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Wonderment_by_dilekt_en_US.jpg 

  0.0% - 5896 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/final_by_active-x_en_US.jpg
  0.0% - 5648 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Fate_X_by_active-x_en_US.jpg

  0.0% - 5128 bytes - 12 alloc. URL: /Main/resources/shared/css/page.css

  0.0% - 5080 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/download_en_US.png
  0.0% - 5040 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Ashundar_Mystical_Canyon_Final_by_merlin-tbd_en_US.jpg 

  0.0% - 4864 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/page.css
  0.0% - 4760 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Eternal_Origin_by_j4m3sb0nd_en_US.jpg 

  0.0% - 4632 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Glacial_Haven_by_dilekt_en_US.jpg 

  0.0% - 4376 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Sol_Scape_by_dilekt_en_US.jpg
  0.0% - 4296 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Blue_Mist_by_dilekt_en_US.jpg 

  0.0% - 4040 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Ethereal_Bloom_by_sumopiggy_en_US.jpg 

  0.0% - 3456 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Chasm_by_alyn_en_US.jpg
  0.0% - 2920 bytes - 14 alloc. URL: 
/Main/resources/base/css/ie5/navigation-shadow-top.png
  0.0% - 1992 bytes - 12 alloc. URL: 
/Main/resources/shared/css/header-right-corner.png
  0.0% - 1944 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-bottom-right.png
  0.0% - 1904 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/ie5.css
  0.0% - 1880 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-bottom.png

  0.0% - 1872 bytes - 12 alloc. URL: /Main/resources/shared/css/ie5.css

  0.0% - 1696 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-top-right.png
  0.0% - 1656 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-bottom-left.png
  0.0% - 1488 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-top-left.png

  0.0% - 1352 bytes - 12 alloc. URL: /Main/resources/shared/css/css2.css

  0.0% - 1336 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/css2.css
  0.0% - 1320 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-bottom-right.png
  0.0% - 1208 bytes - 12 alloc. URL: 
/Main/resources/base/css/ie5/navigation-shadow-bottom.png
  0.0% - 1200 bytes - 12 alloc. URL: 
/Main/resources/shared/css/ie5/navigation-gradient-bottom.png
  0.0% - 1192 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-bottom-left.png
  0.0% - 1176 bytes - 12 alloc. URL: 
/Main/resources/shared/css/header-right-padding.png
  0.0% - 1160 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-bottom.png
  0.0% - 1152 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-right.png
  0.0% - 1144 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-right.png
  0.0% - 1128 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-left.png
  0.0% - 1120 bytes - 12 alloc. URL: 
/Main/resources/shared/css/css2/gradient-bottom.png
  0.0% - 1120 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-top.png
  0.0% - 1112 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-left.png
  0.0% - 1104 bytes - 12 alloc. URL: 
/Main/resources/base/css/css2/shadow-bottom.png
  1.9% - 832 kB - 36 alloc. 
wicket.protocol.http.WicketServlet.getLastModified(javax.servlet.http.HttpServletRequest) 
(line: 162)
  0.1% - 30 kB - 157 alloc. 
wicket.util.resource.locator.ResourceStreamLocator.locate (line: 83)
  0.3% - 116 kB - 12 alloc. webwork.util.ClassLoaderUtils.getResource 
(line: 46)
  0.2% - 108 kB - 135 alloc. org.ofbiz.core.util.UtilURL.fromResource 
(line: 49)
  0.2% - 96 kB - 156 alloc. 
com.atlassian.core.util.ClassLoaderUtils.getResource (line: 143)
  0.0% - 17 kB - 12 alloc. 
com.opensymphony.workflow.loader.XMLWorkflowFactory$WorkflowConfig.<init> 
(line: 136)
  0.0% - 8152 bytes - 34 alloc. 
webwork.util.ClassLoaderUtils.getResource (line: 35)
  0.0% - 4272 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 72)
  0.0% - 4224 bytes - 24 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 70)
  0.0% - 4024 bytes - 81 alloc. 
wicket.util.resource.locator.ClassLoaderResourceStreamLocator.locate 
(line: 144)
  0.0% - 3648 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 134)
  0.0% - 1704 bytes - 71 alloc. 
wicket.util.resource.locator.ClassLoaderResourceStreamLocator.locate 
(line: 169)
  0.0% - 1272 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 44)
  0.0% - 1224 bytes - 12 alloc. 
com.atlassian.core.util.ClassLoaderUtils.getResource (line: 44)
  0.0% - 928 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 84)
    java.io.ByteArrayOutputStream.toByteArray      4,535 kB (10 %)     
310
    org.apache.velocity.servlet.VelocityServlet.init      4,225 kB (9 
%) 62,618
    org.apache.catalina.startup.Bootstrap.main      2,947 kB (6 %)     
42,434

    java.sql.PreparedStatement.executeQuery      1,652 kB (3 %)     81
    java.lang.Object.<init>      1,513 kB (3 %)     37,146

    org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run      
1,509 kB (3 %)     7,533

    java.lang.StringBuffer.toString      1,381 kB (3 %)     18,176

    java.lang.reflect.Constructor.newInstance      980 kB (2 %)     
18,263

    java.util.Map.put      923 kB (2 %)     23,728

    org.hibernate.type.BinaryType.deepCopyNotNull      809 kB (1 
%)     18

    java.lang.Class.forName      637 kB (1 %)     10,029
    javax.sql.DataSource.getConnection      606 kB (1 %)     9,113
    java.sql.Connection.prepareStatement      595 kB (1 %)     3,123
    java.lang.Class.getDeclaredConstructors      591 kB (1 %)     8,700
    org.apache.velocity.Template.merge      489 kB (1 %)     11,521
    java.util.Properties.load      332 kB (0 %)     7,372
    java.lang.reflect.Method.invoke      332 kB (0 %)     6,275

    javax.xml.parsers.DocumentBuilder.parse(java.io.InputStream)      
322 kB (0 %)     4,815

    java.lang.ClassLoader.loadClass      319 kB (0 %)     4,685

    java.awt.GraphicsEnvironment.getAvailableFontFamilyNames      297 
kB (0 %)     9,306

    org.apache.commons.lang.StringUtils.split      262 kB (0 %)     10
    org.dom4j.io.SAXReader.read      188 kB (0 %)     5,440

    java.util.ResourceBundle.getBundle(java.lang.String)      181 kB 
(0 %) 3,592

    java.lang.Integer.<init>      180 kB (0 %)     11,298

    javax.servlet.http.HttpServletRequest.getReader      174 kB (0 
%)     86

    java.io.OutputStream.write      163 kB (0 %)     6

    java.awt.image.BufferedImage.createGraphics      135 kB (0 %)     
3,362
    desktopbeautifier.server.database.AbstractCachedImage$$EnhancerByCGLIB$$125e308b.newInstance 
     127 kB (0 %)     3,982

    java.util.HashMap.<init>()      127 kB (0 %)     2,136

    java.util.ResourceBundle.getBundle(java.lang.String, 
java.util.Locale, java.lang.ClassLoader)      125 kB (0 %)     69

    java.sql.PreparedStatement.executeUpdate      122 kB (0 %)     10
    java.util.ArrayList.<init>()      120 kB (0 %)     3,025
    org.apache.log4j.Category.getInstance      112 kB (0 %)     1,962

    javax.servlet.http.HttpServletResponse.getWriter      102 kB (0 
%)     132

    java.lang.Class.newInstance      96 kB (0 %)     2,104
    java.lang.Class.getDeclaredMethod      91 kB (0 %)     1,602

    javax.xml.parsers.DocumentBuilder.parse(java.lang.String)      88 
kB (0 %)     2,162

    java.util.Iterator.next      79 kB (0 %)     233
    java.beans.Introspector.getBeanInfo      77 kB (0 %)     2,149
    java.lang.Class.getDeclaredMethods      77 kB (0 %)     1,477
    java.lang.ClassLoader.getResourceAsStream      67 kB (0 %)     176
    java.sql.ResultSet.getString      66 kB (0 %)     1,015

    org.hibernate.persister.entity.BasicEntityPersister.hydrate      
64 kB (0 %)     2,009

    java.lang.String.getBytes      64 kB (0 %)     2,169

    javax.xml.parsers.DocumentBuilderFactory.newDocumentBuilder      
57 kB (0 %)     977
    org.apache.commons.logging.LogFactory.getLog      56 kB (0 %)     
1,617

    javax.imageio.ImageIO.createImageInputStream      52 kB (0 %)     408
    java.util.Hashtable.put      49 kB (0 %)     1,663

    org.hibernate.sql.Alias.toUnquotedAliasStrings      49 kB (0 
%)     3,063

    java.lang.Class.getMethod      48 kB (0 %)     996
    java.util.HashSet.<init>      45 kB (0 %)     1,000
    java.util.HashMap.<init>(int)      39 kB (0 %)     807

    desktopbeautifier.server.database.Image$$EnhancerByCGLIB$$94713165.newInstance 
     38 kB (0 %)     1,191

    java.security.MessageDigest.getInstance      37 kB (0 %)     883

    net.jforum.dao.generic.DataAccessDriver.<clinit>      37 kB (0 
%)     580

    antlr.ANTLRHashString.<init>      35 kB (0 %)     1,488

    org.apache.log4j.xml.DOMConfigurator.configure      34 kB (0 
%)     429

    javax.servlet.jsp.JspFactory.getPageContext      33 kB (0 %)     38
    net.jforum.ConfigLoader.startCacheEngine      32 kB (0 %)     528
    java.lang.Float.<init>      31 kB (0 %)     1,990
    javax.activation.MimetypesFileTypeMap.<init>      30 kB (0 %)     962

    org.hibernate.loader.OuterJoinLoader.initPersisters      30 kB (0 
%)     1,896

    javax.servlet.FilterChain.doFilter      29 kB (0 %)     610
    java.net.URLConnection.getLastModified      26 kB (0 %)     622

    desktopbeautifier.clientserver.DefaultImageSpecification.<init>      
26 kB (0 %)     856
    org.apache.velocity.tools.view.servlet.ServletToolboxManager.getInstance 
     26 kB (0 %)     538

    java.io.PrintWriter.<init>      24 kB (0 %)     11
    java.io.File.<init>      24 kB (0 %)     245
    java.util.regex.Pattern.compile      24 kB (0 %)     564
    java.net.URLConnection.getContentLength      23 kB (0 %)     605
    URL: /download/DesktopBeautifier-Setup.exe      23 kB (0 %)     103
    java.util.ArrayList.<init>(int)      23 kB (0 %)     592
    java.lang.reflect.Proxy.newProxyInstance      23 kB (0 %)     512
    org.objectweb.asm.Label.<init>      22 kB (0 %)     469




-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop



--
http://www.desktopbeautifier.com/


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop






















					Reply via email to