Re: [Wicket-develop] Memory profiling

Sun, 21 Aug 2005 20:20:32 -0700
Have you tried JProfiler recently? I'm just curious what you thought of one versus the other. 


	I took a quick look at YourKit's feature set. Memory profiling wise, it 
doesn't seem to have more functionality than JProfiler (i.e. I can't see 
*which* objects were added across the two snapshots, simply *how many* 
were added). I think you can achieve this indirectly in JProfiler by 
sorting the live objects by time of creation and then look at the most 
recently created objects; though this isn't very clean in my books.


        Anyway, I'll keep on trying.

Gili

Johan Compagner wrote:

www.yourkit.com
Works perfectly the best i can find and i have tried many

Gili wrote:




    Right... I made sure to force garbage collection before taking 
this snapshot. I'm using JProfiler and it doesn't have such a great 
capability of taking two snapshots and comparing them. It lets you 
compare the number of allocated objects between two snapshots but not 
the actual objects that differ, etc.



    Still, I looked at the Tomcat code so I know for sure that it does 
not release its resources. Very annoying that :( What makes it worse 
is that Remy, who is in charge of Tomcat BugZilla, closes everyone's 
bugs as WON'T FIX or INVALID all the time even when they make perfect 
sense. Like for example, Tomcat has a cacheMaxSize and cacheTTL 
setting, but WebappClassLoader ignores them. Clearly this is a bug, 
but he closed it as INVALID without an explanation.


Gili

Johan Compagner wrote:


ehmmm

i could read this wrong of course. But this is a allocation hotspot 
view..
So course it is allocating more and more. What i see there is who is 
making Objects in memory
And yes resource loaders are making objects. But they will be cleaned 
up.



The question is what is being held in mem so you should compare 2 
memory snapshots.


One where every thing is warmed up. All images are served once
and then do a refresh of those pages
anohter memory snapshot

and compare those. It shouldn't have grown much...



Gili wrote:




    Upon further investigation I think Tomcat is at fault and there 
is nothing I can do to fix it :( See the attached file...



    I see a constant pattern whereby ClassLoader.getResource() keeps 
on allocating more and more memory over the lifetime of my webapp 
(even after I force garbage collection) and all the calls are coming 
out of Wicket. Now, if I understand this correctly... when we use 
ClassLoader.getResource() it returns a URL but Tomcat also caches 
the bytes behind this resource indefinately inside a 
org.apache.catalina.loader.ResourceEntry which it stores inside 
WebappClassLoader.



    It never flushes these entries which is why they keep on 
building indefinately. So coming back to the original point, I'm no 
longer sure whether resolving the different locales to a single 
PackageResource would make a difference or not. But I *do* keep on 
seeing this allocation grow (to over 10MB after 30 minutes) so it's 
worrying me.


Gili

Johan Compagner wrote:


Those resourceStreams of a PackageResource aren't holding the 
complete size of youre stream?
It is just a wrapper around a UrlResourceStream where the real 
resource is closed after streaming it.

So we arenot storing an image in mem. We are just streaming it.


What you keep in mem for youre DynamicImages (in there byte buffer) 
is up to you


johan




Gili wrote:


Hi,


    I'm doing memory profiling on my webapp and it looks to me 
like we allocate a new ResourceStream per locale of the same 
packaged resource. The problem is that if I have a larged image 
exposed through a PackageResource and it is hit from 5 different 
locales, we store the exact same image in memory 5 times.



    Isn't there a way for us to resolve these requests to the same 
image? They are the same file in the underlying file-system. 
Taking this one step further, there should be a way for me to 
signal to Wicket that even my DynamicImages are 
locale-independant. Again, at 500k-1MB per image, that's a lot of 
wasted memory (and I get hits from many different locales on my 
website). Can anyone help me with this optimization?


Thanks,
Gili






-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO

September 19-22, 2005 * San Francisco, CA * Development Lifecycle 
Practices
Agile & Plan-Driven Development * Managing Projects & Teams * 
Testing & QA
Security * Process Improvement & Measurement * 
http://www.sqe.com/bsce5sf

_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop





------------------------------------------------------------------------ 




    Allocation hot spots

*Session:*     Apache Tomcat 5.x (with tomcat5.exe) on localhost
*Time of export:*     Sunday, August 21, 2005 3:03:54 PM EDT
*JVM time:*     16:17
       *Garbage collector mode: *     Live objects
*Allocations of: *     All classes
*Filtered classes: *     show separately


------------------------------------------------------------------------ 



      Hot spot     Allocated memory     Allocations

    org.apache.velocity.app.VelocityEngine.init      8,260 kB (18 
%)     123,791

    java.lang.ClassLoader.getResource      5,798 kB (12 %)     1,380

  12.2% - 5,430 kB - 795 alloc. 
wicket.util.resource.locator.ClassLoaderResourceStreamLocator.locate 
(line: 109)
  12.2% - 5,430 kB - 795 alloc. 
wicket.util.resource.locator.AbstractResourceStreamLocator.locate 
(line: 53)
  12.2% - 5,430 kB - 795 alloc. 
wicket.util.resource.locator.DefaultResourceStreamLocator$1.locate 
(line: 124)
  12.1% - 5,399 kB - 638 alloc. 
wicket.util.resource.locator.ResourceStreamLocator.locate (line: 180)
  12.1% - 5,399 kB - 638 alloc. 
wicket.markup.html.PackageResource.getResourceStream (line: 337)
  10.2% - 4,566 kB - 602 alloc. 
wicket.protocol.http.WicketServlet.getLastModified(javax.servlet.http.HttpServletRequest) 

  1.5% - 648 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/Wonderment_by_dilekt_en_US.jpg
  1.1% - 502 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/Ethereal_Bloom_by_sumopiggy_en_US.jpg
  1.1% - 482 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Ashundar_Mystical_Canyon_Final_by_merlin-tbd_en_US.jpg 

  1.0% - 447 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/The_Silver_Star_by_Skybase_en_US.jpg
  1.0% - 447 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Outburst_by_dilekt_en_US.jpg
  1.0% - 435 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Chasm_by_alyn_en_US.jpg
  1.0% - 434 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/final_by_active-x_en_US.jpg
  0.9% - 383 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Eternal_Origin_by_j4m3sb0nd_en_US.jpg
  0.8% - 346 kB - 12 alloc. URL: 
/Main/resources/wallpapers/space/Glacial_Haven_by_dilekt_en_US.jpg
  0.6% - 281 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/Blue_Mist_by_dilekt_en_US.jpg
  0.1% - 30 kB - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/The_Silver_Star_by_Skybase_en_US.jpg 


  0.0% - 18 kB - 12 alloc. URL: /Main/resources/shared/css/header.png

  0.0% - 8136 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Outburst_by_dilekt_en_US.jpg 

  0.0% - 7344 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Wonderment_by_dilekt_en_US.jpg 

  0.0% - 5896 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/final_by_active-x_en_US.jpg
  0.0% - 5648 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Fate_X_by_active-x_en_US.jpg 

  0.0% - 5128 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page.css
  0.0% - 5080 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/download_en_US.png
  0.0% - 5040 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Ashundar_Mystical_Canyon_Final_by_merlin-tbd_en_US.jpg 

  0.0% - 4864 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/page.css
  0.0% - 4760 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Eternal_Origin_by_j4m3sb0nd_en_US.jpg 

  0.0% - 4632 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Glacial_Haven_by_dilekt_en_US.jpg 

  0.0% - 4376 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Sol_Scape_by_dilekt_en_US.jpg 

  0.0% - 4296 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Blue_Mist_by_dilekt_en_US.jpg 

  0.0% - 4040 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/space/thumbnails/Ethereal_Bloom_by_sumopiggy_en_US.jpg 

  0.0% - 3456 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/sunset/thumbnails/Chasm_by_alyn_en_US.jpg
  0.0% - 2920 bytes - 14 alloc. URL: 
/Main/resources/base/css/ie5/navigation-shadow-top.png
  0.0% - 1992 bytes - 12 alloc. URL: 
/Main/resources/shared/css/header-right-corner.png
  0.0% - 1944 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-bottom-right.png
  0.0% - 1904 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/ie5.css
  0.0% - 1880 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-bottom.png

  0.0% - 1872 bytes - 12 alloc. URL: /Main/resources/shared/css/ie5.css

  0.0% - 1696 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-top-right.png
  0.0% - 1656 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-bottom-left.png
  0.0% - 1488 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-top-left.png
  0.0% - 1352 bytes - 12 alloc. URL: 
/Main/resources/shared/css/css2.css
  0.0% - 1336 bytes - 12 alloc. URL: 
/Main/resources/wallpapers/css/css2.css
  0.0% - 1320 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-bottom-right.png
  0.0% - 1208 bytes - 12 alloc. URL: 
/Main/resources/base/css/ie5/navigation-shadow-bottom.png
  0.0% - 1200 bytes - 12 alloc. URL: 
/Main/resources/shared/css/ie5/navigation-gradient-bottom.png
  0.0% - 1192 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-bottom-left.png
  0.0% - 1176 bytes - 12 alloc. URL: 
/Main/resources/shared/css/header-right-padding.png
  0.0% - 1160 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-bottom.png
  0.0% - 1152 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-right.png
  0.0% - 1144 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-right.png
  0.0% - 1128 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-left.png
  0.0% - 1120 bytes - 12 alloc. URL: 
/Main/resources/shared/css/css2/gradient-bottom.png
  0.0% - 1120 bytes - 12 alloc. URL: 
/Main/resources/shared/css/image-shadow-top.png
  0.0% - 1112 bytes - 12 alloc. URL: 
/Main/resources/shared/css/page-shadow-left.png
  0.0% - 1104 bytes - 12 alloc. URL: 
/Main/resources/base/css/css2/shadow-bottom.png
  1.9% - 832 kB - 36 alloc. 
wicket.protocol.http.WicketServlet.getLastModified(javax.servlet.http.HttpServletRequest) 
(line: 162)
  0.1% - 30 kB - 157 alloc. 
wicket.util.resource.locator.ResourceStreamLocator.locate (line: 83)
  0.3% - 116 kB - 12 alloc. 
webwork.util.ClassLoaderUtils.getResource (line: 46)
  0.2% - 108 kB - 135 alloc. 
org.ofbiz.core.util.UtilURL.fromResource (line: 49)
  0.2% - 96 kB - 156 alloc. 
com.atlassian.core.util.ClassLoaderUtils.getResource (line: 143)
  0.0% - 17 kB - 12 alloc. 
com.opensymphony.workflow.loader.XMLWorkflowFactory$WorkflowConfig.<init> 
(line: 136)
  0.0% - 8152 bytes - 34 alloc. 
webwork.util.ClassLoaderUtils.getResource (line: 35)
  0.0% - 4272 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 72)
  0.0% - 4224 bytes - 24 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 70)
  0.0% - 4024 bytes - 81 alloc. 
wicket.util.resource.locator.ClassLoaderResourceStreamLocator.locate 
(line: 144)
  0.0% - 3648 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 134)
  0.0% - 1704 bytes - 71 alloc. 
wicket.util.resource.locator.ClassLoaderResourceStreamLocator.locate 
(line: 169)
  0.0% - 1272 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 44)
  0.0% - 1224 bytes - 12 alloc. 
com.atlassian.core.util.ClassLoaderUtils.getResource (line: 44)
  0.0% - 928 bytes - 12 alloc. 
com.opensymphony.util.ClassLoaderUtil.getResource (line: 84)
    java.io.ByteArrayOutputStream.toByteArray      4,535 kB (10 
%)     310
    org.apache.velocity.servlet.VelocityServlet.init      4,225 kB 
(9 %) 62,618
    org.apache.catalina.startup.Bootstrap.main      2,947 kB (6 
%)     42,434

    java.sql.PreparedStatement.executeQuery      1,652 kB (3 %)     81
    java.lang.Object.<init>      1,513 kB (3 %)     37,146

    
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run      
1,509 kB (3 %)     7,533

    java.lang.StringBuffer.toString      1,381 kB (3 %)     18,176

    java.lang.reflect.Constructor.newInstance      980 kB (2 %)     
18,263

    java.util.Map.put      923 kB (2 %)     23,728

    org.hibernate.type.BinaryType.deepCopyNotNull      809 kB (1 
%)     18

    java.lang.Class.forName      637 kB (1 %)     10,029
    javax.sql.DataSource.getConnection      606 kB (1 %)     9,113
    java.sql.Connection.prepareStatement      595 kB (1 %)     3,123
    java.lang.Class.getDeclaredConstructors      591 kB (1 %)     8,700
    org.apache.velocity.Template.merge      489 kB (1 %)     11,521
    java.util.Properties.load      332 kB (0 %)     7,372
    java.lang.reflect.Method.invoke      332 kB (0 %)     6,275

    
javax.xml.parsers.DocumentBuilder.parse(java.io.InputStream)      
322 kB (0 %)     4,815

    java.lang.ClassLoader.loadClass      319 kB (0 %)     4,685

    java.awt.GraphicsEnvironment.getAvailableFontFamilyNames      
297 kB (0 %)     9,306

    org.apache.commons.lang.StringUtils.split      262 kB (0 %)     10
    org.dom4j.io.SAXReader.read      188 kB (0 %)     5,440

    java.util.ResourceBundle.getBundle(java.lang.String)      181 kB 
(0 %) 3,592

    java.lang.Integer.<init>      180 kB (0 %)     11,298

    javax.servlet.http.HttpServletRequest.getReader      174 kB (0 
%)     86

    java.io.OutputStream.write      163 kB (0 %)     6

    java.awt.image.BufferedImage.createGraphics      135 kB (0 
%)     3,362
    
desktopbeautifier.server.database.AbstractCachedImage$$EnhancerByCGLIB$$125e308b.newInstance 
     127 kB (0 %)     3,982

    java.util.HashMap.<init>()      127 kB (0 %)     2,136

    java.util.ResourceBundle.getBundle(java.lang.String, 
java.util.Locale, java.lang.ClassLoader)      125 kB (0 %)     69

    java.sql.PreparedStatement.executeUpdate      122 kB (0 %)     10
    java.util.ArrayList.<init>()      120 kB (0 %)     3,025
    org.apache.log4j.Category.getInstance      112 kB (0 %)     1,962

    javax.servlet.http.HttpServletResponse.getWriter      102 kB (0 
%)     132

    java.lang.Class.newInstance      96 kB (0 %)     2,104
    java.lang.Class.getDeclaredMethod      91 kB (0 %)     1,602

    javax.xml.parsers.DocumentBuilder.parse(java.lang.String)      
88 kB (0 %)     2,162

    java.util.Iterator.next      79 kB (0 %)     233
    java.beans.Introspector.getBeanInfo      77 kB (0 %)     2,149
    java.lang.Class.getDeclaredMethods      77 kB (0 %)     1,477
    java.lang.ClassLoader.getResourceAsStream      67 kB (0 %)     176
    java.sql.ResultSet.getString      66 kB (0 %)     1,015

    org.hibernate.persister.entity.BasicEntityPersister.hydrate      
64 kB (0 %)     2,009

    java.lang.String.getBytes      64 kB (0 %)     2,169

    javax.xml.parsers.DocumentBuilderFactory.newDocumentBuilder      
57 kB (0 %)     977
    org.apache.commons.logging.LogFactory.getLog      56 kB (0 
%)     1,617
    javax.imageio.ImageIO.createImageInputStream      52 kB (0 
%)     408

    java.util.Hashtable.put      49 kB (0 %)     1,663

    org.hibernate.sql.Alias.toUnquotedAliasStrings      49 kB (0 
%)     3,063

    java.lang.Class.getMethod      48 kB (0 %)     996
    java.util.HashSet.<init>      45 kB (0 %)     1,000
    java.util.HashMap.<init>(int)      39 kB (0 %)     807

    
desktopbeautifier.server.database.Image$$EnhancerByCGLIB$$94713165.newInstance 
     38 kB (0 %)     1,191

    java.security.MessageDigest.getInstance      37 kB (0 %)     883

    net.jforum.dao.generic.DataAccessDriver.<clinit>      37 kB (0 
%)     580

    antlr.ANTLRHashString.<init>      35 kB (0 %)     1,488

    org.apache.log4j.xml.DOMConfigurator.configure      34 kB (0 
%)     429

    javax.servlet.jsp.JspFactory.getPageContext      33 kB (0 %)     38
    net.jforum.ConfigLoader.startCacheEngine      32 kB (0 %)     528
    java.lang.Float.<init>      31 kB (0 %)     1,990

    javax.activation.MimetypesFileTypeMap.<init>      30 kB (0 
%)     962
    org.hibernate.loader.OuterJoinLoader.initPersisters      30 kB 
(0 %)     1,896

    javax.servlet.FilterChain.doFilter      29 kB (0 %)     610
    java.net.URLConnection.getLastModified      26 kB (0 %)     622

    
desktopbeautifier.clientserver.DefaultImageSpecification.<init>      
26 kB (0 %)     856
    
org.apache.velocity.tools.view.servlet.ServletToolboxManager.getInstance 
     26 kB (0 %)     538

    java.io.PrintWriter.<init>      24 kB (0 %)     11
    java.io.File.<init>      24 kB (0 %)     245
    java.util.regex.Pattern.compile      24 kB (0 %)     564
    java.net.URLConnection.getContentLength      23 kB (0 %)     605
    URL: /download/DesktopBeautifier-Setup.exe      23 kB (0 %)     103
    java.util.ArrayList.<init>(int)      23 kB (0 %)     592
    java.lang.reflect.Proxy.newProxyInstance      23 kB (0 %)     512
    org.objectweb.asm.Label.<init>      22 kB (0 %)     469




-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO

September 19-22, 2005 * San Francisco, CA * Development Lifecycle 
Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing 
& QA
Security * Process Improvement & Measurement * 
http://www.sqe.com/bsce5sf

_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop






-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop



--
http://www.desktopbeautifier.com/


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Wicket-develop mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/wicket-develop






















					Reply via email to