Hi, I have been fighting with OOMEs and yesterday I was able to track down the cause.
There are lot of ConcurrentReaderHashMap entries and it seems that something is repeating itself. I have heapdump (about 300MB) which can be analyzed using MDD4J for IBM JVM. I can provide them for further investigation. Br, Juha Moisio anonymous wrote : | +59 105 812 [36] 2 java/util/ArrayList 0x9009b088 | ++59 105 776 [20] 2 array of java/lang/Object 0x9009b0a0 | +++59 105 756 [40] 5 org/jboss/mx/server/MBeanServerImpl 0x900a0188 | ++++44 764 058 [28] 2 org/jboss/mx/notification/MBeanServerListenerRegistry 0x900a05f8 | ++++14 323 970 [60] 9 org/jboss/mx/server/registry/BasicMBeanRegistry 0x900ab5c8 | +++++14 321 526 [68] 5 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x900ab5f8 | ++++++14 321 406 [124] 28 array of EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91a50d48 | +++++++8 345 240 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9263f7e0 | ++++++++8 344 940 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90980168 | +++++++++8 344 860 [68] 5 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x90980120 | ++++++++++8 344 740 [516] 126 array of EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92e48138 | +++++++++++262 706 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x943ecf68 | ++++++++++++211 488 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92f7e6d0 | ++++++++++++51 138 [48] 5 org/jboss/mx/server/registry/MBeanEntry 0x943e5880 | ++++++++++++40 [40] 2 java/lang/String 0x943ecf48 | ++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | +++++++++++199 550 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94836c48 | ++++++++++++148 348 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x905a77d0 | +++++++++++++119 108 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9252c4f8 | ++++++++++++++67 872 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x917684a0 | +++++++++++++++67 792 [48] 5 org/jboss/mx/server/registry/MBeanEntry 0x9175e208 | ++++++++++++++++49 776 [112] 14 org/jboss/mx/modelmbean/XMBean 0x917573e0 | +++++++++++++++++17 542 [60] 3 java/util/HashMap 0x91757448 | +++++++++++++++++15 888 [76] 12 javax/management/modelmbean/ModelMBeanInfoSupport 0x9175e088 | +++++++++++++++++14 260 [60] 3 java/util/HashMap 0x917574c8 | +++++++++++++++++1 554 [84] 5 org/jboss/mx/server/InvocationContext 0x917637d0 | +++++++++++++++++200 [32] 3 org/jboss/mx/util/JBossNotificationBroadcasterSupport 0x917575d8 | +++++++++++++++++76 [60] 2 java/util/HashMap 0x917574f8 | +++++++++++++++++60 [32] 3 org/jboss/logging/Logger 0x917575a0 | +++++++++++++++++60 [32] 3 org/jboss/logging/Logger 0x91757578 | +++++++++++++++++24 [24] 1 org/jboss/mx/persistence/NullPersistence 0x917575c8 | +++++++++++++++++59 105 756 [40] 5 org/jboss/mx/server/MBeanServerImpl 0x900a0188 | ++++++++++++++++++44 764 058 [28] 2 org/jboss/mx/notification/MBeanServerListenerRegistry 0x900a05f8 | ++++++++++++++++++14 323 970 [60] 9 org/jboss/mx/server/registry/BasicMBeanRegistry 0x900ab5c8 | +++++++++++++++++++14 321 526 [68] 5 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x900ab5f8 | ++++++++++++++++++++14 321 406 [124] 28 array of EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91a50d48 | +++++++++++++++++++++8 345 240 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9263f7e0 | ++++++++++++++++++++++8 344 940 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90980168 | +++++++++++++++++++++++8 344 860 [68] 5 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x90980120 | ++++++++++++++++++++++++8 344 740 [516] 126 array of EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92e48138 | +++++++++++++++++++++++++262 706 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x943ecf68 | +++++++++++++++++++++++++199 550 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94836c48 | +++++++++++++++++++++++++175 748 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9295e938 | +++++++++++++++++++++++++170 902 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9478dd48 | +++++++++++++++++++++++++156 548 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94376df8 | +++++++++++++++++++++++++153 568 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90831260 | +++++++++++++++++++++++++142 646 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9279fc68 | +++++++++++++++++++++++++136 276 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9473f100 | +++++++++++++++++++++++++135 522 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92d5c040 | +++++++++++++++++++++++++123 272 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92850168 | +++++++++++++++++++++++++121 382 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94703e80 | +++++++++++++++++++++++++119 710 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x924de6a8 | +++++++++++++++++++++++++119 114 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92e48588 | +++++++++++++++++++++++++118 962 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90bce940 | +++++++++++++++++++++++++104 730 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x93e8d698 | +++++++++++++++++++++++++104 152 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92e48688 | +++++++++++++++++++++++++103 014 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9426fc18 | +++++++++++++++++++++++++102 486 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9115a2b0 | +++++++++++++++++++++++++102 448 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x943859d8 | +++++++++++++++++++++++++102 436 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91117580 | ++++++++++++++++++++++++There are 106 more children. | ++++++++++++++++++++++++28 [28] 2 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Values 0x902b0370 | ++++++++++++++++++++++++24 [24] 1 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$BarrierLock 0x90980158 | ++++++++++++++++++++++++51 256 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94846e30 | ++++++++++++++++++++++++2 048 [256] 20 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x32a27428 | +++++++++++++++++++++++40 [40] 2 java/lang/String 0x909800e0 | +++++++++++++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | ++++++++++++++++++++++220 [68] 5 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x9263f708 | ++++++++++++++++++++++40 [40] 2 java/lang/String 0x9263f6c8 | ++++++++++++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | +++++++++++++++++++++2 365 226 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90d64f40 | +++++++++++++++++++++1 969 998 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91a50d28 | +++++++++++++++++++++408 298 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90ffd098 | +++++++++++++++++++++375 600 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90424678 | +++++++++++++++++++++175 956 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90407060 | +++++++++++++++++++++170 156 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9235a5f8 | +++++++++++++++++++++117 020 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90702778 | +++++++++++++++++++++82 494 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x919a5ab8 | +++++++++++++++++++++73 486 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x907cd7a0 | +++++++++++++++++++++58 134 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x911aada8 | +++++++++++++++++++++56 234 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x900caf68 | +++++++++++++++++++++44 078 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91a50e98 | +++++++++++++++++++++24 628 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91d18788 | +++++++++++++++++++++18 506 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90620d60 | +++++++++++++++++++++17 536 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91a50e58 | +++++++++++++++++++++11 220 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9072a108 | +++++++++++++++++++++4 100 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x943264a0 | +++++++++++++++++++++552 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92556908 | +++++++++++++++++++++468 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x901e45e8 | ++++++++++++++++++++There are 8 more children. | ++++++++++++++++++++28 [28] 2 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$EntrySet 0x902af4b8 | ++++++++++++++++++++24 [24] 1 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$BarrierLock 0x900ab630 | ++++++++++++++++++++4 100 [40] 3 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x943264a0 | ++++++++++++++++++++2 048 [256] 20 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap 0x32a27428 | +++++++++++++++++++2 184 [256] 70 class org/jboss/mx/server/registry/BasicMBeanRegistry 0x32945e60 | +++++++++++++++++++128 [52] 4 javax/management/ObjectName 0x900ab670 | +++++++++++++++++++36 [36] 2 EDU/oswego/cs/dl/util/concurrent/SynchronizedLong 0x900ab658 | +++++++++++++++++++36 [36] 2 EDU/oswego/cs/dl/util/concurrent/SynchronizedLong 0x900ab640 | +++++++++++++++++++63 000 452 [88] 14 org/jboss/mx/loading/UnifiedLoaderRepository3 0x900a4120 | +++++++++++++++++++59 105 756 [40] 5 org/jboss/mx/server/MBeanServerImpl 0x900a0188 | +++++++++++++++++++11 594 [48] 3 javax/management/MBeanServerDelegate 0x9009ba80 | +++++++++++++++++++62 [40] 2 java/lang/String 0x9004ff40 | ++++++++++++++++++17 688 [256] 161 class org/jboss/mx/server/MBeanServerImpl 0x32a22f30 | ++++++++++++++++++63 000 452 [88] 14 org/jboss/mx/loading/UnifiedLoaderRepository3 0x900a4120 | ++++++++++++++++++59 105 756 [40] 5 org/jboss/mx/server/MBeanServerImpl 0x900a0188 | +++++++++++++++++105 320 [256] 60 class org/jboss/mx/modelmbean/XMBean 0x32948da0 | +++++++++++++++++67 792 [48] 5 org/jboss/mx/server/registry/MBeanEntry 0x9175e208 | +++++++++++++++++17 968 [56] 9 org/jboss/management/j2ee/ServiceModule 0x91756f80 | +++++++++++++++++78 [40] 2 java/lang/String 0x900b1680 | ++++++++++++++++17 968 [56] 9 org/jboss/management/j2ee/ServiceModule 0x91756f80 | ++++++++++++++++652 [52] 5 javax/management/ObjectName 0x91757070 | ++++++++++++++++356 [256] 9 class org/jboss/mx/server/registry/MBeanEntry 0x3294a8b0 | ++++++++++++++++130 [40] 2 java/lang/String 0x9038ed70 | +++++++++++++++40 [40] 2 java/lang/String 0x91768480 | +++++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | ++++++++++++++51 156 [48] 5 org/jboss/mx/server/registry/MBeanEntry 0x925226e0 | ++++++++++++++40 [40] 2 java/lang/String 0x9252c4d8 | ++++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | +++++++++++++29 160 [48] 5 org/jboss/mx/server/registry/MBeanEntry 0x905a3b68 | +++++++++++++40 [40] 2 java/lang/String 0x905a77b0 | +++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | ++++++++++++51 122 [48] 5 org/jboss/mx/server/registry/MBeanEntry 0x9481f978 | ++++++++++++40 [40] 2 java/lang/String 0x94836c28 | ++++++++++++512 [256] 6 class EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x32a27bf0 | +++++++++++175 748 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9295e938 | +++++++++++170 902 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9478dd48 | +++++++++++156 548 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94376df8 | +++++++++++153 568 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90831260 | +++++++++++142 646 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9279fc68 | +++++++++++136 276 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9473f100 | +++++++++++135 522 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92d5c040 | +++++++++++123 272 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92850168 | +++++++++++121 382 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x94703e80 | +++++++++++119 710 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x924de6a8 | +++++++++++119 114 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92e48588 | +++++++++++118 962 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x90bce940 | +++++++++++104 730 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x93e8d698 | +++++++++++104 152 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x92e48688 | +++++++++++103 014 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9426fc18 | +++++++++++102 486 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x9115a2b0 | +++++++++++102 448 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x943859d8 | +++++++++++102 436 [40] 4 EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry 0x91117580 | +++++++++++There are 106 more children. | | View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3996805#3996805 Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=3996805 _______________________________________________ jboss-user mailing list [email protected] https://lists.jboss.org/mailman/listinfo/jboss-user
