Re: ApacheDS causing very high load

Wed, 09 Dec 2009 21:59:55 -0800

Sure, I'll arrange this with our sysadm. Do you have anything particular in mind or will jvisualvm do? 

Y.

On 10 Dec 2009, at 04:00, Alex Karasulu wrote:
It would be nice to run your server with a profiler to be able to catch just when it does get into this state. This would be very valuable since we'd see exactly where the code is getting stuck in this terrible loop that eats 
your CPU.
You think you can give that a try Ioannis? If you can I'm sure we can work 
together to nip this bug quickly.

Thanks,
Alex

On Wed, Dec 9, 2009 at 11:04 AM, Ioannis Mavroukakis <
[email protected]> wrote:


Hi Emmanuel,
Don't think it is too, as the same behaviour persists when we shut down ADS and re-start it. Only clearing the ADS directory allows it to recover! I 
have kept the directory contents though, is there
any way I can run any forensics on them ?

Thanks,

Y.

On 9 Dec 2009, at 15:21, Emmanuel LŽcharny wrote:

Ioannis Mavroukakis a écrit :



Hey fellow listers.


Hi Ioannis,
if you get 100% loaded CPU the it might be MINA which is the problem. There is a painful bug in the way the JVM handle epoll, and in certain conditions, it jump to 100% CPU and never goes down. It seems to be a problem when a connection is opened, and closed before the selector can see the close : the select() methods returns at least one selectedKey, but as the connection has been closed, it does nothing but loop again (as you can 
imagine, the loop is infinite).

There is a fix for that, but it's experimental, in MINA branch
http://svn.apache.org/repos/asf/mina/branches/select-fix/. I will commit this fix immediately in the MINA trunk, and you'll be able to bump up the 
MINA version to 2.0.0-RC2-SNAPSHOT in ADS.

Not sure though that it's really your problem.
I've got an issue with an embedded instance of apacheds in a trivial 
piece of java code. For reasons I have been unable to diagnose
so far, it will run happily for about a week or so, then it will suddenly 
ramp up the load on the server it's running on, with the only
way to recover it, being to delete the physical directory where the
records are stored and start from scratch. What's puzzling me is
that it's definitely not load related as the queries an operations to 
LDAP are very lightweight and infrequent. What I do get in the logs
are loads of these


pool-4-thread-18 ERROR registries.DefaultAttributeTypeRegistry -
attributeType w/ OID 1.3.6.1.4.1.1466.115.121.1.12 not registered!
pool-4-thread-19 ERROR registries.DefaultAttributeTypeRegistry -
attributeType w/ OID 1.3.6.1.4.1.1466.115.121.1.12 not registered!
pool-4-thread-19 WARN context.SearchingOperationContext - Requested 
attribute dn does not exist in the schema, it will be ignored
pool-4-thread-18 WARN context.SearchingOperationContext - Requested 
attribute dn does not exist in the schema, it will be ignored
NioProcessor-2 WARN ldap.LdapProtocolHandler - Null LdapSession given to 
cleanUpSession.
pool-4-thread-19 ERROR registries.DefaultAttributeTypeRegistry -
attributeType w/ OID 1.3.6.1.4.1.1466.115.121.1.12 not registered!
pool-4-thread-17 ERROR registries.DefaultAttributeTypeRegistry -
attributeType w/ OID 1.3.6.1.4.1.1466.115.121.1.12 not registered!


and
pool-4-thread-19 WARN ldap.LdapProtocolHandler - Unexpected exception 
forcing session to close: sending disconnect notice to client.
java.lang.NullPointerException
     at
org .apache .directory .server .ldap .handlers .LdapRequestHandler.handleMessage(LdapRequestHandler.java:129) 
     at
org .apache .directory .server .ldap .handlers .LdapRequestHandler.handleMessage(LdapRequestHandler.java:56) 
     at
org .apache .mina .handler .demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java: 232) 
     at
org .apache .directory .server .ldap .LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:194) 
     at
org.apache.mina.core.filterchain.DefaultIoFilterChain $TailFilter.messageReceived(DefaultIoFilterChain.java:721) 
     at
org .apache .mina .core .filterchain .DefaultIoFilterChain .callNextMessageReceived(DefaultIoFilterChain.java:433) 
     at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access $1200(DefaultIoFilterChain.java:47) 
     at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl $1.messageReceived(DefaultIoFilterChain.java:801) 
     at
org .apache .mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:71) 
     at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
     at
org.apache.mina.filter.executor.UnorderedThreadPoolExecutor $Worker.runTask(UnorderedThreadPoolExecutor.java:480) 
     at
org.apache.mina.filter.executor.UnorderedThreadPoolExecutor $Worker.run(UnorderedThreadPoolExecutor.java:434) 
     at java.lang.Thread.run(Thread.java:619)
pool-4-thread-19 WARN ldap.LdapProtocolHandler - Null LdapSession given 
to cleanUpSession.
NioProcessor-3 WARN ldap.LdapProtocolHandler - Null LdapSession given to 
cleanUpSession.


The only thing I can think of causing the NPE's is a monitoring
application that we have, that connects and disconnects from ADS to confirm that it's still there...Apart from that I have no plausible explanation 
for the high load. I'm currently using 1.5.6 compiled from source.

Thanks,

Yiannis






______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System. 
For more information please visit 
http://www.messagelabs.com/email______________________________________________________________________







--
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________

Reply via email to