Re: [389-users] 389 pauses every 5 minutes under load
On 10/12/2011 05:41 PM, Rich Megginson wrote: Thanks. The select/poll are the server basically idle, waiting on a condition variable for new work to perform. You can eliminate many of these by decreasing your cn=config nsslapd-threadnum setting. The default is 30, but you may find better performance by setting to somewhere around 2 times the number of cpus/cores you have on your machine (but at least 8). Yeah, I knew what the select/poll calls are, but I will look at decreasing the number of threads to a more suitable number. Are these threads bound to a specific connection or is it a thread pool that gets requests delegated to it from the open connections? Do you know if any of these come from a period of time during which the server is consuming a lot of CPU? Yes these were taken starting ~1 second before the spike and going ~2 seconds after. It's easy to start monitoring right before the event when the leading edge to leading edge time is exactly 5 minutes! No, should not be a problem. And it is standard - many apps do this (e.g. a web service that uses ldap for auth will not want to open/close a connection for every single user - it will typically use a connection pool of already open and possibly idle connections). Our app doesn't really use a pool in the traditional sense... just one connection bound as a given user for that user's session... but I'm glad to hear that it shouldn't be the problem. Thanks, Justin -- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Re: [389-users] 389 pauses every 5 minutes under load
- Original Message - On 10/12/2011 05:41 PM, Rich Megginson wrote: Thanks. The select/poll are the server basically idle, waiting on a condition variable for new work to perform. You can eliminate many of these by decreasing your cn=config nsslapd-threadnum setting. The default is 30, but you may find better performance by setting to somewhere around 2 times the number of cpus/cores you have on your machine (but at least 8). Yeah, I knew what the select/poll calls are, but I will look at decreasing the number of threads to a more suitable number. Are these threads bound to a specific connection or is it a thread pool that gets requests delegated to it from the open connections? It is essentially a thread pool, although it has a bit more logic than that. Do you know if any of these come from a period of time during which the server is consuming a lot of CPU? Yes these were taken starting ~1 second before the spike and going ~2 seconds after. It's easy to start monitoring right before the event when the leading edge to leading edge time is exactly 5 minutes! Ok, thanks. I am investigating. No, should not be a problem. And it is standard - many apps do this (e.g. a web service that uses ldap for auth will not want to open/close a connection for every single user - it will typically use a connection pool of already open and possibly idle connections). Our app doesn't really use a pool in the traditional sense... just one connection bound as a given user for that user's session... but I'm glad to hear that it shouldn't be the problem. Thanks, Justin -- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
Re: [389-users] 389 pauses every 5 minutes under load
On 10/07/2011 11:56 AM, Justin Gronfur wrote: Hello all, I need your expertise... please help me! (Disclaimer: I am a relative newcomer to 389ds) I'm running a Java application that keeps user authentication, permissions, and preferences in ldap. And I'm currently load testing this application (using Jmeter, 15 concurrent threads, no think time) and I'm getting really good performance most of the time. However every 5 minutes (from the time I started ldap), 389's CPU usage will spike to 375% (400% = all 4 processors at 100%, 389 normally sits around 15-20%). These pauses last for between 20 - 30 seconds (proportionate to the load I'm throwing at it) during which our application will just sit. Since I'm just running the same set of requests at it constantly, there isn't anything different in terms of our application during those times, which points to 389 as the culprit (or possibly some glassfish ldap pool problem). Some info: Glassfish 3.1 final on Java 1.6.0_26 (64 bit server VM) 389-Directory/1.2.9.10 B2011.250.1455 Fedora 15 64-bit (also observed on Centos 5.4 64-bit) Have any of you run into this problem? Do you have any possible config changes I could try? Any possible leads at all? Are you using replication? Is this a replication master? Is your load tester doing delete operations? Thanks, Justin -- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users -- 389 users mailing list 389-users@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/389-users
