Re: [Freeipa-users] IPA Load Problems?

2013-09-11 Thread Martin Kosek
On 09/04/2013 04:02 PM, Rich Megginson wrote: On 09/04/2013 07:58 AM, John Moyer wrote: It was our opinion that it wasn't an index issue. I cleared the logs from the IPA server, and then just ran a JIRA sync with the server. I gave Rich the log file from my IPA for that sync. I can't find

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Dmitri Pal
On 09/04/2013 08:01 AM, John Moyer wrote: Martin, I apologize there was a large offline conversation between Rich and myself. Rich was kind enough to help me through some of my issues. We did a lot more tests and poking and prodding. We discovered that IPA is not as efficient when

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
That summary is correct. The only thing I would add is that other applications could easily bring the IPA server to it's knees as well. Our artifact server also did many connections per sec when used, and one person doing a build could bring IPA to it's knees as well. Also, not only would

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Dmitri Pal
On 09/04/2013 08:53 AM, John Moyer wrote: That summary is correct. The only thing I would add is that other applications could easily bring the IPA server to it's knees as well. Yes this is what I meant. It is not only JIRA. Any client that creates a lot of connections can cause problems.

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
Sure, just let me know what needs to be run/applied. I've already rolled back to LDAP, so if the fix looks like it works I can then roll it out again. Thanks, _ John Moyer Director, IT Operations On Sep 4, 2013, at 9:12 AM, Dmitri Pal

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Alexander Bokovoy
On Wed, 04 Sep 2013, Dmitri Pal wrote: On 09/04/2013 08:01 AM, John Moyer wrote: Martin, I apologize there was a large offline conversation between Rich and myself. Rich was kind enough to help me through some of my issues. We did a lot more tests and poking and prodding. We discovered

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Martin Kosek
Ah, ok. One of the reasons why I was poking to this thread is exactly this ticket. It does not contain much information _what exactly_ is making IPA performance poor - whether it is missing indices (which ones?) or some issue in IPA plugins during binds, etc. Without more information, we do not

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Rich Megginson
On 09/04/2013 07:51 AM, Martin Kosek wrote: Ah, ok. One of the reasons why I was poking to this thread is exactly this ticket. It does not contain much information _what exactly_ is making IPA performance poor - whether it is missing indices (which ones?) or some issue in IPA plugins during

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
It was our opinion that it wasn't an index issue. I cleared the logs from the IPA server, and then just ran a JIRA sync with the server. I gave Rich the log file from my IPA for that sync. I can't find the exact conversation, but we determined that JIRA was connecting to LDAP some 1000 times

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Rich Megginson
On 09/04/2013 07:58 AM, John Moyer wrote: It was our opinion that it wasn't an index issue. I cleared the logs from the IPA server, and then just ran a JIRA sync with the server. I gave Rich the log file from my IPA for that sync. I can't find the exact conversation, but we determined that

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread John Moyer
Rob or anyone else, So while struggling along on this server I just grabbed the logs off it and ran that log program with the options you suggested. There are a lot of unindexed requests. These are the top issues I've removed the one username that showed up. So just to double check

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread John Moyer
If objectclass eq is already indexed how are these on my top unindexed list? Wouldn't objectclass eq cover this (objectclass=inetorgperson)? and the third and fourth entry? I apologize if I'm way off as I am new to the intricacies of LDAP indexing. Thanks,

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread Rich Megginson
On 08/30/2013 01:31 PM, John Moyer wrote: Rob or anyone else, So while struggling along on this server I just grabbed the logs off it and ran that log program with the options you suggested. There are a lot of unindexed requests. These are the top issues I've removed the one username

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread John Moyer
I'm sorry that was my top unique filter list not my unindexed list. Please disregard my last email. Thanks, _ John Moyer Director, IT Operations Digital Reasoning Systems, Inc. john.mo...@digitalreasoning.com Office: 703.678.2311 Mobile:

Re: [Freeipa-users] IPA Load Problems?

2013-08-28 Thread John Moyer
So this method of search logs is great, and it shows some indexes that would likely highly increase efficiency with my usage. So, are there instructions how to do that? or do you know off hand how to do that? Thanks, _ John Moyer

Re: [Freeipa-users] IPA Load Problems?

2013-08-28 Thread Rob Crittenden
John Moyer wrote: So this method of search logs is great, and it shows some indexes that would likely highly increase efficiency with my usage. So, are there instructions how to do that? or do you know off hand how to do that? I'd start with

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
Ok, so we tried to implement this again, and as soon as we put on a server that authenticates heavily the IPA came to it's knees again. This time I was able to watch it closely and try to troubleshoot a lot more, and also know exactly what server caused it (Mercurial with help of bamboo).

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread Rob Crittenden
John Moyer wrote: Ok, so we tried to implement this again, and as soon as we put on a server that authenticates heavily the IPA came to it's knees again. This time I was able to watch it closely and try to troubleshoot a lot more, and also know exactly what server caused it (Mercurial with help

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
That looks like the output I just got shown below: dn: cn=mapping tree,cn=config dn: cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config dn: cn=meToipa2.example.com,cn=replica,cn=dc\3Dexample\ 2Cdc\3Dcom,cn=mapping

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
Is there any way to see what fields are index'ed? Thanks, _ John Moyer Director, IT Operations Digital Reasoning Systems, Inc. john.mo...@digitalreasoning.com Office: 703.678.2311 Mobile: 240.460.0023 Fax:703.678.2312

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread Rob Crittenden
John Moyer wrote: Is there any way to see what fields are index'ed? $ ldapsearch -LLL -D 'cn=directory manager' -W -x -b 'cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config' Your best bet is to use the logconv.pl tool to examine your logs. rob Thanks,

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
Wow, this is quite insightful, this is the output from that, it looks like there aren't many unindexed searches (319 doesn't seem like a lot to me at least). Do you have any suggestions from this output? Start of Log:27/Aug/2013:02:36:08 End of Log: 27/Aug/2013:12:17:15 Processed

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread Rob Crittenden
John Moyer wrote: Wow, this is quite insightful, this is the output from that, it looks like there aren't many unindexed searches (319 doesn't seem like a lot to me at least). Do you have any suggestions from this output? There are a slew of options you can provide to logconv.pl. I

Re: [Freeipa-users] IPA Load Problems?

2013-08-06 Thread Rich Megginson
On 08/05/2013 09:17 PM, John Moyer wrote: Hello, So I've been preparing my infrastructure for a big change from an older openldap system to a nice new IPA server. I have a redundant secondary server and snapshots taken daily. I populated all my user data into IPA, and gave the users a

Re: [Freeipa-users] IPA Load Problems?

2013-08-06 Thread Rob Crittenden
John Moyer wrote: Hello, So I've been preparing my infrastructure for a big change from an older openldap system to a nice new IPA server. I have a redundant secondary server and snapshots taken daily. I populated all my user data into IPA, and gave the users a week to set a password. They