Re: [Freeipa-users] IPA Load Problems?

2013-09-11 Thread Martin Kosek
On 09/04/2013 04:02 PM, Rich Megginson wrote: > On 09/04/2013 07:58 AM, John Moyer wrote: >> It was our opinion that it wasn't an index issue. I cleared the logs from >> the IPA server, and then just ran a JIRA sync with the server. I gave Rich >> the log file from my IPA for that sync. I can't

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
It was our opinion that it wasn't an index issue. I cleared the logs from the IPA server, and then just ran a JIRA sync with the server. I gave Rich the log file from my IPA for that sync. I can't find the exact conversation, but we determined that JIRA was connecting to LDAP some 1000 times

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Rich Megginson
On 09/04/2013 07:51 AM, Martin Kosek wrote: Ah, ok. One of the reasons why I was poking to this thread is exactly this ticket. It does not contain much information _what exactly_ is making IPA performance poor - whether it is missing indices (which ones?) or some issue in IPA plugins during binds

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Rich Megginson
On 09/04/2013 07:58 AM, John Moyer wrote: It was our opinion that it wasn't an index issue. I cleared the logs from the IPA server, and then just ran a JIRA sync with the server. I gave Rich the log file from my IPA for that sync. I can't find the exact conversation, but we determined that J

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Martin Kosek
Ah, ok. One of the reasons why I was poking to this thread is exactly this ticket. It does not contain much information _what exactly_ is making IPA performance poor - whether it is missing indices (which ones?) or some issue in IPA plugins during binds, etc. Without more information, we do not kn

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Alexander Bokovoy
On Wed, 04 Sep 2013, Dmitri Pal wrote: On 09/04/2013 08:01 AM, John Moyer wrote: Martin, I apologize there was a large offline conversation between Rich and myself. Rich was kind enough to help me through some of my issues. We did a lot more tests and poking and prodding. We discovered tha

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
Sure, just let me know what needs to be run/applied. I've already rolled back to LDAP, so if the fix looks like it works I can then roll it out again. Thanks, _ John Moyer Director, IT Operations On Sep 4, 2013, at 9:12 AM, Dmitri Pal wrote:

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Dmitri Pal
On 09/04/2013 08:53 AM, John Moyer wrote: > That summary is correct. The only thing I would add is that other > applications could easily bring the IPA server to it's knees as well. Yes this is what I meant. It is not only JIRA. Any client that creates a lot of connections can cause problems.

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
That summary is correct. The only thing I would add is that other applications could easily bring the IPA server to it's knees as well. Our artifact server also did many connections per sec when used, and one person doing a build could bring IPA to it's knees as well. Also, not only would I

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Dmitri Pal
On 09/04/2013 08:01 AM, John Moyer wrote: > Martin, > > I apologize there was a large offline conversation between Rich and > myself. Rich was kind enough to help me through some of my issues. > We did a lot more tests and poking and prodding. We discovered that > IPA is not as efficient when

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread John Moyer
Martin, I apologize there was a large offline conversation between Rich and myself. Rich was kind enough to help me through some of my issues. We did a lot more tests and poking and prodding. We discovered that IPA is not as efficient when dealing with large number of connections.

Re: [Freeipa-users] IPA Load Problems?

2013-09-04 Thread Martin Kosek
On 08/30/2013 11:08 PM, John Moyer wrote: > Well IPA has machine entries on some test clusters that I'm rolling IPA > out on (20 machines maybe) but the user base is the same (about 80 ~ 100) > accounts with maybe 40 to 50 groups? > > I've stood up a clone of the jira server along with IPA. I cl

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread Rich Megginson
On 08/30/2013 01:31 PM, John Moyer wrote: Rob or anyone else, So while struggling along on this server I just grabbed the logs off it and ran that log program with the options you suggested. There are a lot of unindexed requests. These are the top issues I've removed the one username that

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread John Moyer
I'm sorry that was my top unique filter list not my unindexed list. Please disregard my last email. Thanks, _ John Moyer Director, IT Operations Digital Reasoning Systems, Inc. john.mo...@digitalreasoning.com Office: 703.678.2311 Mobile: 240

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread John Moyer
If objectclass eq is already indexed how are these on my top unindexed list? Wouldn't objectclass eq cover this (objectclass=inetorgperson)? and the third and fourth entry? I apologize if I'm way off as I am new to the intricacies of LDAP indexing. Thanks, __

Re: [Freeipa-users] IPA Load Problems?

2013-08-30 Thread John Moyer
Rob or anyone else, So while struggling along on this server I just grabbed the logs off it and ran that log program with the options you suggested. There are a lot of unindexed requests. These are the top issues I've removed the one username that showed up. So just to double check wh

Re: [Freeipa-users] IPA Load Problems?

2013-08-28 Thread Rob Crittenden
John Moyer wrote: So this method of search logs is great, and it shows some indexes that would likely highly increase efficiency with my usage. So, are there instructions how to do that? or do you know off hand how to do that? I'd start with https://access.redhat.com/site/documentation/en

Re: [Freeipa-users] IPA Load Problems?

2013-08-28 Thread John Moyer
So this method of search logs is great, and it shows some indexes that would likely highly increase efficiency with my usage. So, are there instructions how to do that? or do you know off hand how to do that? Thanks, _ John Moyer Directo

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread Rob Crittenden
John Moyer wrote: Wow, this is quite insightful, this is the output from that, it looks like there aren't many unindexed searches (319 doesn't seem like a lot to me at least). Do you have any suggestions from this output? There are a slew of options you can provide to logconv.pl. I typically

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
Wow, this is quite insightful, this is the output from that, it looks like there aren't many unindexed searches (319 doesn't seem like a lot to me at least). Do you have any suggestions from this output? Start of Log:27/Aug/2013:02:36:08 End of Log: 27/Aug/2013:12:17:15 Processed Lo

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread Rob Crittenden
John Moyer wrote: Is there any way to see what fields are index'ed? $ ldapsearch -LLL -D 'cn=directory manager' -W -x -b 'cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config' Your best bet is to use the logconv.pl tool to examine your logs. rob Thanks,

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
Is there any way to see what fields are index'ed? Thanks, _ John Moyer Director, IT Operations Digital Reasoning Systems, Inc. john.mo...@digitalreasoning.com Office: 703.678.2311 Mobile: 240.460.0023 Fax:703.678.2312 www.digitalre

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
That looks like the output I just got shown below: dn: cn=mapping tree,cn=config dn: cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config dn: cn=meToipa2.example.com,cn=replica,cn=dc\3Dexample\ 2Cdc\3Dcom,cn=mapping tree,cn=

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread Rob Crittenden
John Moyer wrote: Ok, so we tried to implement this again, and as soon as we put on a server that authenticates heavily the IPA came to it's knees again. This time I was able to watch it closely and try to troubleshoot a lot more, and also know exactly what server caused it (Mercurial with help o

Re: [Freeipa-users] IPA Load Problems?

2013-08-27 Thread John Moyer
Ok, so we tried to implement this again, and as soon as we put on a server that authenticates heavily the IPA came to it's knees again. This time I was able to watch it closely and try to troubleshoot a lot more, and also know exactly what server caused it (Mercurial with help of bamboo). Th

Re: [Freeipa-users] IPA Load Problems?

2013-08-06 Thread Rob Crittenden
John Moyer wrote: Hello, So I've been preparing my infrastructure for a big change from an older openldap system to a nice new IPA server. I have a redundant secondary server and snapshots taken daily. I populated all my user data into IPA, and gave the users a week to set a password. They a

Re: [Freeipa-users] IPA Load Problems?

2013-08-06 Thread Rich Megginson
On 08/05/2013 09:17 PM, John Moyer wrote: Hello, So I've been preparing my infrastructure for a big change from an older openldap system to a nice new IPA server. I have a redundant secondary server and snapshots taken daily. I populated all my user data into IPA, and gave the users a week