On Tue, Jul 17, 2012 at 2:01 PM, Rob Crittenden <rcrit...@redhat.com> wrote: > Stephen Ingram wrote: >> >> On Mon, Jul 16, 2012 at 12:23 PM, Rob Crittenden <rcrit...@redhat.com> >> wrote: >>> >>> Stephen Ingram wrote: >>>> >>>> >>>> On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson <rmegg...@redhat.com> >>>> wrote: >>>>> >>>>> >>>>> On 07/16/2012 11:48 AM, Stephen Ingram wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson<rmegg...@redhat.com> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 07/16/2012 10:19 AM, Stephen Ingram wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden<rcrit...@redhat.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Stephen Ingram wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jul 12, 2012 at 2:59 PM, Steven >>>>>>>>>> Jones<steven.jo...@vuw.ac.nz> >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I had huge memory issues pre 6.3, now its low and flat....Sounds >>>>>>>>>>> like >>>>>>>>>>> you >>>>>>>>>>> have an issue somewhere. My normal cpu use is a few hundred >>>>>>>>>>> mhz....but >>>>>>>>>>> when >>>>>>>>>>> "something" goes wrong such as replication failing that >>>>>>>>>>> climbs...ditto >>>>>>>>>>> memory use.... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, I saw your conversation with Rich on this list about that. >>>>>>>>>> And, >>>>>>>>>> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is >>>>>>>>>> still >>>>>>>>>> having issues. It was an upgrade from 2.1.3, but the upgrade >>>>>>>>>> seemed >>>>>>>>>> to >>>>>>>>>> complete without issue. I'm also not even doing replication yet so >>>>>>>>>> I'm >>>>>>>>>> not sure why memory is so high. Web interface is much slower too >>>>>>>>>> so >>>>>>>>>> perhaps something else is wrong. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Can you tell where it is being slow? Does it seem related to >>>>>>>>> retrieving >>>>>>>>> data >>>>>>>>> from LDAP? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I'm not really sure yet what is causing the slowness. I have the >>>>>>>> same >>>>>>>> number of directory entries as before the upgrade. It was very quick >>>>>>>> with 2.1.3, but once I upgraded, I felt like I was back to the >>>>>>>> pre-2.0 >>>>>>>> days--without a doubt much, much slower. >>>>>>>> >>>>>>>>> You might check your 389-ds access logs and look for searches with >>>>>>>>> notes=U. >>>>>>>>> Perhaps you are missing an index. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Yes there are lots of notes=U. What does this mean? Was something >>>>>>>> missed in the upgrade script? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Try running logconv.pl >>>>>> >>>>>> >>>>>> >>>>>> Nice! I'm guessing that notes=U are unindexed searches then. I have 34 >>>>>> over the last 24 hours so I'm not sure this would be causing the issue >>>>>> as the slowness persists through every click. >>>>> >>>>> >>>>> >>>>> Yeah, I would expect to see a lot more than 34 if that were the cause. >>>>> >>>>> Can you post the search filters that are unindexed? >>>> >>>> >>>> >>>> Sure, here's a partial list (sanitized): >>>> >>>> >>>> >>>> filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com) >>>> attrs="fqdn" >>>> >>>> >>>> filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)" >>>> attrs="fqdn" >>>> >>>> >>>> filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)" >>>> attrs="fqdn" >>>> >>>> >>>> filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)" >>>> attrs="fqdn" >>>> >>>> >>>> filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)" >>>> attrs="fqdn" >>>> >>>> All the rest are the same, just with different hosts. >>>> >>>>>> I've traced the >>>>>> unindexed searches back to the time of Web UI access and they don't >>>>>> match. I also don't see any other obvious errors when running >>>>>> logconv.pl. >>>>>> >>>>>> One strange thing I have noticed is that the 389 server logs seem to >>>>>> update in "spurts". If I'm tailing the logs while I access a Web UI >>>>>> page, there is nothing, then a couple of seconds later, I see the logs >>>>>> quickly scroll with new entires. Has this always been the case? I >>>>>> don't seem to remember this before. >>>>> >>>>> >>>>> >>>>> Yes. The 389 access log is buffered, for performance reasons. >>>> >>>> >>>> >>>> Just thought it might be relevant. I'm not sure what is causing the >>>> extreme slowness. I've also shut off memcached and tried without it >>>> with no discernible difference. The directory seems to be handling the >>>> load of external queries just fine, although I'm not sure I've solved >>>> the memory issue--I'm still testing with the compat plugin disabled to >>>> see if I can stop the memory creep. Maybe it's something in the code >>>> of the Web UI itself as its even slow when changing from page to page >>>> of users and hosts. >>> >>> >>> >>> Shutting of memcached is just going to make things even slower. >> >> >> I really didn't see much difference so I turned it back on right away. >> >>> Things you can try on a quiet system: >>> >>> 1. Create /etc/ipa/server.conf: >>> >>> [global] >>> debug=True >>> >>> Restart Apache >>> >>> Use the UI to do a few things. Verify in the logs that the session cache >>> is >>> being used. >> >> >> Yes, it is. It's interesting, 2.2 is slower such that you can see the >> frame load, and then the loading symbol spins below in the display >> area for a few seconds while that loads up. Before, with 2.1.3, you >> really couldn't distinguish the two as they loaded so quickly. > > > A lot of what gets loaded a just big javascript files. I wonder if there is > a DNS problem, that would explain the slowness. The javascript and much of > the UI is completely unprotected by anything.
Hmmm. DNS issues? What sort of things would I be looking for? ipa1.example.com resolves both ways both from IPA and outside nameserver that doesn't connect at all with IPA. Would there be anything else? Could a DNS conflict within the DNS portion of IPA cause an issue? >>> 2. Check your browser configuration. You don't need delegation-uris set >>> any >>> more. Having it set might mask other problems (you still need >>> negotiation-auth.trusted-uris set). >> >> >> I forgot about this. I changed it, completely cleared the browser >> cache and accessed without any noticeable difference. >> >>> 3. Watch what URI is being used in the Apache access log. It should be >>> /ipa/session/json. >> >> >> Check. this is where it lands. >> >> I'm beginning to think this is just the Web UI itself instead of 389 >> although it is really difficult to tell. I've poured over the debug >> logs and didn't see anything that caused me concern. >> >> It's certainly usable, but I just got really spoiled by the >> unbelievable quickness of 2.1.3. When your release notes indicate it >> should be faster, what are you comparing it to? Maybe this only >> happens with upgraded instances and not fresh installs. > > > It is always possible something didn't get upgraded properly but I've done > 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is > faster we're always referring to the previous version (or versions). Maybe I was just lucky with 2.1.3. On a first load it might take some time to load the "frame" as I call it. But the data would load almost instantaneously from there (certainly no more than 1 s) as you moved from page to page. Here, even if I return to the same page, the system acts as if the data is begin fetched for the very first time as it is no faster than the first load. Maybe that is significant to the problem? > Since sessions are being used the only bottleneck should be 389-ds and our > massaging of that data. The only thing that should be slow is the retrieval > of actual data. The pages themselves should return and be rendered fairly > quickly and the data should follow shortly. If everything is slower then it > is something else (network, DNS, browser, etc). Perhaps the problem is in another package that came along for the ride with the upgrade to 6.3. As the memory requirements are substantially higher, maybe there were some adjustments elsewhere (new kadmind, httpd). I'll start hunting through other logs as well to see what I can find. > You might try creating a new browser profile to rule things out. I tried this with no success. Steve _______________________________________________ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
