Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-19 Thread Dmitri Pal
On 07/19/2012 02:29 AM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 2:26 PM, Dmitri Pal  wrote:
>> On 07/18/2012 05:09 PM, Stephen Ingram wrote:
>>> On Wed, Jul 18, 2012 at 1:52 PM, Dmitri Pal  wrote:
 On 07/18/2012 04:27 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 1:06 PM, Dmitri Pal  wrote:
>> On 07/18/2012 03:45 PM, Stephen Ingram wrote:
>>> On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  
>>> wrote:
 On 07/18/2012 02:59 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
> wrote:
>> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>>
>> 8><--
>>
>>
> I'm beginning to think this is just the Web UI itself instead of 
> 389
> although it is really difficult to tell. I've poured over the 
> debug
> logs and didn't see anything that caused me concern.
>
> It's certainly usable, but I just got really spoiled by the
> unbelievable quickness of 2.1.3. When your release notes indicate 
> it
> should be faster, what are you comparing it to? Maybe this only
> happens with upgraded instances and not fresh installs.
 It is always possible something didn't get upgraded properly but 
 I've
 done
 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say 
 something is
 faster we're always referring to the previous version (or 
 versions).
>>> Maybe I was just lucky with 2.1.3. On a first load it might take 
>>> some
>>> time to load the "frame" as I call it. But the data would load 
>>> almost
>>> instantaneously from there (certainly no more than 1 s) as you moved
>>> from page to page. Here, even if I return to the same page, the 
>>> system
>>> acts as if the data is begin fetched for the very first time as it 
>>> is
>>> no faster than the first load. Maybe that is significant to the
>>> problem?
>> I think the culprit is Web UI paging capabilities introduced in 2.2. 
>> With
>> lot of users, responses might grow in size. You can check their size 
>> and
>> duration in browser developers tools. I suggest chrome/chromium - 
>> press
>> F12
>> and choose 'network' tab.
>>
>> This new feature can't be disabled in configuration. To test if the
>> slowdown
>> is done by paging you can (at own risk) replace line
>> /usr/share/ipa/ui/facet.js:538
>>
>> that.pagination = spec.pagination === undefined ? true : 
>> spec.pagination;
>>
>> with:
>>
>> that.pagination = false;
>>
>> Note: It will break some other parts of the UI - so for testing only.
> I've made the substitution in the code (was line 507 for me-do I have
> a different version?). Looking at the time chart in Chrome I see that
> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
> waiting for the directory server or memcached?
 Actually neither, it means waiting for a response from the web server
 (technically it's making an RPC call via HTTP Ajax). The RPC call 
 needs to
 go through the web server, memcached, and typically will invoke one or 
 more
 directory server queries, and run a bunch of Python to massage 
 everything
 before the RPC returns with the result.

 It doesn't look like you've got much difference in times between with
 pagination on and pagination off. I don't know the pagination code but 
 I
 suspect it's run after the RPC call returns so the RPC timing is not 
 telling
 us much with respect to that.

 Waiting for up to 3 seconds for an RPC call does seem on the high 
 side. Do
 you have a lot of LDAP data?
>>> No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
>>> any significant amount of hosts in it.
>>>
 But really, unless we get timing results for each component we're 
 grasping
 at straws :-(
>>> Understood.
>>>
>>> Steve
>> Do you have a replica and does this replica behave the same?
> No replica yet. I wanted to get the memory leak issue solved first.
> All I have to compare to is the old 2.1.3 before. This one is much
> slower. I just can't seem to figure out what's wrong. The upgrade
> seemed to complete successfully and there were no errors in the log.
> The only things I've found thus far (earlier in this thread) are the
> unindexed entries (all hosts entries) that Rich seemed to think might
> be slowing things up. As the slowness is on every

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-19 Thread Alexander Bokovoy

On Thu, 19 Jul 2012, John Dennis wrote:
Rob may have already contacted you with this, but if not we would 
like to get more debugging information by have the server log what is 
occurring when it processes your requests.


To do this you'll need to turn on the debug flag in the IPA 
configuration file /etc/ipa/default.conf, add a line that says:


debug = True

Then restart the server to pick up the new configuration. The 
information will be written to /var/log/httpd/error_log.


We only need the contents of the log from when the server was 
restarted with debug logging enabled. For privacy reasons I suggest 
you send the contents of the log to one of the IPA team members 
directly in a private email, not to the public freeipa list.

In addition we would like to see what's happening with krb5
communication under httpd processes.

In order to obtain that tracing information you need to do following:

1. Add KRB5_TRACE=/tmp/http_krb5_trace.log to /etc/sysconfig/httpd

2. Restart httpd (or httpd.service in Fedora)

3. Now you need to create the file and chown it to apache's user so that
   httpd processes would be able to write to it:

 find out PID of any of httpd processes, doesn't matter which one
 touch /proc/$PID/cwd/tmp/http_krb5_trace.log
 chown apache /proc/$PID/cwd/tmp/http_krb5_trace.log

4. Now you can issue IPA commands and you'll get krb5 client tracing in
   /proc/$PID/cwd/tmp/http_krb5_trace.log

The reason why (3) talks about PID of httpd process is because in
Fedora, unlike in RHEL6.x, systemd is handling services startup and
systemd confines httpd to a private /tmp. Using /proc/$PID/cwd/tmp is
the easiest way to reach that private tmp.

5. Once finished and copied /proc/$PID/cwd/tmp/http_krb5_trace.log to an
archive location, make sure to remove the file and its reference from
/etc/sysconfig/httpd and restart the service.

--
/ Alexander Bokovoy

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-19 Thread John Dennis
Rob may have already contacted you with this, but if not we would like 
to get more debugging information by have the server log what is 
occurring when it processes your requests.


To do this you'll need to turn on the debug flag in the IPA 
configuration file /etc/ipa/default.conf, add a line that says:


debug = True

Then restart the server to pick up the new configuration. The 
information will be written to /var/log/httpd/error_log.


We only need the contents of the log from when the server was restarted 
with debug logging enabled. For privacy reasons I suggest you send the 
contents of the log to one of the IPA team members directly in a private 
email, not to the public freeipa list.


Thanks!

John


--
John Dennis 

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-19 Thread Petr Vobornik

On 07/18/2012 08:59 PM, Stephen Ingram wrote:

On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik  wrote:

On 07/17/2012 11:43 PM, Stephen Ingram wrote:

8><--




I'm beginning to think this is just the Web UI itself instead of 389
although it is really difficult to tell. I've poured over the debug
logs and didn't see anything that caused me concern.

It's certainly usable, but I just got really spoiled by the
unbelievable quickness of 2.1.3. When your release notes indicate it
should be faster, what are you comparing it to? Maybe this only
happens with upgraded instances and not fresh installs.




It is always possible something didn't get upgraded properly but I've
done
2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
faster we're always referring to the previous version (or versions).



Maybe I was just lucky with 2.1.3. On a first load it might take some
time to load the "frame" as I call it. But the data would load almost
instantaneously from there (certainly no more than 1 s) as you moved
from page to page. Here, even if I return to the same page, the system
acts as if the data is begin fetched for the very first time as it is
no faster than the first load. Maybe that is significant to the
problem?



I think the culprit is Web UI paging capabilities introduced in 2.2. With
lot of users, responses might grow in size. You can check their size and
duration in browser developers tools. I suggest chrome/chromium - press F12
and choose 'network' tab.

This new feature can't be disabled in configuration. To test if the slowdown
is done by paging you can (at own risk) replace line
/usr/share/ipa/ui/facet.js:538

that.pagination = spec.pagination === undefined ? true : spec.pagination;

with:

that.pagination = false;

Note: It will break some other parts of the UI - so for testing only.


I've made the substitution in the code (was line 507 for me-do I have
a different version?).


I was looking at the top of FreeIPA 2.2 branch. RHEL version differs a 
bit. It shouldn't matter in this case though.



Looking at the time chart in Chrome I see that
the bulk of the time is for /ipa/session waiting. Would "waiting" mean
waiting for the directory server or memcached?


Basically all the stuff, which is needed for processing of the request. 
The pipeline is something like (I don't want to go into details): httpd 
-> mod_wsgi -> python -> memcache, dir server request  and back.


From what I see I think the problem is not on Web UI side. Most of the 
time is waiting for server response.


I initially thought the problem lies in a large number of users (1000+). 
But in other post you mentioned that it is under 100. Hence this new 
paging feature shouldn't be a big problem.





Here's a portion of the initial load of the Users page:


The first 3 requests are inital load of web UI (not including .js files 
and such) - you can see they are the same in both cases. I don't see a 
login request so session is already established.



json/ipa/session POST 200 Success application/json jquery.js:7365
Script 33.94KB 33.10KB 1.57s 1.47s 96ms (1.37s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 568.09KB 564.36KB 3.92s 2.95s 963ms (2.85s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 556.94KB 553.40KB 3.78s 2.94s 836ms (2.83s waiting)


This one is user load:

json/ipa/session POST 200 Success application/json jquery.js:7365
Script 46.93KB 46.38KB 1.87s 1.71s (1.60s waiting)

Now, with the pagination turned back on:
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 33.94KB 33.10KB 1.58s 1.48s 100ms (1.38s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 568.09KB 564.36KB 4.05s 3.09s 964ms (2.98s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 556.94KB 553.40KB 3.84s 2.99s 855ms (2.88s waiting)


Here you can see a change. With pagination turned on it should be two 
request. One to get primary keys (logins) and second to get users. The 
latter is missing in this list. With a lot of users the first response 
grows. With low number of users it is in fact smaller than with 
pagination turned off.

json/ipa/session POST 200 Success application/json jquery.js:7365
Script 46.93KB 46.38KB 1.52s 1.51s (1.40s waiting)


As I mentioned you are missing one request so it will add aprox. 1.5s. 
The second request is kinda a slowdown from IPA 2.1.4 but the main issue 
is still the long server processing time.




Steve




--
Petr Vobornik


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Wed, Jul 18, 2012 at 2:26 PM, Dmitri Pal  wrote:
> On 07/18/2012 05:09 PM, Stephen Ingram wrote:
>> On Wed, Jul 18, 2012 at 1:52 PM, Dmitri Pal  wrote:
>>> On 07/18/2012 04:27 PM, Stephen Ingram wrote:
 On Wed, Jul 18, 2012 at 1:06 PM, Dmitri Pal  wrote:
> On 07/18/2012 03:45 PM, Stephen Ingram wrote:
>> On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
>>> On 07/18/2012 02:59 PM, Stephen Ingram wrote:
 On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
 wrote:
> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>
> 8><--
>
>
 I'm beginning to think this is just the Web UI itself instead of 
 389
 although it is really difficult to tell. I've poured over the debug
 logs and didn't see anything that caused me concern.

 It's certainly usable, but I just got really spoiled by the
 unbelievable quickness of 2.1.3. When your release notes indicate 
 it
 should be faster, what are you comparing it to? Maybe this only
 happens with upgraded instances and not fresh installs.
>>>
>>> It is always possible something didn't get upgraded properly but 
>>> I've
>>> done
>>> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say 
>>> something is
>>> faster we're always referring to the previous version (or versions).
>> Maybe I was just lucky with 2.1.3. On a first load it might take some
>> time to load the "frame" as I call it. But the data would load almost
>> instantaneously from there (certainly no more than 1 s) as you moved
>> from page to page. Here, even if I return to the same page, the 
>> system
>> acts as if the data is begin fetched for the very first time as it is
>> no faster than the first load. Maybe that is significant to the
>> problem?
> I think the culprit is Web UI paging capabilities introduced in 2.2. 
> With
> lot of users, responses might grow in size. You can check their size 
> and
> duration in browser developers tools. I suggest chrome/chromium - 
> press
> F12
> and choose 'network' tab.
>
> This new feature can't be disabled in configuration. To test if the
> slowdown
> is done by paging you can (at own risk) replace line
> /usr/share/ipa/ui/facet.js:538
>
> that.pagination = spec.pagination === undefined ? true : 
> spec.pagination;
>
> with:
>
> that.pagination = false;
>
> Note: It will break some other parts of the UI - so for testing only.
 I've made the substitution in the code (was line 507 for me-do I have
 a different version?). Looking at the time chart in Chrome I see that
 the bulk of the time is for /ipa/session waiting. Would "waiting" mean
 waiting for the directory server or memcached?
>>> Actually neither, it means waiting for a response from the web server
>>> (technically it's making an RPC call via HTTP Ajax). The RPC call needs 
>>> to
>>> go through the web server, memcached, and typically will invoke one or 
>>> more
>>> directory server queries, and run a bunch of Python to massage 
>>> everything
>>> before the RPC returns with the result.
>>>
>>> It doesn't look like you've got much difference in times between with
>>> pagination on and pagination off. I don't know the pagination code but I
>>> suspect it's run after the RPC call returns so the RPC timing is not 
>>> telling
>>> us much with respect to that.
>>>
>>> Waiting for up to 3 seconds for an RPC call does seem on the high side. 
>>> Do
>>> you have a lot of LDAP data?
>> No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
>> any significant amount of hosts in it.
>>
>>> But really, unless we get timing results for each component we're 
>>> grasping
>>> at straws :-(
>> Understood.
>>
>> Steve
> Do you have a replica and does this replica behave the same?
 No replica yet. I wanted to get the memory leak issue solved first.
 All I have to compare to is the old 2.1.3 before. This one is much
 slower. I just can't seem to figure out what's wrong. The upgrade
 seemed to complete successfully and there were no errors in the log.
 The only things I've found thus far (earlier in this thread) are the
 unindexed entries (all hosts entries) that Rich seemed to think might
 be slowing things up. As the slowness is on every page, I wouldn't
 think that would be the problem.

 I wouldn't have said as much about this were it not for the promised
 faster speed mentioned in the release notes. It's comparable to the
 old 2.0 r

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Dmitri Pal
On 07/18/2012 05:09 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 1:52 PM, Dmitri Pal  wrote:
>> On 07/18/2012 04:27 PM, Stephen Ingram wrote:
>>> On Wed, Jul 18, 2012 at 1:06 PM, Dmitri Pal  wrote:
 On 07/18/2012 03:45 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
>> On 07/18/2012 02:59 PM, Stephen Ingram wrote:
>>> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
>>> wrote:
 On 07/17/2012 11:43 PM, Stephen Ingram wrote:

 8><--


>>> I'm beginning to think this is just the Web UI itself instead of 389
>>> although it is really difficult to tell. I've poured over the debug
>>> logs and didn't see anything that caused me concern.
>>>
>>> It's certainly usable, but I just got really spoiled by the
>>> unbelievable quickness of 2.1.3. When your release notes indicate it
>>> should be faster, what are you comparing it to? Maybe this only
>>> happens with upgraded instances and not fresh installs.
>>
>> It is always possible something didn't get upgraded properly but I've
>> done
>> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something 
>> is
>> faster we're always referring to the previous version (or versions).
> Maybe I was just lucky with 2.1.3. On a first load it might take some
> time to load the "frame" as I call it. But the data would load almost
> instantaneously from there (certainly no more than 1 s) as you moved
> from page to page. Here, even if I return to the same page, the system
> acts as if the data is begin fetched for the very first time as it is
> no faster than the first load. Maybe that is significant to the
> problem?
 I think the culprit is Web UI paging capabilities introduced in 2.2. 
 With
 lot of users, responses might grow in size. You can check their size 
 and
 duration in browser developers tools. I suggest chrome/chromium - press
 F12
 and choose 'network' tab.

 This new feature can't be disabled in configuration. To test if the
 slowdown
 is done by paging you can (at own risk) replace line
 /usr/share/ipa/ui/facet.js:538

 that.pagination = spec.pagination === undefined ? true : 
 spec.pagination;

 with:

 that.pagination = false;

 Note: It will break some other parts of the UI - so for testing only.
>>> I've made the substitution in the code (was line 507 for me-do I have
>>> a different version?). Looking at the time chart in Chrome I see that
>>> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
>>> waiting for the directory server or memcached?
>> Actually neither, it means waiting for a response from the web server
>> (technically it's making an RPC call via HTTP Ajax). The RPC call needs 
>> to
>> go through the web server, memcached, and typically will invoke one or 
>> more
>> directory server queries, and run a bunch of Python to massage everything
>> before the RPC returns with the result.
>>
>> It doesn't look like you've got much difference in times between with
>> pagination on and pagination off. I don't know the pagination code but I
>> suspect it's run after the RPC call returns so the RPC timing is not 
>> telling
>> us much with respect to that.
>>
>> Waiting for up to 3 seconds for an RPC call does seem on the high side. 
>> Do
>> you have a lot of LDAP data?
> No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
> any significant amount of hosts in it.
>
>> But really, unless we get timing results for each component we're 
>> grasping
>> at straws :-(
> Understood.
>
> Steve
>
> ___
> Freeipa-users mailing list
> Freeipa-users@redhat.com
> https://www.redhat.com/mailman/listinfo/freeipa-users
 Do you have a replica and does this replica behave the same?
>>> No replica yet. I wanted to get the memory leak issue solved first.
>>> All I have to compare to is the old 2.1.3 before. This one is much
>>> slower. I just can't seem to figure out what's wrong. The upgrade
>>> seemed to complete successfully and there were no errors in the log.
>>> The only things I've found thus far (earlier in this thread) are the
>>> unindexed entries (all hosts entries) that Rich seemed to think might
>>> be slowing things up. As the slowness is on every page, I wouldn't
>>> think that would be the problem.
>>>
>>> I wouldn't have said as much about this were it not for the promised
>>> faster speed mentioned in the release notes. It's comparable to the
>>> old 2.0 release candidates so I thought it might h

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Wed, Jul 18, 2012 at 1:52 PM, Dmitri Pal  wrote:
> On 07/18/2012 04:27 PM, Stephen Ingram wrote:
>> On Wed, Jul 18, 2012 at 1:06 PM, Dmitri Pal  wrote:
>>> On 07/18/2012 03:45 PM, Stephen Ingram wrote:
 On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
> On 07/18/2012 02:59 PM, Stephen Ingram wrote:
>> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
>> wrote:
>>> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>>>
>>> 8><--
>>>
>>>
>> I'm beginning to think this is just the Web UI itself instead of 389
>> although it is really difficult to tell. I've poured over the debug
>> logs and didn't see anything that caused me concern.
>>
>> It's certainly usable, but I just got really spoiled by the
>> unbelievable quickness of 2.1.3. When your release notes indicate it
>> should be faster, what are you comparing it to? Maybe this only
>> happens with upgraded instances and not fresh installs.
>
>
> It is always possible something didn't get upgraded properly but I've
> done
> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something 
> is
> faster we're always referring to the previous version (or versions).

 Maybe I was just lucky with 2.1.3. On a first load it might take some
 time to load the "frame" as I call it. But the data would load almost
 instantaneously from there (certainly no more than 1 s) as you moved
 from page to page. Here, even if I return to the same page, the system
 acts as if the data is begin fetched for the very first time as it is
 no faster than the first load. Maybe that is significant to the
 problem?
>>>
>>> I think the culprit is Web UI paging capabilities introduced in 2.2. 
>>> With
>>> lot of users, responses might grow in size. You can check their size and
>>> duration in browser developers tools. I suggest chrome/chromium - press
>>> F12
>>> and choose 'network' tab.
>>>
>>> This new feature can't be disabled in configuration. To test if the
>>> slowdown
>>> is done by paging you can (at own risk) replace line
>>> /usr/share/ipa/ui/facet.js:538
>>>
>>> that.pagination = spec.pagination === undefined ? true : 
>>> spec.pagination;
>>>
>>> with:
>>>
>>> that.pagination = false;
>>>
>>> Note: It will break some other parts of the UI - so for testing only.
>> I've made the substitution in the code (was line 507 for me-do I have
>> a different version?). Looking at the time chart in Chrome I see that
>> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
>> waiting for the directory server or memcached?
> Actually neither, it means waiting for a response from the web server
> (technically it's making an RPC call via HTTP Ajax). The RPC call needs to
> go through the web server, memcached, and typically will invoke one or 
> more
> directory server queries, and run a bunch of Python to massage everything
> before the RPC returns with the result.
>
> It doesn't look like you've got much difference in times between with
> pagination on and pagination off. I don't know the pagination code but I
> suspect it's run after the RPC call returns so the RPC timing is not 
> telling
> us much with respect to that.
>
> Waiting for up to 3 seconds for an RPC call does seem on the high side. Do
> you have a lot of LDAP data?
 No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
 any significant amount of hosts in it.

> But really, unless we get timing results for each component we're grasping
> at straws :-(
 Understood.

 Steve

 ___
 Freeipa-users mailing list
 Freeipa-users@redhat.com
 https://www.redhat.com/mailman/listinfo/freeipa-users
>>> Do you have a replica and does this replica behave the same?
>> No replica yet. I wanted to get the memory leak issue solved first.
>> All I have to compare to is the old 2.1.3 before. This one is much
>> slower. I just can't seem to figure out what's wrong. The upgrade
>> seemed to complete successfully and there were no errors in the log.
>> The only things I've found thus far (earlier in this thread) are the
>> unindexed entries (all hosts entries) that Rich seemed to think might
>> be slowing things up. As the slowness is on every page, I wouldn't
>> think that would be the problem.
>>
>> I wouldn't have said as much about this were it not for the promised
>> faster speed mentioned in the release notes. It's comparable to the
>> old 2.0 release candidates so I thought it might have been due to the
>> complexity of the feature additions.
>>
>> Steve
> Is the time correct on this system?

Yes. HW clock is GMT and localtime is Paci

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Dmitri Pal
On 07/18/2012 04:27 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 1:06 PM, Dmitri Pal  wrote:
>> On 07/18/2012 03:45 PM, Stephen Ingram wrote:
>>> On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
 On 07/18/2012 02:59 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
> wrote:
>> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>>
>> 8><--
>>
>>
> I'm beginning to think this is just the Web UI itself instead of 389
> although it is really difficult to tell. I've poured over the debug
> logs and didn't see anything that caused me concern.
>
> It's certainly usable, but I just got really spoiled by the
> unbelievable quickness of 2.1.3. When your release notes indicate it
> should be faster, what are you comparing it to? Maybe this only
> happens with upgraded instances and not fresh installs.


 It is always possible something didn't get upgraded properly but I've
 done
 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
 faster we're always referring to the previous version (or versions).
>>>
>>> Maybe I was just lucky with 2.1.3. On a first load it might take some
>>> time to load the "frame" as I call it. But the data would load almost
>>> instantaneously from there (certainly no more than 1 s) as you moved
>>> from page to page. Here, even if I return to the same page, the system
>>> acts as if the data is begin fetched for the very first time as it is
>>> no faster than the first load. Maybe that is significant to the
>>> problem?
>>
>> I think the culprit is Web UI paging capabilities introduced in 2.2. With
>> lot of users, responses might grow in size. You can check their size and
>> duration in browser developers tools. I suggest chrome/chromium - press
>> F12
>> and choose 'network' tab.
>>
>> This new feature can't be disabled in configuration. To test if the
>> slowdown
>> is done by paging you can (at own risk) replace line
>> /usr/share/ipa/ui/facet.js:538
>>
>> that.pagination = spec.pagination === undefined ? true : spec.pagination;
>>
>> with:
>>
>> that.pagination = false;
>>
>> Note: It will break some other parts of the UI - so for testing only.
> I've made the substitution in the code (was line 507 for me-do I have
> a different version?). Looking at the time chart in Chrome I see that
> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
> waiting for the directory server or memcached?
 Actually neither, it means waiting for a response from the web server
 (technically it's making an RPC call via HTTP Ajax). The RPC call needs to
 go through the web server, memcached, and typically will invoke one or more
 directory server queries, and run a bunch of Python to massage everything
 before the RPC returns with the result.

 It doesn't look like you've got much difference in times between with
 pagination on and pagination off. I don't know the pagination code but I
 suspect it's run after the RPC call returns so the RPC timing is not 
 telling
 us much with respect to that.

 Waiting for up to 3 seconds for an RPC call does seem on the high side. Do
 you have a lot of LDAP data?
>>> No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
>>> any significant amount of hosts in it.
>>>
 But really, unless we get timing results for each component we're grasping
 at straws :-(
>>> Understood.
>>>
>>> Steve
>>>
>>> ___
>>> Freeipa-users mailing list
>>> Freeipa-users@redhat.com
>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>> Do you have a replica and does this replica behave the same?
> No replica yet. I wanted to get the memory leak issue solved first.
> All I have to compare to is the old 2.1.3 before. This one is much
> slower. I just can't seem to figure out what's wrong. The upgrade
> seemed to complete successfully and there were no errors in the log.
> The only things I've found thus far (earlier in this thread) are the
> unindexed entries (all hosts entries) that Rich seemed to think might
> be slowing things up. As the slowness is on every page, I wouldn't
> think that would be the problem.
>
> I wouldn't have said as much about this were it not for the promised
> faster speed mentioned in the release notes. It's comparable to the
> old 2.0 release candidates so I thought it might have been due to the
> complexity of the feature additions.
>
> Steve
Is the time correct on this system?

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freei

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Wed, Jul 18, 2012 at 1:06 PM, Dmitri Pal  wrote:
> On 07/18/2012 03:45 PM, Stephen Ingram wrote:
>> On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
>>> On 07/18/2012 02:59 PM, Stephen Ingram wrote:
 On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
 wrote:
> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>
> 8><--
>
>
 I'm beginning to think this is just the Web UI itself instead of 389
 although it is really difficult to tell. I've poured over the debug
 logs and didn't see anything that caused me concern.

 It's certainly usable, but I just got really spoiled by the
 unbelievable quickness of 2.1.3. When your release notes indicate it
 should be faster, what are you comparing it to? Maybe this only
 happens with upgraded instances and not fresh installs.
>>>
>>>
>>>
>>> It is always possible something didn't get upgraded properly but I've
>>> done
>>> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
>>> faster we're always referring to the previous version (or versions).
>>
>>
>> Maybe I was just lucky with 2.1.3. On a first load it might take some
>> time to load the "frame" as I call it. But the data would load almost
>> instantaneously from there (certainly no more than 1 s) as you moved
>> from page to page. Here, even if I return to the same page, the system
>> acts as if the data is begin fetched for the very first time as it is
>> no faster than the first load. Maybe that is significant to the
>> problem?
>
>
> I think the culprit is Web UI paging capabilities introduced in 2.2. With
> lot of users, responses might grow in size. You can check their size and
> duration in browser developers tools. I suggest chrome/chromium - press
> F12
> and choose 'network' tab.
>
> This new feature can't be disabled in configuration. To test if the
> slowdown
> is done by paging you can (at own risk) replace line
> /usr/share/ipa/ui/facet.js:538
>
> that.pagination = spec.pagination === undefined ? true : spec.pagination;
>
> with:
>
> that.pagination = false;
>
> Note: It will break some other parts of the UI - so for testing only.

 I've made the substitution in the code (was line 507 for me-do I have
 a different version?). Looking at the time chart in Chrome I see that
 the bulk of the time is for /ipa/session waiting. Would "waiting" mean
 waiting for the directory server or memcached?
>>>
>>> Actually neither, it means waiting for a response from the web server
>>> (technically it's making an RPC call via HTTP Ajax). The RPC call needs to
>>> go through the web server, memcached, and typically will invoke one or more
>>> directory server queries, and run a bunch of Python to massage everything
>>> before the RPC returns with the result.
>>>
>>> It doesn't look like you've got much difference in times between with
>>> pagination on and pagination off. I don't know the pagination code but I
>>> suspect it's run after the RPC call returns so the RPC timing is not telling
>>> us much with respect to that.
>>>
>>> Waiting for up to 3 seconds for an RPC call does seem on the high side. Do
>>> you have a lot of LDAP data?
>> No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
>> any significant amount of hosts in it.
>>
>>> But really, unless we get timing results for each component we're grasping
>>> at straws :-(
>> Understood.
>>
>> Steve
>>
>> ___
>> Freeipa-users mailing list
>> Freeipa-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/freeipa-users
>
> Do you have a replica and does this replica behave the same?

No replica yet. I wanted to get the memory leak issue solved first.
All I have to compare to is the old 2.1.3 before. This one is much
slower. I just can't seem to figure out what's wrong. The upgrade
seemed to complete successfully and there were no errors in the log.
The only things I've found thus far (earlier in this thread) are the
unindexed entries (all hosts entries) that Rich seemed to think might
be slowing things up. As the slowness is on every page, I wouldn't
think that would be the problem.

I wouldn't have said as much about this were it not for the promised
faster speed mentioned in the release notes. It's comparable to the
old 2.0 release candidates so I thought it might have been due to the
complexity of the feature additions.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Dmitri Pal
On 07/18/2012 03:45 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
>> On 07/18/2012 02:59 PM, Stephen Ingram wrote:
>>> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
>>> wrote:
 On 07/17/2012 11:43 PM, Stephen Ingram wrote:

 8><--


>>> I'm beginning to think this is just the Web UI itself instead of 389
>>> although it is really difficult to tell. I've poured over the debug
>>> logs and didn't see anything that caused me concern.
>>>
>>> It's certainly usable, but I just got really spoiled by the
>>> unbelievable quickness of 2.1.3. When your release notes indicate it
>>> should be faster, what are you comparing it to? Maybe this only
>>> happens with upgraded instances and not fresh installs.
>>
>>
>>
>> It is always possible something didn't get upgraded properly but I've
>> done
>> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
>> faster we're always referring to the previous version (or versions).
>
>
> Maybe I was just lucky with 2.1.3. On a first load it might take some
> time to load the "frame" as I call it. But the data would load almost
> instantaneously from there (certainly no more than 1 s) as you moved
> from page to page. Here, even if I return to the same page, the system
> acts as if the data is begin fetched for the very first time as it is
> no faster than the first load. Maybe that is significant to the
> problem?


 I think the culprit is Web UI paging capabilities introduced in 2.2. With
 lot of users, responses might grow in size. You can check their size and
 duration in browser developers tools. I suggest chrome/chromium - press
 F12
 and choose 'network' tab.

 This new feature can't be disabled in configuration. To test if the
 slowdown
 is done by paging you can (at own risk) replace line
 /usr/share/ipa/ui/facet.js:538

 that.pagination = spec.pagination === undefined ? true : spec.pagination;

 with:

 that.pagination = false;

 Note: It will break some other parts of the UI - so for testing only.
>>>
>>> I've made the substitution in the code (was line 507 for me-do I have
>>> a different version?). Looking at the time chart in Chrome I see that
>>> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
>>> waiting for the directory server or memcached?
>>
>> Actually neither, it means waiting for a response from the web server
>> (technically it's making an RPC call via HTTP Ajax). The RPC call needs to
>> go through the web server, memcached, and typically will invoke one or more
>> directory server queries, and run a bunch of Python to massage everything
>> before the RPC returns with the result.
>>
>> It doesn't look like you've got much difference in times between with
>> pagination on and pagination off. I don't know the pagination code but I
>> suspect it's run after the RPC call returns so the RPC timing is not telling
>> us much with respect to that.
>>
>> Waiting for up to 3 seconds for an RPC call does seem on the high side. Do
>> you have a lot of LDAP data?
> No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
> any significant amount of hosts in it.
>
>> But really, unless we get timing results for each component we're grasping
>> at straws :-(
> Understood.
>
> Steve
>
> ___
> Freeipa-users mailing list
> Freeipa-users@redhat.com
> https://www.redhat.com/mailman/listinfo/freeipa-users

Do you have a replica and does this replica behave the same?

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Wed, Jul 18, 2012 at 10:59 AM, Dmitri Pal  wrote:
> On 07/18/2012 01:53 PM, Stephen Ingram wrote:
>> On Tue, Jul 17, 2012 at 3:56 PM, John Dennis  wrote:
>>> On 07/17/2012 05:43 PM, Stephen Ingram wrote:
>>>
 [ details of performance analysis snipped for brevity ]
>>> I wonder if we shouldn't add some timing metrics to our code. As it is it's
>>> very hard to know where time is being spent.
>>>
>>> When I wrote the session code I added some timestamps used for managing
>>> session timeouts. It wouldn't be too hard to expand this to time how long it
>>> takes a command to execute because it's evaluated for every command.
>>> Combined with timestamping in the UI code we could get a reasonable idea of
>>> where some bottlenecks lie (or don't).
>> I've never used this before so I'm not sure how it would work, but it
>> sounds great. It's really difficult to tell what's causing the issue
>> when there are so many processes occurring.
>>
>
>
> While we are going with the technical digging let us also try to collect
> the sufficient information about the problem.
>
> Here is some questions that would help us to reproduce the issue.
>
> 1) If the problem with every frame of just some specific UI pages?

The frame seems to load quickly. It is the inside part that contains
the data that is much slower.

> Can you for example see IPA Configuration panel or log as a self service
> user? Are those fast?

I'm not sure what you mean by configuration panel, but if I login as
admin or self-service user, they are both equally slow.

> 2) Say it is users is so how many users do you have? Is it thousands?

No, only 49 users at the moment. We're still adding people. There
isn't a lot of data in the directory period--another reason I'm so
surprised by the slowness.

> Or may be it is a specific group?

I notice that everyone is automatically subscribed to the ipausers
group. Hasn't that been changed such that the subscription is no
longer automatic? Maybe it is taking too long to enumerate that group?
I can unsubscribe the users if needed. Our groups only contain 3 users
on average.

> We might need to reproduce the same setup and see what is going on.

I'm more than willing to help in any way I can. I'm even considering
pulling our old 2.1.3 system from backup, but it would be difficult as
this on is in production now. I switched because of the memory leak in
2.1.3.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Wed, Jul 18, 2012 at 12:28 PM, John Dennis  wrote:
> On 07/18/2012 02:59 PM, Stephen Ingram wrote:
>>
>> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik 
>> wrote:
>>>
>>> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>>>
>>> 8><--
>>>
>>>
>>
>> I'm beginning to think this is just the Web UI itself instead of 389
>> although it is really difficult to tell. I've poured over the debug
>> logs and didn't see anything that caused me concern.
>>
>> It's certainly usable, but I just got really spoiled by the
>> unbelievable quickness of 2.1.3. When your release notes indicate it
>> should be faster, what are you comparing it to? Maybe this only
>> happens with upgraded instances and not fresh installs.
>
>
>
>
> It is always possible something didn't get upgraded properly but I've
> done
> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
> faster we're always referring to the previous version (or versions).



 Maybe I was just lucky with 2.1.3. On a first load it might take some
 time to load the "frame" as I call it. But the data would load almost
 instantaneously from there (certainly no more than 1 s) as you moved
 from page to page. Here, even if I return to the same page, the system
 acts as if the data is begin fetched for the very first time as it is
 no faster than the first load. Maybe that is significant to the
 problem?
>>>
>>>
>>>
>>> I think the culprit is Web UI paging capabilities introduced in 2.2. With
>>> lot of users, responses might grow in size. You can check their size and
>>> duration in browser developers tools. I suggest chrome/chromium - press
>>> F12
>>> and choose 'network' tab.
>>>
>>> This new feature can't be disabled in configuration. To test if the
>>> slowdown
>>> is done by paging you can (at own risk) replace line
>>> /usr/share/ipa/ui/facet.js:538
>>>
>>> that.pagination = spec.pagination === undefined ? true : spec.pagination;
>>>
>>> with:
>>>
>>> that.pagination = false;
>>>
>>> Note: It will break some other parts of the UI - so for testing only.
>>
>>
>> I've made the substitution in the code (was line 507 for me-do I have
>> a different version?). Looking at the time chart in Chrome I see that
>> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
>> waiting for the directory server or memcached?
>
>
> Actually neither, it means waiting for a response from the web server
> (technically it's making an RPC call via HTTP Ajax). The RPC call needs to
> go through the web server, memcached, and typically will invoke one or more
> directory server queries, and run a bunch of Python to massage everything
> before the RPC returns with the result.
>
> It doesn't look like you've got much difference in times between with
> pagination on and pagination off. I don't know the pagination code but I
> suspect it's run after the RPC call returns so the RPC timing is not telling
> us much with respect to that.
>
> Waiting for up to 3 seconds for an RPC call does seem on the high side. Do
> you have a lot of LDAP data?

No. 49 users, 17 hosts, 25 services, 6 DNS zones, only 1 of which has
any significant amount of hosts in it.

> But really, unless we get timing results for each component we're grasping
> at straws :-(

Understood.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread John Dennis

On 07/18/2012 02:59 PM, Stephen Ingram wrote:

On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik  wrote:

On 07/17/2012 11:43 PM, Stephen Ingram wrote:

8><--




I'm beginning to think this is just the Web UI itself instead of 389
although it is really difficult to tell. I've poured over the debug
logs and didn't see anything that caused me concern.

It's certainly usable, but I just got really spoiled by the
unbelievable quickness of 2.1.3. When your release notes indicate it
should be faster, what are you comparing it to? Maybe this only
happens with upgraded instances and not fresh installs.




It is always possible something didn't get upgraded properly but I've
done
2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
faster we're always referring to the previous version (or versions).



Maybe I was just lucky with 2.1.3. On a first load it might take some
time to load the "frame" as I call it. But the data would load almost
instantaneously from there (certainly no more than 1 s) as you moved
from page to page. Here, even if I return to the same page, the system
acts as if the data is begin fetched for the very first time as it is
no faster than the first load. Maybe that is significant to the
problem?



I think the culprit is Web UI paging capabilities introduced in 2.2. With
lot of users, responses might grow in size. You can check their size and
duration in browser developers tools. I suggest chrome/chromium - press F12
and choose 'network' tab.

This new feature can't be disabled in configuration. To test if the slowdown
is done by paging you can (at own risk) replace line
/usr/share/ipa/ui/facet.js:538

that.pagination = spec.pagination === undefined ? true : spec.pagination;

with:

that.pagination = false;

Note: It will break some other parts of the UI - so for testing only.


I've made the substitution in the code (was line 507 for me-do I have
a different version?). Looking at the time chart in Chrome I see that
the bulk of the time is for /ipa/session waiting. Would "waiting" mean
waiting for the directory server or memcached?


Actually neither, it means waiting for a response from the web server 
(technically it's making an RPC call via HTTP Ajax). The RPC call needs 
to go through the web server, memcached, and typically will invoke one 
or more directory server queries, and run a bunch of Python to massage 
everything before the RPC returns with the result.


It doesn't look like you've got much difference in times between with 
pagination on and pagination off. I don't know the pagination code but I 
suspect it's run after the RPC call returns so the RPC timing is not 
telling us much with respect to that.


Waiting for up to 3 seconds for an RPC call does seem on the high side. 
Do you have a lot of LDAP data?


But really, unless we get timing results for each component we're 
grasping at straws :-(




Here's a portion of the initial load of the Users page:
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 33.94KB 33.10KB 1.57s 1.47s 96ms (1.37s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 568.09KB 564.36KB 3.92s 2.95s 963ms (2.85s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 556.94KB 553.40KB 3.78s 2.94s 836ms (2.83s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 46.93KB 46.38KB 1.87s 1.71s (1.60s waiting)

Now, with the pagination turned back on:
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 33.94KB 33.10KB 1.58s 1.48s 100ms (1.38s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 568.09KB 564.36KB 4.05s 3.09s 964ms (2.98s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 556.94KB 553.40KB 3.84s 2.99s 855ms (2.88s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 46.93KB 46.38KB 1.52s 1.51s (1.40s waiting)

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users




--
John Dennis 

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik  wrote:
> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>
> 8><--
>
>

 I'm beginning to think this is just the Web UI itself instead of 389
 although it is really difficult to tell. I've poured over the debug
 logs and didn't see anything that caused me concern.

 It's certainly usable, but I just got really spoiled by the
 unbelievable quickness of 2.1.3. When your release notes indicate it
 should be faster, what are you comparing it to? Maybe this only
 happens with upgraded instances and not fresh installs.
>>>
>>>
>>>
>>> It is always possible something didn't get upgraded properly but I've
>>> done
>>> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
>>> faster we're always referring to the previous version (or versions).
>>
>>
>> Maybe I was just lucky with 2.1.3. On a first load it might take some
>> time to load the "frame" as I call it. But the data would load almost
>> instantaneously from there (certainly no more than 1 s) as you moved
>> from page to page. Here, even if I return to the same page, the system
>> acts as if the data is begin fetched for the very first time as it is
>> no faster than the first load. Maybe that is significant to the
>> problem?
>
>
> I think the culprit is Web UI paging capabilities introduced in 2.2. With
> lot of users, responses might grow in size. You can check their size and
> duration in browser developers tools. I suggest chrome/chromium - press F12
> and choose 'network' tab.
>
> This new feature can't be disabled in configuration. To test if the slowdown
> is done by paging you can (at own risk) replace line
> /usr/share/ipa/ui/facet.js:538
>
> that.pagination = spec.pagination === undefined ? true : spec.pagination;
>
> with:
>
> that.pagination = false;
>
> Note: It will break some other parts of the UI - so for testing only.

I've made the substitution in the code (was line 507 for me-do I have
a different version?). Looking at the time chart in Chrome I see that
the bulk of the time is for /ipa/session waiting. Would "waiting" mean
waiting for the directory server or memcached?

Here's a portion of the initial load of the Users page:
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 33.94KB 33.10KB 1.57s 1.47s 96ms (1.37s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 568.09KB 564.36KB 3.92s 2.95s 963ms (2.85s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 556.94KB 553.40KB 3.78s 2.94s 836ms (2.83s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 46.93KB 46.38KB 1.87s 1.71s (1.60s waiting)

Now, with the pagination turned back on:
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 33.94KB 33.10KB 1.58s 1.48s 100ms (1.38s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 568.09KB 564.36KB 4.05s 3.09s 964ms (2.98s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 556.94KB 553.40KB 3.84s 2.99s 855ms (2.88s waiting)
json/ipa/session POST 200 Success application/json jquery.js:7365
Script 46.93KB 46.38KB 1.52s 1.51s (1.40s waiting)

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Dmitri Pal
On 07/18/2012 01:53 PM, Stephen Ingram wrote:
> On Tue, Jul 17, 2012 at 3:56 PM, John Dennis  wrote:
>> On 07/17/2012 05:43 PM, Stephen Ingram wrote:
>>
>>> [ details of performance analysis snipped for brevity ]
>> I wonder if we shouldn't add some timing metrics to our code. As it is it's
>> very hard to know where time is being spent.
>>
>> When I wrote the session code I added some timestamps used for managing
>> session timeouts. It wouldn't be too hard to expand this to time how long it
>> takes a command to execute because it's evaluated for every command.
>> Combined with timestamping in the UI code we could get a reasonable idea of
>> where some bottlenecks lie (or don't).
> I've never used this before so I'm not sure how it would work, but it
> sounds great. It's really difficult to tell what's causing the issue
> when there are so many processes occurring.
>


While we are going with the technical digging let us also try to collect
the sufficient information about the problem.

Here is some questions that would help us to reproduce the issue.

1) If the problem with every frame of just some specific UI pages?
Can you for example see IPA Configuration panel or log as a self service
user? Are those fast?
2) Say it is users is so how many users do you have? Is it thousands?
Or may be it is a specific group?

We might need to reproduce the same setup and see what is going on.

> Steve
>
> ___
> Freeipa-users mailing list
> Freeipa-users@redhat.com
> https://www.redhat.com/mailman/listinfo/freeipa-users


-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Stephen Ingram
On Tue, Jul 17, 2012 at 3:56 PM, John Dennis  wrote:
> On 07/17/2012 05:43 PM, Stephen Ingram wrote:
>
>> [ details of performance analysis snipped for brevity ]
>
> I wonder if we shouldn't add some timing metrics to our code. As it is it's
> very hard to know where time is being spent.
>
> When I wrote the session code I added some timestamps used for managing
> session timeouts. It wouldn't be too hard to expand this to time how long it
> takes a command to execute because it's evaluated for every command.
> Combined with timestamping in the UI code we could get a reasonable idea of
> where some bottlenecks lie (or don't).

I've never used this before so I'm not sure how it would work, but it
sounds great. It's really difficult to tell what's causing the issue
when there are so many processes occurring.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-18 Thread Petr Vobornik

On 07/17/2012 11:43 PM, Stephen Ingram wrote:

8><--



I'm beginning to think this is just the Web UI itself instead of 389
although it is really difficult to tell. I've poured over the debug
logs and didn't see anything that caused me concern.

It's certainly usable, but I just got really spoiled by the
unbelievable quickness of 2.1.3. When your release notes indicate it
should be faster, what are you comparing it to? Maybe this only
happens with upgraded instances and not fresh installs.



It is always possible something didn't get upgraded properly but I've done
2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
faster we're always referring to the previous version (or versions).


Maybe I was just lucky with 2.1.3. On a first load it might take some
time to load the "frame" as I call it. But the data would load almost
instantaneously from there (certainly no more than 1 s) as you moved
from page to page. Here, even if I return to the same page, the system
acts as if the data is begin fetched for the very first time as it is
no faster than the first load. Maybe that is significant to the
problem?


I think the culprit is Web UI paging capabilities introduced in 2.2. 
With lot of users, responses might grow in size. You can check their 
size and duration in browser developers tools. I suggest chrome/chromium 
- press F12 and choose 'network' tab.


This new feature can't be disabled in configuration. To test if the 
slowdown is done by paging you can (at own risk) replace line 
/usr/share/ipa/ui/facet.js:538


that.pagination = spec.pagination === undefined ? true : spec.pagination;

with:

that.pagination = false;

Note: It will break some other parts of the UI - so for testing only.


Steve



--
Petr Vobornik


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-17 Thread John Dennis

On 07/17/2012 05:43 PM, Stephen Ingram wrote:

> [ details of performance analysis snipped for brevity ]

I wonder if we shouldn't add some timing metrics to our code. As it is 
it's very hard to know where time is being spent.


When I wrote the session code I added some timestamps used for managing 
session timeouts. It wouldn't be too hard to expand this to time how 
long it takes a command to execute because it's evaluated for every 
command. Combined with timestamping in the UI code we could get a 
reasonable idea of where some bottlenecks lie (or don't).


--
John Dennis 

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-17 Thread Stephen Ingram
On Tue, Jul 17, 2012 at 2:01 PM, Rob Crittenden  wrote:
> Stephen Ingram wrote:
>>
>> On Mon, Jul 16, 2012 at 12:23 PM, Rob Crittenden 
>> wrote:
>>>
>>> Stephen Ingram wrote:


 On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson 
 wrote:
>
>
> On 07/16/2012 11:48 AM, Stephen Ingram wrote:
>>
>>
>>
>> On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson
>> wrote:
>>>
>>>
>>>
>>> On 07/16/2012 10:19 AM, Stephen Ingram wrote:



 On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
 wrote:
>
>
>
> Stephen Ingram wrote:
>>
>>
>>
>> On Thu, Jul 12, 2012 at 2:59 PM, Steven
>> Jones
>> wrote:
>>>
>>>
>>>
>>> Hi,
>>>
>>> I had huge memory issues pre 6.3, now its low and flatSounds
>>> like
>>> you
>>> have an issue somewhere. My normal cpu use is a few hundred
>>> mhzbut
>>> when
>>> "something" goes wrong such as replication failing that
>>> climbs...ditto
>>> memory use
>>
>>
>>
>>
>> Yes, I saw your conversation with Rich on this list about that.
>> And,
>> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is
>> still
>> having issues. It was an upgrade from 2.1.3, but the upgrade
>> seemed
>> to
>> complete without issue. I'm also not even doing replication yet so
>> I'm
>> not sure why memory is so high. Web interface is much slower too
>> so
>> perhaps something else is wrong.
>
>
>
>
> Can you tell where it is being slow? Does it seem related to
> retrieving
> data
> from LDAP?



 I'm not really sure yet what is causing the slowness. I have the
 same
 number of directory entries as before the upgrade. It was very quick
 with 2.1.3, but once I upgraded, I felt like I was back to the
 pre-2.0
 days--without a doubt much, much slower.

> You might check your 389-ds access logs and look for searches with
> notes=U.
> Perhaps you are missing an index.



 Yes there are lots of notes=U. What does this mean? Was something
 missed in the upgrade script?
>>>
>>>
>>>
>>> Try running logconv.pl
>>
>>
>>
>> Nice! I'm guessing that notes=U are unindexed searches then. I have 34
>> over the last 24 hours so I'm not sure this would be causing the issue
>> as the slowness persists through every click.
>
>
>
> Yeah, I would expect to see a lot more than 34 if that were the cause.
>
> Can you post the search filters that are unindexed?



 Sure, here's a partial list (sanitized):



 filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com)
 attrs="fqdn"


 filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
 attrs="fqdn"


 filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
 attrs="fqdn"


 filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
 attrs="fqdn"


 filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
 attrs="fqdn"

 All the rest are the same, just with different hosts.

>> I've traced the
>> unindexed searches back to the time of Web UI access and they don't
>> match. I also don't see any other obvious errors when running
>> logconv.pl.
>>
>> One strange thing I have noticed is that the 389 server logs seem to
>> update in "spurts". If I'm tailing the logs while I access a Web UI
>> page, there is nothing, then a couple of seconds later, I see the logs
>> quickly scroll with new entires. Has this always been the case? I
>> don't seem to remember this before.
>
>
>
> Yes.  The 389 access log is buffered, for performance reasons.



 Just thought it might be relevant. I'm not sure what is causing the
 extreme slowness. I've also shut off memcached and tried without it
 with no discernible difference. The directory seems to be handling the
 load of external queries just fine, although I'm not sure I've solved
 the memory issue--I'm still testing with the compat plugin disabled to
 see if I can stop the memory creep. Maybe it's something in the code
 of the Web UI itself as its even slow when changing from page to page
 of users and hosts.
>>>
>>>
>>>
>>> Shutting of memcached is just going to make things even slower.
>>
>>
>> I 

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-17 Thread Rob Crittenden

Stephen Ingram wrote:

On Mon, Jul 16, 2012 at 12:23 PM, Rob Crittenden  wrote:

Stephen Ingram wrote:


On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson 
wrote:


On 07/16/2012 11:48 AM, Stephen Ingram wrote:



On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson
wrote:



On 07/16/2012 10:19 AM, Stephen Ingram wrote:



On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
wrote:



Stephen Ingram wrote:



On Thu, Jul 12, 2012 at 2:59 PM, Steven
Jones
wrote:



Hi,

I had huge memory issues pre 6.3, now its low and flatSounds
like
you
have an issue somewhere. My normal cpu use is a few hundred
mhzbut
when
"something" goes wrong such as replication failing that
climbs...ditto
memory use




Yes, I saw your conversation with Rich on this list about that. And,
yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
having issues. It was an upgrade from 2.1.3, but the upgrade seemed
to
complete without issue. I'm also not even doing replication yet so
I'm
not sure why memory is so high. Web interface is much slower too so
perhaps something else is wrong.




Can you tell where it is being slow? Does it seem related to
retrieving
data
from LDAP?



I'm not really sure yet what is causing the slowness. I have the same
number of directory entries as before the upgrade. It was very quick
with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
days--without a doubt much, much slower.


You might check your 389-ds access logs and look for searches with
notes=U.
Perhaps you are missing an index.



Yes there are lots of notes=U. What does this mean? Was something
missed in the upgrade script?



Try running logconv.pl



Nice! I'm guessing that notes=U are unindexed searches then. I have 34
over the last 24 hours so I'm not sure this would be causing the issue
as the slowness persists through every click.



Yeah, I would expect to see a lot more than 34 if that were the cause.

Can you post the search filters that are unindexed?



Sure, here's a partial list (sanitized):


filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com)
attrs="fqdn"

filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"

filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"

filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"

filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"

All the rest are the same, just with different hosts.


I've traced the
unindexed searches back to the time of Web UI access and they don't
match. I also don't see any other obvious errors when running
logconv.pl.

One strange thing I have noticed is that the 389 server logs seem to
update in "spurts". If I'm tailing the logs while I access a Web UI
page, there is nothing, then a couple of seconds later, I see the logs
quickly scroll with new entires. Has this always been the case? I
don't seem to remember this before.



Yes.  The 389 access log is buffered, for performance reasons.



Just thought it might be relevant. I'm not sure what is causing the
extreme slowness. I've also shut off memcached and tried without it
with no discernible difference. The directory seems to be handling the
load of external queries just fine, although I'm not sure I've solved
the memory issue--I'm still testing with the compat plugin disabled to
see if I can stop the memory creep. Maybe it's something in the code
of the Web UI itself as its even slow when changing from page to page
of users and hosts.



Shutting of memcached is just going to make things even slower.


I really didn't see much difference so I turned it back on right away.


Things you can try on a quiet system:

1. Create /etc/ipa/server.conf:

[global]
debug=True

Restart Apache

Use the UI to do a few things. Verify in the logs that the session cache is
being used.


Yes, it is. It's interesting, 2.2 is slower such that you can see the
frame load, and then the loading symbol spins below in the display
area for a few seconds while that loads up. Before, with 2.1.3, you
really couldn't distinguish the two as they loaded so quickly.


A lot of what gets loaded a just big javascript files. I wonder if there 
is a DNS problem, that would explain the slowness. The javascript and 
much of the UI is completely unprotected by anything.



2. Check your browser configuration. You don't need delegation-uris set any
more. Having it set might mask other problems (you still need
negotiation-auth.trusted-uris set).


I forgot about this. I changed it, completely cleared the browser
cache and accessed without any noticeable difference.


3. Watch what URI is being used in the Apache access log. It should be
/ipa/session/json.


Check. this is where it lands.

I'm beginning to think this is just the Web UI itself instead of 389
although it is really difficult to 

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-17 Thread Stephen Ingram
On Mon, Jul 16, 2012 at 12:23 PM, Rob Crittenden  wrote:
> Stephen Ingram wrote:
>>
>> On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson 
>> wrote:
>>>
>>> On 07/16/2012 11:48 AM, Stephen Ingram wrote:


 On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson
 wrote:
>
>
> On 07/16/2012 10:19 AM, Stephen Ingram wrote:
>>
>>
>> On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
>> wrote:
>>>
>>>
>>> Stephen Ingram wrote:


 On Thu, Jul 12, 2012 at 2:59 PM, Steven
 Jones
 wrote:
>
>
> Hi,
>
> I had huge memory issues pre 6.3, now its low and flatSounds
> like
> you
> have an issue somewhere. My normal cpu use is a few hundred
> mhzbut
> when
> "something" goes wrong such as replication failing that
> climbs...ditto
> memory use



 Yes, I saw your conversation with Rich on this list about that. And,
 yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
 having issues. It was an upgrade from 2.1.3, but the upgrade seemed
 to
 complete without issue. I'm also not even doing replication yet so
 I'm
 not sure why memory is so high. Web interface is much slower too so
 perhaps something else is wrong.
>>>
>>>
>>>
>>> Can you tell where it is being slow? Does it seem related to
>>> retrieving
>>> data
>>> from LDAP?
>>
>>
>> I'm not really sure yet what is causing the slowness. I have the same
>> number of directory entries as before the upgrade. It was very quick
>> with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
>> days--without a doubt much, much slower.
>>
>>> You might check your 389-ds access logs and look for searches with
>>> notes=U.
>>> Perhaps you are missing an index.
>>
>>
>> Yes there are lots of notes=U. What does this mean? Was something
>> missed in the upgrade script?
>
>
> Try running logconv.pl


 Nice! I'm guessing that notes=U are unindexed searches then. I have 34
 over the last 24 hours so I'm not sure this would be causing the issue
 as the slowness persists through every click.
>>>
>>>
>>> Yeah, I would expect to see a lot more than 34 if that were the cause.
>>>
>>> Can you post the search filters that are unindexed?
>>
>>
>> Sure, here's a partial list (sanitized):
>>
>>
>> filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com)
>> attrs="fqdn"
>>
>> filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
>> attrs="fqdn"
>>
>> filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
>> attrs="fqdn"
>>
>> filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
>> attrs="fqdn"
>>
>> filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
>> attrs="fqdn"
>>
>> All the rest are the same, just with different hosts.
>>
 I've traced the
 unindexed searches back to the time of Web UI access and they don't
 match. I also don't see any other obvious errors when running
 logconv.pl.

 One strange thing I have noticed is that the 389 server logs seem to
 update in "spurts". If I'm tailing the logs while I access a Web UI
 page, there is nothing, then a couple of seconds later, I see the logs
 quickly scroll with new entires. Has this always been the case? I
 don't seem to remember this before.
>>>
>>>
>>> Yes.  The 389 access log is buffered, for performance reasons.
>>
>>
>> Just thought it might be relevant. I'm not sure what is causing the
>> extreme slowness. I've also shut off memcached and tried without it
>> with no discernible difference. The directory seems to be handling the
>> load of external queries just fine, although I'm not sure I've solved
>> the memory issue--I'm still testing with the compat plugin disabled to
>> see if I can stop the memory creep. Maybe it's something in the code
>> of the Web UI itself as its even slow when changing from page to page
>> of users and hosts.
>
>
> Shutting of memcached is just going to make things even slower.

I really didn't see much difference so I turned it back on right away.

> Things you can try on a quiet system:
>
> 1. Create /etc/ipa/server.conf:
>
> [global]
> debug=True
>
> Restart Apache
>
> Use the UI to do a few things. Verify in the logs that the session cache is
> being used.

Yes, it is. It's interesting, 2.2 is slower such that you can see the
frame load, and then the loading symbol spins below in the display
area for a few seconds while that loads up. Before, with 2.1.3, you
really couldn't distinguish the two as they loaded so quickly.

> 2. Check your browser configuration. You 

Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Rob Crittenden

Stephen Ingram wrote:

On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson  wrote:

On 07/16/2012 11:48 AM, Stephen Ingram wrote:


On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson
wrote:


On 07/16/2012 10:19 AM, Stephen Ingram wrote:


On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
wrote:


Stephen Ingram wrote:


On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones
wrote:


Hi,

I had huge memory issues pre 6.3, now its low and flatSounds like
you
have an issue somewhere. My normal cpu use is a few hundred
mhzbut
when
"something" goes wrong such as replication failing that
climbs...ditto
memory use



Yes, I saw your conversation with Rich on this list about that. And,
yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
complete without issue. I'm also not even doing replication yet so I'm
not sure why memory is so high. Web interface is much slower too so
perhaps something else is wrong.



Can you tell where it is being slow? Does it seem related to retrieving
data
from LDAP?


I'm not really sure yet what is causing the slowness. I have the same
number of directory entries as before the upgrade. It was very quick
with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
days--without a doubt much, much slower.


You might check your 389-ds access logs and look for searches with
notes=U.
Perhaps you are missing an index.


Yes there are lots of notes=U. What does this mean? Was something
missed in the upgrade script?


Try running logconv.pl


Nice! I'm guessing that notes=U are unindexed searches then. I have 34
over the last 24 hours so I'm not sure this would be causing the issue
as the slowness persists through every click.


Yeah, I would expect to see a lot more than 34 if that were the cause.

Can you post the search filters that are unindexed?


Sure, here's a partial list (sanitized):

filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com)
attrs="fqdn"
filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"
filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"
filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"
filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"

All the rest are the same, just with different hosts.


I've traced the
unindexed searches back to the time of Web UI access and they don't
match. I also don't see any other obvious errors when running
logconv.pl.

One strange thing I have noticed is that the 389 server logs seem to
update in "spurts". If I'm tailing the logs while I access a Web UI
page, there is nothing, then a couple of seconds later, I see the logs
quickly scroll with new entires. Has this always been the case? I
don't seem to remember this before.


Yes.  The 389 access log is buffered, for performance reasons.


Just thought it might be relevant. I'm not sure what is causing the
extreme slowness. I've also shut off memcached and tried without it
with no discernible difference. The directory seems to be handling the
load of external queries just fine, although I'm not sure I've solved
the memory issue--I'm still testing with the compat plugin disabled to
see if I can stop the memory creep. Maybe it's something in the code
of the Web UI itself as its even slow when changing from page to page
of users and hosts.


Shutting of memcached is just going to make things even slower.

Things you can try on a quiet system:

1. Create /etc/ipa/server.conf:

[global]
debug=True

Restart Apache

Use the UI to do a few things. Verify in the logs that the session cache 
is being used.


2. Check your browser configuration. You don't need delegation-uris set 
any more. Having it set might mask other problems (you still need 
negotiation-auth.trusted-uris set).


3. Watch what URI is being used in the Apache access log. It should be 
/ipa/session/json.


rob

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Simo Sorce
On Mon, 2012-07-16 at 12:11 -0700, Stephen Ingram wrote:
> On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson  wrote:
> > On 07/16/2012 11:48 AM, Stephen Ingram wrote:
> >>
> >> On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson
> >> wrote:
> >>>
> >>> On 07/16/2012 10:19 AM, Stephen Ingram wrote:
> 
>  On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
>  wrote:
> >
> > Stephen Ingram wrote:
> >>
> >> On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I had huge memory issues pre 6.3, now its low and flatSounds like
> >>> you
> >>> have an issue somewhere. My normal cpu use is a few hundred
> >>> mhzbut
> >>> when
> >>> "something" goes wrong such as replication failing that
> >>> climbs...ditto
> >>> memory use
> >>
> >>
> >> Yes, I saw your conversation with Rich on this list about that. And,
> >> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
> >> having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
> >> complete without issue. I'm also not even doing replication yet so I'm
> >> not sure why memory is so high. Web interface is much slower too so
> >> perhaps something else is wrong.
> >
> >
> > Can you tell where it is being slow? Does it seem related to retrieving
> > data
> > from LDAP?
> 
>  I'm not really sure yet what is causing the slowness. I have the same
>  number of directory entries as before the upgrade. It was very quick
>  with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
>  days--without a doubt much, much slower.
> 
> > You might check your 389-ds access logs and look for searches with
> > notes=U.
> > Perhaps you are missing an index.
> 
>  Yes there are lots of notes=U. What does this mean? Was something
>  missed in the upgrade script?
> >>>
> >>> Try running logconv.pl
> >>
> >> Nice! I'm guessing that notes=U are unindexed searches then. I have 34
> >> over the last 24 hours so I'm not sure this would be causing the issue
> >> as the slowness persists through every click.
> >
> > Yeah, I would expect to see a lot more than 34 if that were the cause.
> >
> > Can you post the search filters that are unindexed?
> 
> Sure, here's a partial list (sanitized):
> 
> filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com)
> attrs="fqdn"
> filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
> attrs="fqdn"
> filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
> attrs="fqdn"
> filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
> attrs="fqdn"
> filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
> attrs="fqdn"
> 
> All the rest are the same, just with different hosts.
> 
> >> I've traced the
> >> unindexed searches back to the time of Web UI access and they don't
> >> match. I also don't see any other obvious errors when running
> >> logconv.pl.
> >>
> >> One strange thing I have noticed is that the 389 server logs seem to
> >> update in "spurts". If I'm tailing the logs while I access a Web UI
> >> page, there is nothing, then a couple of seconds later, I see the logs
> >> quickly scroll with new entires. Has this always been the case? I
> >> don't seem to remember this before.
> >
> > Yes.  The 389 access log is buffered, for performance reasons.
> 
> Just thought it might be relevant. I'm not sure what is causing the
> extreme slowness. I've also shut off memcached and tried without it
> with no discernible difference. The directory seems to be handling the
> load of external queries just fine, although I'm not sure I've solved
> the memory issue--I'm still testing with the compat plugin disabled to
> see if I can stop the memory creep. Maybe it's something in the code
> of the Web UI itself as its even slow when changing from page to page
> of users and hosts.

Looks like the symptoms of not using session cookies.
Do you see constant activity getting tickets for ldap/ipa.server.fqdn in
the krb5kdc.log ?

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Stephen Ingram
On Mon, Jul 16, 2012 at 11:34 AM, Rich Megginson  wrote:
> On 07/16/2012 11:48 AM, Stephen Ingram wrote:
>>
>> On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson
>> wrote:
>>>
>>> On 07/16/2012 10:19 AM, Stephen Ingram wrote:

 On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
 wrote:
>
> Stephen Ingram wrote:
>>
>> On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones
>> wrote:
>>>
>>> Hi,
>>>
>>> I had huge memory issues pre 6.3, now its low and flatSounds like
>>> you
>>> have an issue somewhere. My normal cpu use is a few hundred
>>> mhzbut
>>> when
>>> "something" goes wrong such as replication failing that
>>> climbs...ditto
>>> memory use
>>
>>
>> Yes, I saw your conversation with Rich on this list about that. And,
>> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
>> having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
>> complete without issue. I'm also not even doing replication yet so I'm
>> not sure why memory is so high. Web interface is much slower too so
>> perhaps something else is wrong.
>
>
> Can you tell where it is being slow? Does it seem related to retrieving
> data
> from LDAP?

 I'm not really sure yet what is causing the slowness. I have the same
 number of directory entries as before the upgrade. It was very quick
 with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
 days--without a doubt much, much slower.

> You might check your 389-ds access logs and look for searches with
> notes=U.
> Perhaps you are missing an index.

 Yes there are lots of notes=U. What does this mean? Was something
 missed in the upgrade script?
>>>
>>> Try running logconv.pl
>>
>> Nice! I'm guessing that notes=U are unindexed searches then. I have 34
>> over the last 24 hours so I'm not sure this would be causing the issue
>> as the slowness persists through every click.
>
> Yeah, I would expect to see a lot more than 34 if that were the cause.
>
> Can you post the search filters that are unindexed?

Sure, here's a partial list (sanitized):

filter="(managedBy=fqdn=ec2-x.x.x.us-west-2.compute.amazonaws.com,cn=computers,cn=accounts,dc=example,dc=com)
attrs="fqdn"
filter="(managedBy=fqdn=imap.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"
filter="(managedBy=fqdn=imap1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"
filter="(managedBy=fqdn=imap2.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"
filter="(managedBy=fqdn=ipa1.example.com,cn=computers,cn=accounts,dc=example,dc=com)"
attrs="fqdn"

All the rest are the same, just with different hosts.

>> I've traced the
>> unindexed searches back to the time of Web UI access and they don't
>> match. I also don't see any other obvious errors when running
>> logconv.pl.
>>
>> One strange thing I have noticed is that the 389 server logs seem to
>> update in "spurts". If I'm tailing the logs while I access a Web UI
>> page, there is nothing, then a couple of seconds later, I see the logs
>> quickly scroll with new entires. Has this always been the case? I
>> don't seem to remember this before.
>
> Yes.  The 389 access log is buffered, for performance reasons.

Just thought it might be relevant. I'm not sure what is causing the
extreme slowness. I've also shut off memcached and tried without it
with no discernible difference. The directory seems to be handling the
load of external queries just fine, although I'm not sure I've solved
the memory issue--I'm still testing with the compat plugin disabled to
see if I can stop the memory creep. Maybe it's something in the code
of the Web UI itself as its even slow when changing from page to page
of users and hosts.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Rich Megginson

On 07/16/2012 11:48 AM, Stephen Ingram wrote:

On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson  wrote:

On 07/16/2012 10:19 AM, Stephen Ingram wrote:

On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
wrote:

Stephen Ingram wrote:

On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones
wrote:

Hi,

I had huge memory issues pre 6.3, now its low and flatSounds like
you
have an issue somewhere. My normal cpu use is a few hundred mhzbut
when
"something" goes wrong such as replication failing that climbs...ditto
memory use


Yes, I saw your conversation with Rich on this list about that. And,
yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
complete without issue. I'm also not even doing replication yet so I'm
not sure why memory is so high. Web interface is much slower too so
perhaps something else is wrong.


Can you tell where it is being slow? Does it seem related to retrieving
data
from LDAP?

I'm not really sure yet what is causing the slowness. I have the same
number of directory entries as before the upgrade. It was very quick
with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
days--without a doubt much, much slower.


You might check your 389-ds access logs and look for searches with
notes=U.
Perhaps you are missing an index.

Yes there are lots of notes=U. What does this mean? Was something
missed in the upgrade script?

Try running logconv.pl

Nice! I'm guessing that notes=U are unindexed searches then. I have 34
over the last 24 hours so I'm not sure this would be causing the issue
as the slowness persists through every click.

Yeah, I would expect to see a lot more than 34 if that were the cause.

Can you post the search filters that are unindexed?


I've traced the
unindexed searches back to the time of Web UI access and they don't
match. I also don't see any other obvious errors when running
logconv.pl.

One strange thing I have noticed is that the 389 server logs seem to
update in "spurts". If I'm tailing the logs while I access a Web UI
page, there is nothing, then a couple of seconds later, I see the logs
quickly scroll with new entires. Has this always been the case? I
don't seem to remember this before.

Yes.  The 389 access log is buffered, for performance reasons.


Steve


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Stephen Ingram
On Mon, Jul 16, 2012 at 9:35 AM, Rich Megginson  wrote:
> On 07/16/2012 10:19 AM, Stephen Ingram wrote:
>>
>> On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden
>> wrote:
>>>
>>> Stephen Ingram wrote:

 On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones
 wrote:
>
> Hi,
>
> I had huge memory issues pre 6.3, now its low and flatSounds like
> you
> have an issue somewhere. My normal cpu use is a few hundred mhzbut
> when
> "something" goes wrong such as replication failing that climbs...ditto
> memory use


 Yes, I saw your conversation with Rich on this list about that. And,
 yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
 having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
 complete without issue. I'm also not even doing replication yet so I'm
 not sure why memory is so high. Web interface is much slower too so
 perhaps something else is wrong.
>>>
>>>
>>> Can you tell where it is being slow? Does it seem related to retrieving
>>> data
>>> from LDAP?
>>
>> I'm not really sure yet what is causing the slowness. I have the same
>> number of directory entries as before the upgrade. It was very quick
>> with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
>> days--without a doubt much, much slower.
>>
>>> You might check your 389-ds access logs and look for searches with
>>> notes=U.
>>> Perhaps you are missing an index.
>>
>> Yes there are lots of notes=U. What does this mean? Was something
>> missed in the upgrade script?
>
> Try running logconv.pl

Nice! I'm guessing that notes=U are unindexed searches then. I have 34
over the last 24 hours so I'm not sure this would be causing the issue
as the slowness persists through every click. I've traced the
unindexed searches back to the time of Web UI access and they don't
match. I also don't see any other obvious errors when running
logconv.pl.

One strange thing I have noticed is that the 389 server logs seem to
update in "spurts". If I'm tailing the logs while I access a Web UI
page, there is nothing, then a couple of seconds later, I see the logs
quickly scroll with new entires. Has this always been the case? I
don't seem to remember this before.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Rich Megginson

On 07/16/2012 10:19 AM, Stephen Ingram wrote:

On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden  wrote:

Stephen Ingram wrote:

On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones
wrote:

Hi,

I had huge memory issues pre 6.3, now its low and flatSounds like you
have an issue somewhere. My normal cpu use is a few hundred mhzbut when
"something" goes wrong such as replication failing that climbs...ditto
memory use


Yes, I saw your conversation with Rich on this list about that. And,
yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
complete without issue. I'm also not even doing replication yet so I'm
not sure why memory is so high. Web interface is much slower too so
perhaps something else is wrong.


Can you tell where it is being slow? Does it seem related to retrieving data
from LDAP?

I'm not really sure yet what is causing the slowness. I have the same
number of directory entries as before the upgrade. It was very quick
with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
days--without a doubt much, much slower.


You might check your 389-ds access logs and look for searches with notes=U.
Perhaps you are missing an index.

Yes there are lots of notes=U. What does this mean? Was something
missed in the upgrade script?

Try running logconv.pl


Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-16 Thread Stephen Ingram
On Fri, Jul 13, 2012 at 6:14 AM, Rob Crittenden  wrote:
> Stephen Ingram wrote:
>>
>> On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones 
>> wrote:
>>>
>>> Hi,
>>>
>>> I had huge memory issues pre 6.3, now its low and flatSounds like you
>>> have an issue somewhere. My normal cpu use is a few hundred mhzbut when
>>> "something" goes wrong such as replication failing that climbs...ditto
>>> memory use
>>
>>
>> Yes, I saw your conversation with Rich on this list about that. And,
>> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
>> having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
>> complete without issue. I'm also not even doing replication yet so I'm
>> not sure why memory is so high. Web interface is much slower too so
>> perhaps something else is wrong.
>
>
> Can you tell where it is being slow? Does it seem related to retrieving data
> from LDAP?

I'm not really sure yet what is causing the slowness. I have the same
number of directory entries as before the upgrade. It was very quick
with 2.1.3, but once I upgraded, I felt like I was back to the pre-2.0
days--without a doubt much, much slower.

> You might check your 389-ds access logs and look for searches with notes=U.
> Perhaps you are missing an index.

Yes there are lots of notes=U. What does this mean? Was something
missed in the upgrade script?

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-13 Thread Rob Crittenden

Stephen Ingram wrote:

On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  wrote:

Hi,

I had huge memory issues pre 6.3, now its low and flatSounds like you have an issue 
somewhere. My normal cpu use is a few hundred mhzbut when "something" goes 
wrong such as replication failing that climbs...ditto memory use


Yes, I saw your conversation with Rich on this list about that. And,
yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
complete without issue. I'm also not even doing replication yet so I'm
not sure why memory is so high. Web interface is much slower too so
perhaps something else is wrong.


Can you tell where it is being slow? Does it seem related to retrieving 
data from LDAP?


You might check your 389-ds access logs and look for searches with 
notes=U. Perhaps you are missing an index.


rob

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Dmitri Pal
On 07/12/2012 06:55 PM, Stephen Ingram wrote:
> On Thu, Jul 12, 2012 at 3:41 PM, Dmitri Pal  wrote:
>> On 07/12/2012 06:19 PM, Stephen Ingram wrote:
>>> On Thu, Jul 12, 2012 at 3:10 PM, Stephen Ingram  wrote:
 On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  
 wrote:
> Hi,
>
> I had huge memory issues pre 6.3, now its low and flatSounds like you 
> have an issue somewhere. My normal cpu use is a few hundred mhzbut 
> when "something" goes wrong such as replication failing that 
> climbs...ditto memory use
 Yes, I saw your conversation with Rich on this list about that. And,
 yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
 having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
 complete without issue. I'm also not even doing replication yet so I'm
 not sure why memory is so high. Web interface is much slower too so
 perhaps something else is wrong.
>>> Oops, I meant Rob, not Rich.
>> Do you use any things exposed via compat tree?
>> Do you have a lot of modifications that affect the data that is exposed
>> via this tree?
>> I suspect that the leak is somewhere there.
>>
>> Try turning off the things that you do not use if there are any.
> I only query cn=users,cn=accounts,dc=example,dc=com and
> cn=groups,cn=accounts,dc=example,dc=com containers for mail info and
> use for Kerberos auth. There are only very infrequent mods to those
> trees previously mentioned via the Web UI. Almost all activity is
> reads, but lots of them for the mail servers (validating users, etc.).
> I plan to use replication and DNS, but not using now. From this, I
> don't think I'm using the compat tree. Would turning it off help
> anyway?
>
> Steve
Please use documented tools to disable it. Do not do it manually via LDAP.

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Dmitri Pal
On 07/12/2012 06:55 PM, Stephen Ingram wrote:
> On Thu, Jul 12, 2012 at 3:41 PM, Dmitri Pal  wrote:
>> On 07/12/2012 06:19 PM, Stephen Ingram wrote:
>>> On Thu, Jul 12, 2012 at 3:10 PM, Stephen Ingram  wrote:
 On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  
 wrote:
> Hi,
>
> I had huge memory issues pre 6.3, now its low and flatSounds like you 
> have an issue somewhere. My normal cpu use is a few hundred mhzbut 
> when "something" goes wrong such as replication failing that 
> climbs...ditto memory use
 Yes, I saw your conversation with Rich on this list about that. And,
 yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
 having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
 complete without issue. I'm also not even doing replication yet so I'm
 not sure why memory is so high. Web interface is much slower too so
 perhaps something else is wrong.
>>> Oops, I meant Rob, not Rich.
>> Do you use any things exposed via compat tree?
>> Do you have a lot of modifications that affect the data that is exposed
>> via this tree?
>> I suspect that the leak is somewhere there.
>>
>> Try turning off the things that you do not use if there are any.
> I only query cn=users,cn=accounts,dc=example,dc=com and
> cn=groups,cn=accounts,dc=example,dc=com containers for mail info and
> use for Kerberos auth. There are only very infrequent mods to those
> trees previously mentioned via the Web UI. Almost all activity is
> reads, but lots of them for the mail servers (validating users, etc.).
> I plan to use replication and DNS, but not using now. From this, I
> don't think I'm using the compat tree. Would turning it off help
> anyway?
>
> Steve
It is worth a try. There is a known slow minor leak in the compat plugin.

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Stephen Ingram
On Thu, Jul 12, 2012 at 3:41 PM, Dmitri Pal  wrote:
> On 07/12/2012 06:19 PM, Stephen Ingram wrote:
>> On Thu, Jul 12, 2012 at 3:10 PM, Stephen Ingram  wrote:
>>> On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  
>>> wrote:
 Hi,

 I had huge memory issues pre 6.3, now its low and flatSounds like you 
 have an issue somewhere. My normal cpu use is a few hundred mhzbut 
 when "something" goes wrong such as replication failing that 
 climbs...ditto memory use
>>> Yes, I saw your conversation with Rich on this list about that. And,
>>> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
>>> having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
>>> complete without issue. I'm also not even doing replication yet so I'm
>>> not sure why memory is so high. Web interface is much slower too so
>>> perhaps something else is wrong.
>> Oops, I meant Rob, not Rich.
>
> Do you use any things exposed via compat tree?
> Do you have a lot of modifications that affect the data that is exposed
> via this tree?
> I suspect that the leak is somewhere there.
>
> Try turning off the things that you do not use if there are any.

I only query cn=users,cn=accounts,dc=example,dc=com and
cn=groups,cn=accounts,dc=example,dc=com containers for mail info and
use for Kerberos auth. There are only very infrequent mods to those
trees previously mentioned via the Web UI. Almost all activity is
reads, but lots of them for the mail servers (validating users, etc.).
I plan to use replication and DNS, but not using now. From this, I
don't think I'm using the compat tree. Would turning it off help
anyway?

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Dmitri Pal
On 07/12/2012 06:19 PM, Stephen Ingram wrote:
> On Thu, Jul 12, 2012 at 3:10 PM, Stephen Ingram  wrote:
>> On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  wrote:
>>> Hi,
>>>
>>> I had huge memory issues pre 6.3, now its low and flatSounds like you 
>>> have an issue somewhere. My normal cpu use is a few hundred mhzbut when 
>>> "something" goes wrong such as replication failing that climbs...ditto 
>>> memory use
>> Yes, I saw your conversation with Rich on this list about that. And,
>> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
>> having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
>> complete without issue. I'm also not even doing replication yet so I'm
>> not sure why memory is so high. Web interface is much slower too so
>> perhaps something else is wrong.
> Oops, I meant Rob, not Rich.

Do you use any things exposed via compat tree?
Do you have a lot of modifications that affect the data that is exposed
via this tree?
I suspect that the leak is somewhere there.

Try turning off the things that you do not use if there are any.


> Steve
>
> ___
> Freeipa-users mailing list
> Freeipa-users@redhat.com
> https://www.redhat.com/mailman/listinfo/freeipa-users


-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.


---
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/



___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Stephen Ingram
On Thu, Jul 12, 2012 at 3:10 PM, Stephen Ingram  wrote:
> On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  wrote:
>> Hi,
>>
>> I had huge memory issues pre 6.3, now its low and flatSounds like you 
>> have an issue somewhere. My normal cpu use is a few hundred mhzbut when 
>> "something" goes wrong such as replication failing that climbs...ditto 
>> memory use
>
> Yes, I saw your conversation with Rich on this list about that. And,
> yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
> having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
> complete without issue. I'm also not even doing replication yet so I'm
> not sure why memory is so high. Web interface is much slower too so
> perhaps something else is wrong.

Oops, I meant Rob, not Rich.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Stephen Ingram
On Thu, Jul 12, 2012 at 2:59 PM, Steven Jones  wrote:
> Hi,
>
> I had huge memory issues pre 6.3, now its low and flatSounds like you 
> have an issue somewhere. My normal cpu use is a few hundred mhzbut when 
> "something" goes wrong such as replication failing that climbs...ditto memory 
> use

Yes, I saw your conversation with Rich on this list about that. And,
yes, 6.2 (2.1.3) was bad for me too. I'm not sure why 2.2.0 is still
having issues. It was an upgrade from 2.1.3, but the upgrade seemed to
complete without issue. I'm also not even doing replication yet so I'm
not sure why memory is so high. Web interface is much slower too so
perhaps something else is wrong.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


Re: [Freeipa-users] 2.20 dirsrv memory usage

2012-07-12 Thread Steven Jones
Hi,

I had huge memory issues pre 6.3, now its low and flatSounds like you have 
an issue somewhere. My normal cpu use is a few hundred mhzbut when 
"something" goes wrong such as replication failing that climbs...ditto memory 
use

regards

Steven Jones

Technical Specialist - Linux RHCE

Victoria University, Wellington, NZ

0064 4 463 6272


From: freeipa-users-boun...@redhat.com [freeipa-users-boun...@redhat.com] on 
behalf of Stephen Ingram [sbing...@gmail.com]
Sent: Friday, 13 July 2012 9:29 a.m.
To: freeipa-users
Subject: [Freeipa-users] 2.20 dirsrv memory usage

I was previously using 2.1.4 and know that there was a substantial
memory leak in the directory server. After upgrading to 2.20, I notice
that although overall memory usage seems higher, the "creep" upwards
is not as quick. Although memory still tends to trend upward leaving
me to worry that dirsrv will crash when it runs out of memory. I've
checked the entrycachehitratio and it is 99. I also then checked the
size of id2entry.db4 and found it to be 1024000. So I then checked
nssldap-cachesize and found it to be 10485760. According to what I've
read on the list, this seems about right. Is there anything else I can
check? This is a pretty small directory, but gets quite a bit of
activity from serving mail configuration in addition to
authentication. However, I can't imagine that it would consume 1.5GB
and keep climbing in memory usage.

Steve

___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users


___
Freeipa-users mailing list
Freeipa-users@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-users