On Wed, July 25, 2012 09:54, Sigbjorn Lie wrote:
> On Tue, July 24, 2012 20:29, Simo Sorce wrote:
>> On Tue, 2012-07-24 at 10:22 +0200, Sigbjorn Lie wrote:
>>> I keep seing this error message in our production environment "Request is a
>>> replay" in
>>> variuos services using kerberos like ssh, sssd, automounter, squid +++
>>> after the upgrade to
>>> RHEL 6.3 /
>>> Jul 24 10:16:11 server027 sssd_be: GSSAPI Error: Unspecified GSS failure.
>>> Minor code may
>>> provide more information (Request is a replay)
>>> Seaching google seem to suggest that this is an error with time. However we
>>> have NTP
>>> configured (IPA servers as NTP servers) which is synchronized to external
>>> NTP servers. There
>>> has been no issue before, and I cannot find issue with the time being out
>>> of sync on the
>>> machines where this is happening.
>> This error usually appears only when a same request is found in the
>> replay cache. It shouldn't be related to time issues, in that case you
>> usually get clock-skew.
>> Can you tell me what operation was being performed by sssd when you
>> caught that error ? Can you check if immediately before another identical
>> operation had been
>> performed ?
> That being said, I do have 1 IPA server (out of 3) that has significantly
> higher CPU usage than
> the other 2, the 15-minute load average is sitting at between 0.85 and 0.95
> the entire day, where
> ns-slapd 389-ds process is running at 100% most of the time.
> Load: 1.02, 0.94, 0.87
> In comparison the other two IPA servers has a 15-minute average between 0.10
> - 0.30 throughout
> the day, and the ns-slapd process is far from being such a cpu hog.
> On the server having high load, running even a command such as "ipactl
> status" can take up to 20
> seconds to complete, where "Directory Service: RUNNING" returns after a
> second or so, and to list
> the rest of the services takes the remainding 19 seconds.
> Also the web interface on this particular IPA server is rendered unusable,
> returning "Limits
> exceeded for the query" for almost any action.
> Restarting all the IPA servies (ipactl restart) on the problematic host
> soemwhat improves the
> situation, however that particular server returns to having heavy load
> Using logconv.pl to analyze the dirsrv access log file displays that the
> server in question has
> the lowest search queries per min with 106 queries/min. The other servers
> have 710 search
> queries/sec and 168 queries/sec.
> For modifications all the IPA servers has about 5-6 queries/sec. For
> unindexed searches the
> problematic server is the server with the lowest number. It does however have
> more than twice the
> amount of GSSAPI binds than the other servers with over 61000 GSSAPI binds
> over a 17 hour period.
> The problematic server is a physical server with 2 x AMD 2.4GHz Quad core CPU
> and 8GB of RAM.
> This issue is also impacting all the clients, where I see random hangs with
> anything involving a
> ldap or kerberos query to the IPA servers.
> Any suggestions?
I am starting to see the Replay error when using the "ipa" CLI tool as well,
causing the request
to drop out in an error.
ipa dnsrecord-show example.com hostname
ipa: ERROR: Local error: SASL(-1): generic failure: GSSAPI Error: Unspecified
GSS failure. Minor
code may provide more information (Request is a replay)
Freeipa-users mailing list