Re: [Freeipa-users] backing up and starting over...
On Thu, 22 Dec 2016 16:48:10 -0500 Robert wrote: RS> I tried to create a replica. It went well for the directory server, but RS> then: RS> RS> Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 30 RS> seconds [1/27]: creating certificate server user RS> [2/27]: configuring certificate server instance RS> ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to configure RS> CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpqYyqJJ' returned RS> non-zero exit status 1 ipa.ipaserver.install.cainstance.CAInstance: RS> CRITICAL See the installation logs and the following files/directories for RS> more information: ipa.ipaserver.install.cainstance.CAInstance: RS> CRITICAL /var/log/pki/pki-tomcat [error] RuntimeError: CA configuration RS> failed. RS> [...] RS> So this looks like the culprit: RS> RS> [22/Dec/2016:16:07:48][http-bio-8443-exec-3]: updateNumberRange: Failed to contact master using admin portjavax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error So eventually I found proxy errors like this in a logfile: proxy_ajp:error (70007)The timeout specified has expired: I added large timeouts to /etc/httpd/conf.d/ipa-pki-proxy.conf Timeout 900 ProxyTimeout 900 This allowed my replica install to complete. However, when I logged in to the new replica, I was getting the same long timeout trying to load users. The error log had this: [Fri Dec 23 00:50:39.206858 2016] [proxy_ajp:error] [pid 31182] [client 10.71.10.118:49784] AH00896: failed to make connection to backend: localhost This started ringing a little bell in my head about localhost and ipv4 vs ipv6. I disabled ipv6 in /etc/sysctl.conf, and voila, users load in less than 5 seconds instead of 5 minutes or timing out. Hopefully this will also resolve the other weirdness I've been seeing. I'm keeping my fingers crossed. Robert -- Senior Software Engineer @ Parsons pgpqGB0jo68SB.pgp Description: OpenPGP digital signature -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] backing up and starting over...
On Thu, 22 Dec 2016 09:25:52 +0100 Florence wrote:
FBR> you can find more information about backup and restore procedure in this
FBR> guide [1]. But, as stated in the documentation, the safest method would
FBR> rather be to install a replica [2].
FBR> [...]
FBR> [2]
FBR>
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/install-replica.html
I tried to create a replica. It went well for the directory server, but
then:
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 30
seconds [1/27]: creating certificate server user
[2/27]: configuring certificate server instance
ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to configure
CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpqYyqJJ' returned
non-zero exit status 1 ipa.ipaserver.install.cainstance.CAInstance:
CRITICAL See the installation logs and the following files/directories for
more information: ipa.ipaserver.install.cainstance.CAInstance:
CRITICAL /var/log/pki/pki-tomcat [error] RuntimeError: CA configuration
failed.
from ipa-replica-install.log:
2016-12-22T21:00:53Z DEBUG Starting external process
2016-12-22T21:00:53Z DEBUG args=/usr/sbin/pkispawn -s CA -f /tmp/tmpqYyqJJ
2016-12-22T21:10:08Z DEBUG Process finished, return code=1
2016-12-22T21:10:08Z DEBUG stdout=Log file:
/var/log/pki/pki-ca-spawn.20161222160055.log
Loading deployment configuration from /tmp/tmpqYyqJJ.
Installing CA into /var/lib/pki/pki-tomcat.
Storing deployment configuration into
/etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg.
Importing certificates from /tmp/ca.p12:
...
Import complete
---
Imported certificates in /etc/pki/pki-tomcat/alias:
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
ocspSigningCert cert-pki-ca u,u,u
subsystemCert cert-pki-cau,u,u
caSigningCert cert-pki-caCTu,Cu,Cu
Server-Cert cert-pki-ca u,u,u
auditSigningCert cert-pki-ca u,u,Pu
Installation failed:
Please check the CA logs in /var/log/pki/pki-tomcat/ca.
2016-12-22T21:10:08Z DEBUG stderr=
2016-12-22T21:10:08Z CRITICAL Failed to configure CA instance: Command
'/usr/sbin/pkispawn -s CA -f /tmp/tmpqYyqJJ' returned non-zero exit status 1
2016-12-22T21:10:08Z CRITICAL See the installation logs and the following
files/directories for more information:
2016-12-22T21:10:08Z CRITICAL /var/log/pki/pki-tomcat
2016-12-22T21:10:08Z DEBUG Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
448, in start_creation
run_step(full_msg, method)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
438, in run_step
method()
File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line
590, in __spawn_instance
DogtagInstance.spawn_instance(self, cfg_file)
File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py",
line 181, in spawn_instance
self.handle_setup_error(e)
File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py",
line 420, in handle_setup_error
raise RuntimeError("%s configuration failed." % self.subsystem)
RuntimeError: CA configuration failed.
2016-12-22T21:10:08Z DEBUG [error] RuntimeError: CA configuration failed.
2016-12-22T21:10:08Z DEBUG File
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute
return_value = self.run()
File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318,
in run
cfgr.run()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310,
in run
self.execute()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332,
in execute
for nothing in self._executor():
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372,
in __runner
self._handle_exception(exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394,
in _handle_exception
six.reraise(*exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362,
in __runner
step()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359,
in
step = lambda: next(self.__gen)
File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81,
in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59,
in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 586,
in _configure
next(executor)
File
Re: [Freeipa-users] backing up and starting over...
On Thu, 22 Dec 2016 13:02:18 +0100 Martin wrote: MB> On 22.12.2016 09:25, Florence Blanc-Renaud wrote: MB> > On 12/21/2016 10:26 PM, Robert Story wrote: MB> >> I'm running a small instance of freeipa on CentOS 7 in our lab, for MB> >> about 20 MB> >> machines. Since CentOS 7.3 came out and upgraded from 4.2 to 4.4, things MB> >> have gotten flaky. e.g. clicking on a user get the spinning 'Working' MB> >> dialog and can take 3-5 minutes to load the page. But often it will die MB> >> with 'internal error'. MB> MB> Could you check in /var/log/httpd/error_log what is it? MB> Does cli work well? ipa user-find Yes, cli works, and ldap mostly works, but not always. GUI works occasionally. Here's one: mod_wsgi (pid=6358): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'. Traceback (most recent call last): File "/usr/share/ipa/wsgi.py", line 49, in application return api.Backend.wsgi_dispatch(environ, start_response) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 262, in __call__ return self.route(environ, start_response) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 274, in route return app(environ, start_response) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 833, in __call__ self.create_context(ccache=ipa_ccache_name) File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 123, in create_context self.Backend.ldap2.connect(ccache=ccache) File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 66, in connect conn = self.create_connection(*args, **kw) File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 205, in create_connection client_controls=clientctrls) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1108, in gssapi_bind '', auth_tokens, server_controls, client_controls) File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ self.gen.throw(type, value, traceback) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1007, in error_handler raise errors.DatabaseError(desc=desc, info=info) DatabaseError: Server is unwilling to perform: Too many failed logins. and this: ipa: INFO: 401 Unauthorized: kinit: Clients credentials have been revoked while getting initial credentials and ipa: ERROR: non-public: IOError: request data read error Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 358, in wsgi_execute data = read_input(environ) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 195, in read_input return environ['wsgi.input'].read(length) IOError: request data read error rstory@EXAMPLE: None: IOError and AH00163: Apache/2.4.6 (CentOS) mod_auth_gssapi/1.4.0 mod_nss/1.0.14 NSS/3.21 Basic ECC mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' ipa: INFO: *** PROCESS START *** ipa: INFO: *** PROCESS START *** ipa: INFO: 401 Unauthorized: kinit: Cannot contact any KDC for realm 'EXAMPLE' while getting initial credentials [pid 3714] ipa: INFO: 401 Unauthorized: kinit: Cannot contact any KDC for realm 'EXAMPLE' while getting initial credentials [pid 3715] ipa: ERROR: release_ipa_ccache: ccache_name (FILE:/var/run/ipa_memcached/krbcc_3714) != KRB5CCNAME environment variable (/var/run/httpd/ipa/krbcache/krb5ccache) ipa: INFO: 401 Unauthorized: Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Cannot contact any KDC for realm 'EXAMPLE') mod_wsgi (pid=3714): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'. Traceback (most recent call last): File "/usr/share/ipa/wsgi.py", line 49, in application return api.Backend.wsgi_dispatch(environ, start_response) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 262, in __call__ return self.route(environ, start_response) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 274, in route return app(environ, start_response) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 978, in __call__ self.kinit(user, self.api.env.realm, password, ipa_ccache_name) File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 1010, in kinit raise CCacheError(message=unicode(e)) CCacheError: Major (851968): Unspecified GSS failure. Minor code may provide more information, Minor (2529639068): Cannot contact any KDC for realm 'EXAMPLE' AH00170: caught SIGWINCH, shutting down gracefully and Script timed out before returning headers: wsgi.py, referer: https://auth-1.example/ipa/ui/ Script timed out before returning headers: wsgi.py, referer: https://auth-1.example/ipa/ui/ Script timed out before returning headers: wsgi.py, referer: https://auth-1.example/ipa/ui/ and SSL Library Error: -12195 Peer does
Re: [Freeipa-users] backing up and starting over...
On 12/21/2016 10:26 PM, Robert Story wrote: I'm running a small instance of freeipa on CentOS 7 in our lab, for about 20 machines. Since CentOS 7.3 came out and upgraded from 4.2 to 4.4, things have gotten flaky. e.g. clicking on a user get the spinning 'Working' dialog and can take 3-5 minutes to load the page. But often it will die with 'internal error'. Is there a way to back up data so that I can re-install 4.4 and restore the data? Specifically users+uids/groups+gids, HBAC and sudo rules? Robert Hi, you can find more information about backup and restore procedure in this guide [1]. But, as stated in the documentation, the safest method would rather be to install a replica [2]. HTH, Flo [1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/backup-restore.html [2] https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/install-replica.html -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
