That was it. Thank you so much. I can get in the webgui again and I can get into spacecmd. Everything looks good again.
You just made my weekend. I can’t thank you enough From: [email protected] [mailto:[email protected]] On Behalf Of Schindler, Daniel (STEAG Energy Services GmbH) Sent: Friday, September 1, 2017 10:08 AM To: [email protected]; [email protected] Subject: Re: [Spacewalk-list] can't log in to web gui or run spacecmd Have you also changed the db password in /etc/rhn/rhn.conf after changing users password via sqlplus/em? Regards, Daniel ------ Originalnachricht ------ Von: "Robin Beardsley" <[email protected]<mailto:[email protected]>> An: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Gesendet: 01.09.2017 15:41:08 Betreff: Re: [Spacewalk-list] can't log in to web gui or run spacecmd I went a step further and discovered if I shut down all spacewalk services, connect to the database or sysdba, run the commands below to unlock/reset the spacewalk user password, the spacewalk account will then show as open in enterprise manager but as soon as I start up the spacewalk processes, it immediately locks the account again and it shows as LOCKED(TIMED). I tried setting the spacewalk user’s password to a different password and then set it back to the original password with no success. At this point my server is not usable, I’m not getting syncs, none of my clients can connect, etc. Has anyone run into this? Please help From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Robin Beardsley Sent: Friday, September 1, 2017 8:08 AM To: [email protected]<mailto:[email protected]> Subject: Re: [Spacewalk-list] can't log in to web gui or run spacecmd Thanks so much for your reply. I did all of the below command as sysdba. I then restarted spacewalk and the database. However I still can’t connect to the webgui or spacecmd. In my Oracle Enterprise Manager (Express Edition) the spacewalk user shows as LOCKED(TIMED) and the SYSTEM user shows as EXPIRED(GRACE) Any idea on how to fix this? Thanks From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Schindler, Daniel (STEAG Energy Services GmbH) Sent: Friday, September 1, 2017 4:09 AM To: [email protected]<mailto:[email protected]> Subject: Re: [Spacewalk-list] can't log in to web gui or run spacecmd Hi Robin, I assume you have a standalone database setup and you have access to the database. We ran into such problem about two weeks ago. The problem was that the password for spacewalk db user was expired. Here are the basic steps we performed: [oracle@s01spw02 ~]$ sqlplus "/ as sysdba" SQL> alter user <your_spacewalk_db_user> account unlock; SQL> alter user <your_spacewalk_db_user> identified by newpa$$word; SQL> alter profile DEFAULT limit PASSWORD_REUSE_TIME unlimited; SQL> alter profile DEFAULT limit PASSWORD_LIFE_TIME unlimited; I know this might be a security issue to disable password expiration in general. I will also write Oracle a service request to add this to their official documentation or at least provide different ways to adjust password expiration policies. Regards, Daniel Von: [email protected]<mailto:[email protected]> [mailto:[email protected]] Im Auftrag von Robin Beardsley Gesendet: Donnerstag, 31. August 2017 21:04 An: [email protected]<mailto:[email protected]> Betreff: Re: [Spacewalk-list] can't log in to web gui or run spacecmd No, c3p0 and cglib have not been updated. Currently they are at [rbeards@oelsandbox ~]$ rpm -qa | grep c3p0 c3p0-0.9.1.2-2.jpp5.noarch [rbeards@oelsandbox ~]$ rpm -qa | grep cglib cglib-2.1.3-4.jpp5.noarch I noticed complaints in http errors logs about password expiration. If this is the real cause, does anyone know how to fix it? I have an oracle 12 database but I am not a dba. [Thu Aug 31 14:39:08.592224 2017] [:error] [pid 34501] Spacewalk 34501 2017/08/31 14:39:08 -04:00: ("DATABASE CONNECTION TO 'spacewalk@//localhost/orcl.nedelta.com' LOST", "Exception information: 'NoneType' object has no attribute 'cursor'") [Thu Aug 31 14:39:08.607568 2017] [:error] [pid 34501] Spacewalk 34501 2017/08/31 14:39:08 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') [Thu Aug 31 14:39:37.653577 2017] [:error] [pid 33509] Spacewalk 33509 2017/08/31 14:39:37 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') [Thu Aug 31 14:41:42.794240 2017] [:error] [pid 33516] Spacewalk 33516 2017/08/31 14:41:42 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') [Thu Aug 31 14:41:57.138221 2017] [mpm_prefork:notice] [pid 33508] AH00170: caught SIGWINCH, shutting down gracefully [Thu Aug 31 14:43:10.838257 2017] [suexec:notice] [pid 1719] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Thu Aug 31 14:43:10.848720 2017] [:notice] [pid 1719] ModSecurity for Apache/2.7.3 (http://www.modsecurity.org/<http://www.modsecurity.org/>) configured. [Thu Aug 31 14:43:10.848731 2017] [:notice] [pid 1719] ModSecurity: APR compiled version="1.4.8"; loaded version="1.4.8" [Thu Aug 31 14:43:10.848734 2017] [:notice] [pid 1719] ModSecurity: PCRE compiled version="8.32 "; loaded version="8.32 2012-11-30" [Thu Aug 31 14:43:10.848736 2017] [:notice] [pid 1719] ModSecurity: LUA compiled version="Lua 5.1" [Thu Aug 31 14:43:10.848737 2017] [:notice] [pid 1719] ModSecurity: LIBXML compiled version="2.9.1" [Thu Aug 31 14:43:10.893313 2017] [auth_digest:notice] [pid 1719] AH01757: generating secret for digest authentication ... [Thu Aug 31 14:43:10.893804 2017] [lbmethod_heartbeat:notice] [pid 1719] AH02282: No slotmem from mod_heartmonitor [Thu Aug 31 14:43:10.914765 2017] [mpm_prefork:notice] [pid 1719] AH00163: Apache/2.4.6 () OpenSSL/1.0.2k-fips mod_wsgi/3.4 Python/2.7.5 configured -- resuming normal operations [Thu Aug 31 14:43:10.914790 2017] [core:notice] [pid 1719] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND' [Thu Aug 31 14:43:11.177830 2017] [proxy:error] [pid 1888] (111)Connection refused: AH00957: AJP: attempt to connect to 127.0.0.1:8009 (localhost) failed [Thu Aug 31 14:43:11.177859 2017] [proxy:error] [pid 1888] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 60s [Thu Aug 31 14:43:11.177879 2017] [proxy_ajp:error] [pid 1888] [client 127.0.0.1:36349] AH00896: failed to make connection to backend: localhost [Thu Aug 31 14:44:05.825586 2017] [:error] [pid 1890] Spacewalk 1890 2017/08/31 14:44:05 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') [Thu Aug 31 14:44:15.398432 2017] [:error] [pid 1895] Spacewalk 1895 2017/08/31 14:44:15 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') [Thu Aug 31 14:44:41.161174 2017] [:error] [pid 1891] Spacewalk 1891 2017/08/31 14:44:41 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') [Thu Aug 31 14:55:00.796406 2017] [:error] [pid 1895] Spacewalk 1895 2017/08/31 14:55:00 -04:00: ("DATABASE CONNECTION TO 'spacewalk@//localhost/orcl.nedelta.com' LOST", "Exception information: 'NoneType' object has no attribute 'cursor'") [Thu Aug 31 14:55:00.814582 2017] [:error] [pid 1895] Spacewalk 1895 2017/08/31 14:55:00 -04:00: ('Connection attempt failed', 28001, 'ORA-28001: the password has expired\\n') From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Thomas Foster Sent: Thursday, August 31, 2017 1:53 PM To: [email protected]<mailto:[email protected]> Subject: Re: [Spacewalk-list] can't log in to web gui or run spacecmd Robin, Did it update c3p0 and cglib? If so that could be what's causing your server not to start. What version are you on? Also check it out could help you identify the issue: https://bugzilla.redhat.com/show_bug.cgi?id=1450906<https://bugzilla.redhat.com/show_bug.cgi?id=1450906> On Thu, Aug 31, 2017 at 1:25 PM, Robin Beardsley <[email protected]<mailto:[email protected]>> wrote: Today I tried to log in to my SW server and could not get in. I also tried to get in via spacecmd and could not get in. This is from the rhn_taskomatic_daemon.log INFO | jvm 5 | 2017/08/31 12:43:15 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org<http://wrapper.tanukisoftware.org> INFO | jvm 5 | 2017/08/31 12:43:15 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. INFO | jvm 5 | 2017/08/31 12:43:15 | ERROR | wrapper | 2017/08/31 12:43:44 | Startup failed: Timed out waiting for signal from JVM. ERROR | wrapper | 2017/08/31 12:43:45 | JVM did not exit on request, terminated INFO | wrapper | 2017/08/31 12:43:45 | JVM exited on its own while waiting to kill the application. STATUS | wrapper | 2017/08/31 12:43:45 | JVM exited in response to signal SIGKILL (9). FATAL | wrapper | 2017/08/31 12:43:45 | There were 5 failed launches in a row, each lasting less than 300 seconds. Giving up. FATAL | wrapper | 2017/08/31 12:43:45 | There may be a configuration problem: please check the logs. STATUS | wrapper | 2017/08/31 12:43:45 | <-- Wrapper Stopped [root@oelsandbox rhn]# spacecmd Welcome to spacecmd, a command-line interface to Spacewalk. Type: 'help' for a list of commands 'help <cmd>' for command-specific help 'quit' to quit Traceback (most recent call last): File "/bin/spacecmd", line 166, in <module> if not shell.do_login(''): File "/usr/lib/python2.7/site-packages/spacecmd/misc.py", line 284, in do_login self.api_version = self.client.api.getVersion() File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1587, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1321, in single_request response.msg, xmlrpclib.ProtocolError: <ProtocolError for oelsandbox.nedelta.com/rpc/api<http://oelsandbox.nedelta.com/rpc/api>: 500 Internal Server Error> Everything has been running fine up until today. I did a yum security update yesterday and the only update was a kernel update. I rebooted the server. All spacewalk services were restarted and they all seem to be running. Tail of up2date log [Wed Aug 30 10:52:31 2017] up2date Updating package profile [Wed Aug 30 14:52:30 2017] up2date logging into up2date server [Wed Aug 30 14:52:30 2017] up2date successfully retrieved authentication token from up2date server [Wed Aug 30 18:52:30 2017] up2date logging into up2date server [Wed Aug 30 18:52:30 2017] up2date successfully retrieved authentication token from up2date server [Wed Aug 30 22:52:31 2017] up2date logging into up2date server [Wed Aug 30 22:52:31 2017] up2date successfully retrieved authentication token from up2date server [Thu Aug 31 02:52:31 2017] up2date logging into up2date server [Thu Aug 31 02:52:31 2017] up2date successfully retrieved authentication token from up2date server [Thu Aug 31 06:52:31 2017] up2date logging into up2date server [Thu Aug 31 06:52:31 2017] up2date successfully retrieved authentication token from up2date server [Thu Aug 31 12:32:52 2017] up2date logging into up2date server [Thu Aug 31 12:32:53 2017] up2date A protocol error occurred: Internal Server Error , attempt #1, [Thu Aug 31 12:35:25 2017] up2date logging into up2date server [Thu Aug 31 12:35:32 2017] up2date A protocol error occurred: Internal Server Error , attempt #1, [Thu Aug 31 12:35:48 2017] up2date logging into up2date server [Thu Aug 31 12:36:18 2017] up2date A protocol error occurred: Internal Server Error , attempt #1, [Thu Aug 31 12:48:08 2017] up2date logging into up2date server [Thu Aug 31 12:48:38 2017] up2date A protocol error occurred: Internal Server Error , attempt #1, [Thu Aug 31 12:55:08 2017] up2date logging into up2date server [Thu Aug 31 12:55:35 2017] up2date A protocol error occurred: Internal Server Error , attempt #1, [root@oelsandbox log]# uname -r 3.8.13-118.19.4.el7uek.x86_64 [root@oelsandbox log]# uptrack-uname -r 3.8.13-118.19.4.el7uek.x86_64 Does anyone have any idea what might be the cause of this? Thank you _______________________________________________ Spacewalk-list mailing list [email protected]<mailto:[email protected]> https://www.redhat.com/mailman/listinfo/spacewalk-list<https://www.redhat.com/mailman/listinfo/spacewalk-list>
_______________________________________________ Spacewalk-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/spacewalk-list
