Yeah, osad/rhnsd don't seem to have any bearing on this problem, turning them on/off and trying again was one of the first things I tried. No change; so it's not like osad is somehow half-picking up the task and then not finishing it.
Worth mentioning, if it's not clear: Spacewalk never sees the task even get acknowledged by the remote system. It just stays at: This action will be executed after 1/27/15 10:28:00 AM CST This action's status is: Queued. This action has not yet been picked up. Of course, brand new VMs I set up to try to replicate this issue work fine. Paulo: That's an idea, but not necessarily a *good* one... :) On Tue, Jan 27, 2015 at 10:29 AM, Paulo Silva <[email protected]> wrote: > I've found an ugly fix for this, all I had to do was to use rhnreg_ks to > re-register the system and now events on the new profile are being picked > up by rhn_check. > > 2015-01-27 16:07 GMT+00:00 Paulo Silva <[email protected]>: > >> Hi Glen, >> >> Can't try that since I'm using rhnsd and not osad. >> >> Thanks >> >> 2015-01-27 15:28 GMT+00:00 Glen Collins <[email protected]>: >> >>> Hi Paulo, >>> >>> Why don't you try this on the client.... >>> >>> service osad stop >>> rm -rf /etc/sysconfig/rhn/osad-auth.conf >>> service osad start >>> (Wait for 2 minutes) and run (On the client) >>> /usr/sbin/rhn_check >>> >>> I've had the same issues and just restarting OSAD didn't do it for me. >>> But when I removed the auth file, that seemed to fix things. Not elegant in >>> any sort of means but it seems to work. Have not tried it long term since >>> I'm putting this "fix" into place right now. I guess this is what happens >>> when you cobble a product together and "make it work". I have found jabber >>> to be a PITA! >>> >>> Anyways, hope this helps! >>> >>> Glen Collins >>> >>> ------------------------------ >>> It's not permissions: >>> >>> [root@zbx-prx-elk-04 ~]# rhn-actions-control --report >>> deploy is enabled >>> diff is enabled >>> upload is enabled >>> mtime_upload is enabled >>> run is enabled >>> >>> It's not SSL either, we have our systems configured to speak in the >>> clear while troubleshooting this issue. Flipping over to HTTPS doesn't >>> change anything. >>> >>> This particular server has three tasks pending now: two config file >>> pushes and one remote command (just to run "date"). Here's the output of >>> rhn_check -vvvv: >>> >>> [root@zbx-prx-elk-04 ~]# rhn_check -vvvv >>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv >>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 >>> D: locked db index /var/lib/rpm/Packages >>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key >>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key >>> D: loading keyring from rpmdb >>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0 >>> D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring >>> D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring >>> D: added key gpg-pubkey-66fd4949-4803fe57 to keyring >>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring >>> D: added key gpg-pubkey-217521f6-45e8a532 to keyring >>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring >>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring >>> D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring >>> D: Using legacy gpg-pubkey(s) from rpmdb >>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 >>> D: do_call packages.checkNeedUpdate('rhnsd=1',){} >>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv >>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 >>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key >>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key >>> D: loading keyring from rpmdb >>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0 >>> D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring >>> D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring >>> D: added key gpg-pubkey-66fd4949-4803fe57 to keyring >>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring >>> D: added key gpg-pubkey-217521f6-45e8a532 to keyring >>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring >>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring >>> D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring >>> D: Using legacy gpg-pubkey(s) from rpmdb >>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 >>> D: closed db index /var/lib/rpm/Providename >>> D: closed db index /var/lib/rpm/Name >>> D: closed db index /var/lib/rpm/Packages >>> D: closed db environment /var/lib/rpm >>> Loaded plugins: fastestmirror, rhnplugin >>> Config time: 0.040 >>> D: login(forceUpdate=False) invoked >>> D: readCachedLogin invoked >>> D: Checking pickled loginInfo, currentTime=1422370609.99, >>> createTime=1422370587.99, expire-offset=3600.0 >>> D: readCachedLogin(): using pickled loginInfo set to expire at >>> 1422374187.99 >>> D: rpcServer: Calling XMLRPC up2date.listChannels >>> Setting up Package Sacks >>> Loading mirror speeds from cached hostfile >>> pkgsack time: 0.071 >>> rpmdb time: 0.000 >>> Loading mirror speeds from cached hostfile >>> repo time: 0.000 >>> D: local action status: (0, 'rpm database not modified since last update >>> (or package list recently updated)', {}) >>> D: rpcServer: Calling XMLRPC registration.welcome_message >>> D: closed db index /var/lib/rpm/Providename >>> D: closed db index /var/lib/rpm/Name >>> D: closed db index /var/lib/rpm/Packages >>> D: closed db environment /var/lib/rpm >>> >>> Paulo: To answer your question about the API, spacecmd calls it >>> extensively; and you can look up pending tasks by ID. >>> >>> spacecmd {SSM:0}> schedule_details 578593 >>> ID: 578593 >>> Action: Remote Command on zbx-prx-elk-04.company.com. >>> User: admincswingler >>> Date: 20150126T16:36:00 >>> >>> Completed: 0 >>> Failed: 0 >>> Pending: 1 >>> >>> Pending Systems >>> --------------- >>> zbx-prx-elk-04.company.com >>> spacecmd {SSM:0}> schedule_details 578588 >>> ID: 578588 >>> Action: Deploy config files to system >>> User: admincswingler >>> Date: 20150126T16:22:00 >>> >>> Completed: 0 >>> Failed: 0 >>> Pending: 1 >>> >>> Pending Systems >>> --------------- >>> zbx-prx-elk-04.company.com >>> spacecmd {SSM:0}> schedule_details 578586 >>> ID: 578586 >>> Action: Deploy config files to system >>> User: admincswingler >>> Date: 20150126T16:18:00 >>> >>> Completed: 0 >>> Failed: 0 >>> Pending: 1 >>> >>> Pending Systems >>> --------------- >>> zbx-prx-elk-04.company.com >>> spacecmd {SSM:0}> >>> >>> Still scratching my head on this one, and getting rather frustrated :) >>> >>> On Tue, Jan 27, 2015 at 7:02 AM, Paulo Silva <[email protected]> wrote: >>> >>>> I'm checking pending events using the web interface (URL >>>> /rhn/systems/details/history/Pending.do?sid=1000010296) and it shows 2 >>>> pending events: and Hardware Refresh and an Errata Update. >>>> >>>> The sid seams to be correct with the server I'm running my commands: >>>> >>>> # grep 1000010296 /etc/sysconfig/rhn/systemid >>>> <value><string>ID-1000010296</string></value> >>>> >>>> Is there a way to check for pending events using the API? >>>> >>>> 2015-01-27 12:45 GMT+00:00 Alexander Innes <[email protected]>: >>>> >>>>> Just to make sure im not thinking of the wrong feture are you trying >>>>> to use to remote commands? >>>>> Systems -> System -> somethin -> remote command (or a different one so >>>>> i can test in our environment :) ) >>>>> >>>>> from that output its not seeing anything as you said :o >>>>> >>>>> On 27 January 2015 at 12:16, Paulo Silva <[email protected]> wrote: >>>>> >>>>>> I'm having the same issue, running rhn_check does this: >>>>>> >>>>>> # rhn_check -vvv >>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv >>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 >>>>>> D: locked db index /var/lib/rpm/Packages >>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key >>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key >>>>>> D: loading keyring from rpmdb >>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0 >>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring >>>>>> D: added key gpg-pubkey-b5c61460-41667588 to keyring >>>>>> D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring >>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring >>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring >>>>>> D: Using legacy gpg-pubkey(s) from rpmdb >>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 >>>>>> D: do_call packages.checkNeedUpdate('rhnsd=1',){} >>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv >>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 >>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key >>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key >>>>>> D: loading keyring from rpmdb >>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0 >>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring >>>>>> D: added key gpg-pubkey-b5c61460-41667588 to keyring >>>>>> D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring >>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring >>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring >>>>>> D: Using legacy gpg-pubkey(s) from rpmdb >>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 >>>>>> D: closed db index /var/lib/rpm/Providename >>>>>> D: closed db index /var/lib/rpm/Name >>>>>> D: closed db index /var/lib/rpm/Packages >>>>>> D: closed db environment /var/lib/rpm >>>>>> Loaded plugins: etckeeper, fastestmirror, rhnplugin >>>>>> Config time: 0.073 >>>>>> D: login(forceUpdate=False) invoked >>>>>> D: readCachedLogin invoked >>>>>> D: Checking pickled loginInfo, currentTime=1422360920.07, >>>>>> createTime=1422360336.74, expire-offset=3600.0 >>>>>> D: readCachedLogin(): using pickled loginInfo set to expire at >>>>>> 1422363936.74 >>>>>> D: rpcServer: Calling XMLRPC up2date.listChannels >>>>>> This system is receiving updates from RHN Classic or Red Hat >>>>>> Satellite. >>>>>> rpmdb time: 0.000 >>>>>> Loading mirror speeds from cached hostfile >>>>>> repo time: 0.002 >>>>>> Setting up Package Sacks >>>>>> pkgsack time: 0.253 >>>>>> D: local action status: (0, 'rpm database not modified since last >>>>>> update (or package list recently updated)', {}) >>>>>> D: rpcServer: Calling XMLRPC registration.welcome_message >>>>>> D: closed db index /var/lib/rpm/Providename >>>>>> D: closed db index /var/lib/rpm/Name >>>>>> D: closed db index /var/lib/rpm/Packages >>>>>> D: closed db environment /var/lib/rpm >>>>>> >>>>>> >>>>>> 2015-01-27 10:53 GMT+00:00 Alexander Innes <[email protected]>: >>>>>> >>>>>>> Schedual the action then on the server do rhn-chck -vvv. It will >>>>>>> tell you what the client's doing, my gut feeling is either SSL errors OR >>>>>>> permsions (rhn-actions-control is it?) >>>>>>> >>>>>>> On 26 January 2015 at 23:16, Chris Swingler <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Hey Spacewalk-list! >>>>>>>> >>>>>>>> I have quite a few systems (I haven't dug into seeing if there is >>>>>>>> any consistency in CentOS versions or the RHN tools) that just plain >>>>>>>> do not >>>>>>>> seem to get tasks back from the Spacewalk server. We're running >>>>>>>> Spacewalk >>>>>>>> 2.2 on CentOS release 6.5 (Final). >>>>>>>> >>>>>>>> I can create a simple task in Spacewalk, like just print the output >>>>>>>> of "date", and an rhn_check never seems to execute it. It shows up >>>>>>>> under >>>>>>>> Events > Pending, the SystemIDs match, but it never moves. >>>>>>>> >>>>>>>> Running a packet capture to watch the transaction, it looks like >>>>>>>> Spacewalk itself isn't even sending the task to the remote system. >>>>>>>> Spacewalk happily replies with an HTTP 200 and an empty payload. >>>>>>>> >>>>>>>> HTTP/1.1 200 OK >>>>>>>> Date: Mon, 26 Jan 2015 22:47:55 GMT >>>>>>>> Server: Apache >>>>>>>> Content-Length: 126 >>>>>>>> X-RHN-Server-Capability: registration.finish_message(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.remaining_subscriptions(1)=1 >>>>>>>> X-RHN-Server-Capability: abrt(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.update_contact_info(1)=1 >>>>>>>> X-RHN-Server-Capability: staging_content(1)=1 >>>>>>>> X-RHN-Server-Capability: applet.has_base_channel(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.smbios(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.extended_update_support(1)=1 >>>>>>>> X-RHN-Server-Capability: rhncfg.filetype.directory(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.update_systemid(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.register_osad(1)=1 >>>>>>>> X-RHN-Server-Capability: registration.delta_packages(1)=1 >>>>>>>> X-RHN-Server-Capability: cpu_sockets(1)=1 >>>>>>>> X-RHN-Server-Capability: ipv6(1)=1 >>>>>>>> X-RHN-Server-Capability: rhncfg.content.base64_decode(1)=1 >>>>>>>> X-RHN-Server-Capability: xmlrpc.packages.extended_profile(1-2)=1 >>>>>>>> X-RHN-Server-Capability: xmlrpc.errata.patch_names(1)=1 >>>>>>>> X-RHN-Server-Capability: xmlrpc.packages.checksums(1)=1 >>>>>>>> X-RHN-Server-Capability: xmlrpc.login.extra_data(1)=1 >>>>>>>> X-RHN-Proxy-Version: 0 >>>>>>>> X-Transport-Info: Extended Capabilities Transport (C) Red Hat, Inc >>>>>>>> (version 2.5.72-1.el6) >>>>>>>> X-RHN-Client-Version: 1 >>>>>>>> Connection: close >>>>>>>> Content-Type: text/xml >>>>>>>> >>>>>>>> <?xml version='1.0'?> >>>>>>>> <methodResponse> >>>>>>>> <params> >>>>>>>> <param> >>>>>>>> <value><string></string></value> >>>>>>>> </param> >>>>>>>> </params> >>>>>>>> </methodResponse> >>>>>>>> >>>>>>>> >>>>>>>> Full transaction at >>>>>>>> >>>>>>>> https://gist.github.com/cswingler/f718abcb9ce290adec29 >>>>>>>> >>>>>>>> rhn_server_xmlrpc.log simply shows: >>>>>>>> >>>>>>>> 2015/01/26 16:52:07 -05:00 30997 172.29.7.14: >>>>>>>> xmlrpc/queue.get(1000015056, 2, 'checkins enabled') >>>>>>>> 2015/01/26 16:52:07 -05:00 30998 172.29.7.14: >>>>>>>> xmlrpc/up2date.listChannels(1000015056,) >>>>>>>> 2015/01/26 16:52:08 -05:00 30995 172.29.7.14: >>>>>>>> xmlrpc/registration.welcome_message('lang: None',) >>>>>>>> >>>>>>>> Any basic things I should be checking? I've tried updating the >>>>>>>> spacewalk/rhn tools on the agent, restarting Spacewalk itself, and >>>>>>>> nothing >>>>>>>> seems to make a change. This issue, as far as I can recall, did not >>>>>>>> seem to >>>>>>>> appear until after we upgraded to 2.2. >>>>>>>> >>>>>>>> Some systems seem just fine, and I can't seem to find a consistent >>>>>>>> reason why some succeed and others fail. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Spacewalk-list mailing list >>>>>>>> [email protected] >>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Spacewalk-list mailing list >>>>>>> [email protected] >>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Paulo Silva <[email protected]> >>>>>> >>>>>> _______________________________________________ >>>>>> Spacewalk-list mailing list >>>>>> [email protected] >>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Spacewalk-list mailing list >>>>> [email protected] >>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>>>> >>>> >>>> >>>> >>>> -- >>>> Paulo Silva <[email protected]> >>>> >>>> _______________________________________________ >>>> Spacewalk-list mailing list >>>> [email protected] >>>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>>> >>> >>> >>> _______________________________________________ >>> Spacewalk-list mailing list >>> [email protected] >>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>> >>> >>> _______________________________________________ >>> Spacewalk-list mailing list >>> [email protected] >>> https://www.redhat.com/mailman/listinfo/spacewalk-list >>> >> >> >> >> -- >> Paulo Silva <[email protected]> >> > > > > -- > Paulo Silva <[email protected]> > > _______________________________________________ > Spacewalk-list mailing list > [email protected] > https://www.redhat.com/mailman/listinfo/spacewalk-list >
_______________________________________________ Spacewalk-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/spacewalk-list
