Re: [CentOS] Syncronize systemctl status with reality?

2017-08-30 Thread James Hogarth 
On 29 August 2017 at 21:52, Leroy Tennison <le...@datavoiceint.com> wrote:

> - Original Message -
> From: "James Hogarth" <james.hoga...@gmail.com>
> To: "centos" <centos@centos.org>
> Sent: Tuesday, August 29, 2017 2:03:44 PM
> Subject: Re: [CentOS] Syncronize systemctl status with reality?
>
> On 29 Aug 2017 17:58, "Leroy Tennison" <le...@datavoiceint.com> wrote:
>
> The particular issue is with puppetmaster (which admittedly takes 4 minutes
> to actually start, setting TimeoutStartSec=300 in it's unit file stopped
> the false timeout report) but I have seen it one other time (don't remember
> the details).
>
> systemctl status puppetmaster
> ● puppetmaster.service - Puppet master
> Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled; vendor
> preset: enabled)
> Active: failed (Result: resources) since Tue 2017-08-29 11:24:36 CDT; 22min
> ago
> Process: 897 ExecStart=/usr/bin/puppet master (code=exited,
> status=0/SUCCESS)
>
> Aug 29 11:22:39 puppetmaster02 systemd[1]: Starting Puppet master...
> Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Reopening log files
> Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Starting Puppet master
> version 3.8.5
> Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Could not run: Address
> already in use - listen(2)
> Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: PID 1233
> read from file /run/puppet/master.pid does not exist or is a zombie.
> Aug 29 11:24:36 puppetmaster02 systemd[1]: Failed to start Puppet master.
> Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: Unit
> entered failed state.
> Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: Failed
> with result 'resources'.
>
> However, ps -ef | grep puppet (run just after the above) returns
> puppet 1380 1 0 11:26 ? 00:00:08 Passenger RubyApp: /usr/share/puppet/rack/
> puppetmasterd
> root 2015 1341 0 11:48 pts/0 00:00:00 grep --color=auto puppet
>
> Earlier ps .. also reported
> puppet 1355 1166 3 11:26 ? 00:00:01 Passenger AppPreloader:
> /usr/share/puppet/rack/puppetmasterd
>
> And, the "bottom line", puppet agent -t on a client works. It reports
> finishing the catalog run and the client's yaml files on puppetmaster are
> up to date.
>
> Is there a command to tell systemd to re-scan running state and update its
> understanding on what it finds? I tried systemctl daemon-reload just to be
> sure that didn't solve the problem before posting this.
> ___
> CentOS mailing list
> CentOS@centos.org
> https://lists.centos.org/mailman/listinfo/centos
>
>
> First glance ity looks like someone has started that puppetmaster manually
> at some point. As such it's not in a cgroup systemd is tracking so it isn't
> aware of it.
>
> Your attempts to start the service are failing because that manually
> started instance already has the port open.
>
> Kill it with pkill -f puppet and then use ss -tnp to check for the port
> being freed (wait for any time_wait states to go... which is why I'm not
> filtering by listen).
>
> Once it's clear then try starting with systemctl
> ___
> CentOS mailing list
> CentOS@centos.org
> https://lists.centos.org/mailman/listinfo/centos
>
> OK, something weird is definitely going on here, I have the luxury of
> rebooting this system so it did.  Here's what I got, note the time stamps.
>
> ps -ef | grep puppet
> root   932 1  0 15:23 ?00:00:00 /usr/bin/ruby
> /usr/bin/puppet master
> root  1343  1327  0 15:24 pts/000:00:00 grep --color=auto puppet
> (immediately afterward as fast as I could type:) uptime
>  15:24:56 up 1 min,  1 user,  load average: 0.16, 0.07, 0.02
> systemctl status puppetmaster
> ● puppetmaster.service - Puppet master
>Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled;
> vendor preset: enabled)
>Active: activating (start) since Tue 2017-08-29 15:23:44 CDT; 1min 24s
> ago
>   Control: 932 (puppet)
> Tasks: 1
>Memory: 2.4M
>   CPU: 4ms
>CGroup: /system.slice/puppetmaster.service
>└─932 /usr/bin/ruby /usr/bin/puppet master
>
> Aug 29 15:23:44 puppetmaster02 systemd[1]: Starting Puppet master...
>
>
>
> After a short delay:
> systemctl status puppetmaster
> ● puppetmaster.service - Puppet master
>Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled;
> vendor preset: enabled)
>Active: failed (Result: resources) since Tue 2017-08-29 15:25:11 CDT;
> 11s ago
>   Process: 932 ExecStart=/usr/bin/puppet master (code=exited,
> status=0/SUC

Re: [CentOS] Syncronize systemctl status with reality?

2017-08-29 Thread Leroy Tennison 
Hmmm, that's an interesting option, I'll have to look into it.

- Original Message -
From: "Alexander Dalloz" <ad+li...@uni-x.org>
To: "centos" <centos@centos.org>
Sent: Tuesday, August 29, 2017 4:17:37 PM
Subject: Re: [CentOS] Syncronize systemctl status with reality?

Am 29.08.2017 um 22:52 schrieb Leroy Tennison:
> The AppPreloader is doing things (and probably confusing systemd in the 
> process) but I didn't start that, it was a part of the reboot.  I looked at 
> /etc/init.d/puppetmaster to see if something strange was being done and it 
> was one of the simpler init scripts I've seen, uses start-stop-daemon.
> 
> I don't have any control over this, 'systemctl list-unit-files | grep puppet' 
> shows the puppetmaster.service is enabled, I just need a "cleanup" solution, 
> any ideas?  And, BTW, thanks for any feedback.

Remove the Rack Puppet master server.

https://docs.puppet.com/puppet/5.1/passenger.html#install-the-puppet-master-rack-application

Alexander
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Syncronize systemctl status with reality?

2017-08-29 Thread Alexander Dalloz 

Am 29.08.2017 um 22:52 schrieb Leroy Tennison:

The AppPreloader is doing things (and probably confusing systemd in the 
process) but I didn't start that, it was a part of the reboot.  I looked at 
/etc/init.d/puppetmaster to see if something strange was being done and it was 
one of the simpler init scripts I've seen, uses start-stop-daemon.

I don't have any control over this, 'systemctl list-unit-files | grep puppet' shows the 
puppetmaster.service is enabled, I just need a "cleanup" solution, any ideas?  
And, BTW, thanks for any feedback.


Remove the Rack Puppet master server.

https://docs.puppet.com/puppet/5.1/passenger.html#install-the-puppet-master-rack-application

Alexander
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] Syncronize systemctl status with reality?

2017-08-29 Thread Leroy Tennison 
- Original Message -
From: "James Hogarth" <james.hoga...@gmail.com>
To: "centos" <centos@centos.org>
Sent: Tuesday, August 29, 2017 2:03:44 PM
Subject: Re: [CentOS] Syncronize systemctl status with reality?

On 29 Aug 2017 17:58, "Leroy Tennison" <le...@datavoiceint.com> wrote:

The particular issue is with puppetmaster (which admittedly takes 4 minutes
to actually start, setting TimeoutStartSec=300 in it's unit file stopped
the false timeout report) but I have seen it one other time (don't remember
the details).

systemctl status puppetmaster
● puppetmaster.service - Puppet master
Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled; vendor
preset: enabled)
Active: failed (Result: resources) since Tue 2017-08-29 11:24:36 CDT; 22min
ago
Process: 897 ExecStart=/usr/bin/puppet master (code=exited,
status=0/SUCCESS)

Aug 29 11:22:39 puppetmaster02 systemd[1]: Starting Puppet master...
Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Reopening log files
Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Starting Puppet master
version 3.8.5
Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Could not run: Address
already in use - listen(2)
Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: PID 1233
read from file /run/puppet/master.pid does not exist or is a zombie.
Aug 29 11:24:36 puppetmaster02 systemd[1]: Failed to start Puppet master.
Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: Unit
entered failed state.
Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: Failed
with result 'resources'.

However, ps -ef | grep puppet (run just after the above) returns
puppet 1380 1 0 11:26 ? 00:00:08 Passenger RubyApp: /usr/share/puppet/rack/
puppetmasterd
root 2015 1341 0 11:48 pts/0 00:00:00 grep --color=auto puppet

Earlier ps .. also reported
puppet 1355 1166 3 11:26 ? 00:00:01 Passenger AppPreloader:
/usr/share/puppet/rack/puppetmasterd

And, the "bottom line", puppet agent -t on a client works. It reports
finishing the catalog run and the client's yaml files on puppetmaster are
up to date.

Is there a command to tell systemd to re-scan running state and update its
understanding on what it finds? I tried systemctl daemon-reload just to be
sure that didn't solve the problem before posting this.
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


First glance ity looks like someone has started that puppetmaster manually
at some point. As such it's not in a cgroup systemd is tracking so it isn't
aware of it.

Your attempts to start the service are failing because that manually
started instance already has the port open.

Kill it with pkill -f puppet and then use ss -tnp to check for the port
being freed (wait for any time_wait states to go... which is why I'm not
filtering by listen).

Once it's clear then try starting with systemctl
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

OK, something weird is definitely going on here, I have the luxury of rebooting 
this system so it did.  Here's what I got, note the time stamps.

ps -ef | grep puppet
root   932 1  0 15:23 ?00:00:00 /usr/bin/ruby /usr/bin/puppet 
master
root  1343  1327  0 15:24 pts/000:00:00 grep --color=auto puppet
(immediately afterward as fast as I could type:) uptime
 15:24:56 up 1 min,  1 user,  load average: 0.16, 0.07, 0.02
systemctl status puppetmaster
● puppetmaster.service - Puppet master
   Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled; vendor 
preset: enabled)
   Active: activating (start) since Tue 2017-08-29 15:23:44 CDT; 1min 24s ago
  Control: 932 (puppet)
Tasks: 1
   Memory: 2.4M
  CPU: 4ms
   CGroup: /system.slice/puppetmaster.service
   └─932 /usr/bin/ruby /usr/bin/puppet master

Aug 29 15:23:44 puppetmaster02 systemd[1]: Starting Puppet master...



After a short delay:
systemctl status puppetmaster
● puppetmaster.service - Puppet master
   Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled; vendor 
preset: enabled)
   Active: failed (Result: resources) since Tue 2017-08-29 15:25:11 CDT; 11s ago
  Process: 932 ExecStart=/usr/bin/puppet master (code=exited, status=0/SUCCESS)

Aug 29 15:23:44 puppetmaster02 systemd[1]: Starting Puppet master...
Aug 29 15:25:11 puppetmaster02 puppet-master[1360]: Reopening log files
Aug 29 15:25:11 puppetmaster02 puppet-master[1360]: Starting Puppet master 
version 3.8.5
Aug 29 15:25:11 puppetmaster02 puppet-master[1360]: Could not run: Address 
already in use - listen(2)
Aug 29 15:25:11 puppetmaster02 systemd[1]: puppetmaster.service: PID 1360 read 
from file /run/puppet/master.pid does not exist or is a zombie.
Aug 29 15:25:11 puppetmaster02 systemd[1]: Failed to start Puppet master.
Aug 29 15:25:11 puppetmaster02 systemd[1]: puppetm

Re: [CentOS] Syncronize systemctl status with reality?

2017-08-29 Thread James Hogarth 
On 29 Aug 2017 17:58, "Leroy Tennison"  wrote:

The particular issue is with puppetmaster (which admittedly takes 4 minutes
to actually start, setting TimeoutStartSec=300 in it's unit file stopped
the false timeout report) but I have seen it one other time (don't remember
the details).

systemctl status puppetmaster
● puppetmaster.service - Puppet master
Loaded: loaded (/lib/systemd/system/puppetmaster.service; enabled; vendor
preset: enabled)
Active: failed (Result: resources) since Tue 2017-08-29 11:24:36 CDT; 22min
ago
Process: 897 ExecStart=/usr/bin/puppet master (code=exited,
status=0/SUCCESS)

Aug 29 11:22:39 puppetmaster02 systemd[1]: Starting Puppet master...
Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Reopening log files
Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Starting Puppet master
version 3.8.5
Aug 29 11:24:36 puppetmaster02 puppet-master[1233]: Could not run: Address
already in use - listen(2)
Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: PID 1233
read from file /run/puppet/master.pid does not exist or is a zombie.
Aug 29 11:24:36 puppetmaster02 systemd[1]: Failed to start Puppet master.
Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: Unit
entered failed state.
Aug 29 11:24:36 puppetmaster02 systemd[1]: puppetmaster.service: Failed
with result 'resources'.

However, ps -ef | grep puppet (run just after the above) returns
puppet 1380 1 0 11:26 ? 00:00:08 Passenger RubyApp: /usr/share/puppet/rack/
puppetmasterd
root 2015 1341 0 11:48 pts/0 00:00:00 grep --color=auto puppet

Earlier ps .. also reported
puppet 1355 1166 3 11:26 ? 00:00:01 Passenger AppPreloader:
/usr/share/puppet/rack/puppetmasterd

And, the "bottom line", puppet agent -t on a client works. It reports
finishing the catalog run and the client's yaml files on puppetmaster are
up to date.

Is there a command to tell systemd to re-scan running state and update its
understanding on what it finds? I tried systemctl daemon-reload just to be
sure that didn't solve the problem before posting this.
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos


First glance ity looks like someone has started that puppetmaster manually
at some point. As such it's not in a cgroup systemd is tracking so it isn't
aware of it.

Your attempts to start the service are failing because that manually
started instance already has the port open.

Kill it with pkill -f puppet and then use ss -tnp to check for the port
being freed (wait for any time_wait states to go... which is why I'm not
filtering by listen).

Once it's clear then try starting with systemctl
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos