Robert, the primary VM having an issue is a database server (MySQL). The service will only go from disabled to offline. I think there's a dependency issue but don't know where to start of if this is related to the varpd issue in the GZ. svcs on the host returns these services offline: offline 12:14:52 svc:/milestone/network:default offline 12:14:52 svc:/milestone/single-user:default offline 12:14:52 svc:/system/filesystem/local:default offline 12:14:53 svc:/system/sysidtool:net offline 12:14:53 svc:/network/initial:default offline 12:14:53 svc:/system/sysidtool:system offline 12:14:53 svc:/milestone/sysconfig:default offline 12:14:53 svc:/network/service:default offline 12:14:53 svc:/network/dns/client:default offline 12:14:53 svc:/milestone/name-services:default offline 12:14:53 svc:/network/inetd:default offline 12:14:53 svc:/system/system-log:default offline 12:14:53 svc:/system/utmp:default offline 12:14:53 svc:/system/cron:default offline 12:14:53 svc:/milestone/multi-user:default offline 12:14:53 svc:/system/console-login:default offline 12:14:54 svc:/milestone/multi-user-server:default offline 12:14:54 svc:/network/ssh:default offline 12:14:54 svc:/network/shares/group:default offline 12:14:54 svc:/network/shares/group:zfs offline 12:14:54 svc:/system/sac:default offline 12:14:54 svc:/network/netmask:default offline 12:14:54 svc:/smartdc/mdata:execute offline 12:14:54 svc:/zabbix/agent:default offline* 12:16:07 svc:/network/routing-setup:default offline 12:33:24 svc:/pkgsrc/mysql:default
On Wed, Mar 22, 2017 at 12:20 PM, Mark Creamer <[email protected]> wrote: > First I entered the dtrace command in one window. > > Then opened another window and with the service in maintenance, did svcadm > clear varpd, then svcadm disable varpd, then svcadm enable varpd. > > Nothing new in the log except the notation about clear. Same error as > originally. I did also check, and /dev/overlay does not exist but it does > on my other server. Can that be recreated or copied over if that's the > issue? It looks like it's a symlink to another file so I don't know. > > On Wed, Mar 22, 2017 at 12:13 PM, Robert Mustacchi <[email protected]> wrote: > >> On 3/22/17 9:11 , Mark Creamer wrote: >> > Robert, I did that but nothing happens. I don't have any dtrace >> experience >> > so I'm not sure what to expect. Should I have seen any output in the >> dtrace >> > command window? Thank you >> >> Can you relate the exact steps you took? But yes, you should have seen >> something in the DTrace command window. Did you see additional entries >> in the varpd service log? >> >> Robert >> >> > On Wed, Mar 22, 2017 at 11:53 AM, Robert Mustacchi <[email protected]> >> wrote: >> > >> >> On 3/22/17 8:25 , Mark Creamer wrote: >> >>> I have a host with a service in maintenance after a reboot, and >> several >> >>> services on critical VMs will not start. The service in maintenance is >> >>> varpd. Following is the log. I can't find anything on Google to help >> with >> >>> the error "varpd: failed to open a libvarpd handle: No such file or >> >>> directory". I appreciate any suggestions. If it might be just a >> matter of >> >>> reinstalling something or copying a file over from a working host, I >> just >> >>> need to know what to try. >> >>> Thanks >> >> >> >> Hi Mark, >> >> >> >> Sorry to hear that you're having trouble. While varpd being in >> >> maintenance is something we should understand, it should not be >> blocking >> >> VMs from starting up unless this is Triton and not standalone SmartOS. >> >> Probably worth understanding why they're not starting up. >> >> >> >>> [root@00-25-90-e0-dd-2c ~]# cat /var/svc/log/network-varpd\:de >> fault.log >> >>> [ May 9 19:03:16 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ May 9 19:03:17 Method "start" exited with status 0. ] >> >>> [ Aug 25 23:40:33 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ Aug 25 23:40:35 Method "start" exited with status 0. ] >> >>> [ Mar 19 17:39:35 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ Mar 19 17:39:36 Method "start" exited with status 0. ] >> >>> [ Jun 30 03:46:10 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ Jun 30 03:46:11 Method "start" exited with status 0. ] >> >>> [ Aug 28 23:39:18 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ Aug 28 23:39:20 Method "start" exited with status 0. ] >> >>> [ Mar 22 12:38:48 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ Mar 22 12:39:49 Method or service exit timed out. Killing contract >> >> 44. ] >> >>> [ Mar 22 14:28:26 Leaving maintenance because disable requested. ] >> >>> [ Mar 22 14:28:26 Disabled. ] >> >>> [ Mar 22 14:28:48 Enabled. ] >> >>> [ Mar 22 14:28:48 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> varpd: failed to open a libvarpd handle: No such file or directory >> >>> [ Mar 22 14:28:48 Method "start" exited with status 95. ] >> >>> [ Mar 22 14:30:31 Leaving maintenance because clear requested. ] >> >>> [ Mar 22 14:30:31 Enabled. ] >> >>> [ Mar 22 14:30:32 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> varpd: failed to open a libvarpd handle: No such file or directory >> >>> [ Mar 22 14:30:32 Method "start" exited with status 95. ] >> >>> [ Mar 22 14:56:28 Executing start method >> ("/lib/svc/method/svc-varpd"). ] >> >>> [ Mar 22 14:57:29 Method or service exit timed out. Killing contract >> >> 40. ] >> >>> [root@00-25-90-e0-dd-2c ~]# >> >>> >> >> >> >> This means that we're dying relatively early in the library >> >> initialization -- before we can even open up a library handle to allow >> >> the library to log more. I suspect this means that it's failing to open >> >> the /dev/overlay file. >> >> >> >> I'd recommend confirming that with something like the following: >> >> >> >> dtrace -qn 'syscall::open:entry/execname == "varpd"/{ self->p = arg0; >> }' >> >> -n 'syscall::open:return/self->p/{ printf("%s: %d %d\n", >> >> copyinstr(self->p), arg1, errno); self->p = NULL; }' >> >> >> >> And then in another window restart / clear varpd. >> >> >> >> Robert >> >> >> > >> > >> > >> >> > > > > -- > Mark > -- Mark ------------------------------------------- smartos-discuss Archives: https://www.listbox.com/member/archive/184463/=now RSS Feed: https://www.listbox.com/member/archive/rss/184463/25769125-55cfbc00 Modify Your Subscription: https://www.listbox.com/member/?member_id=25769125&id_secret=25769125-7688e9fb Powered by Listbox: http://www.listbox.com
