Re: [systemd-devel] Cannot start service due to 'systemd-tty-ask-password-agent --watch' not answering
- Original Message - > From: "Lennart Poettering" > To: "Yeela Kaplan" > Cc: systemd-devel@lists.freedesktop.org, "Oved Ourfali" , > "Dan Kenigsberg" > Sent: Tuesday, September 8, 2015 6:51:30 PM > Subject: Re: [systemd-devel] Cannot start service due to > 'systemd-tty-ask-password-agent --watch' not answering > > On Tue, 08.09.15 11:29, Yeela Kaplan (ykap...@redhat.com) wrote: > > > > Hmm? You wrote earlier that systemctl was hanging. Is it hanging or > > > isn't it hanging? > > > > I'm sorry, I used the wrong word. > > I meant that it is not blocking. > > But it also does not start the service. > > So eventually, that's our problem, that the service does not start.. even > > though systemd is not blocking. > > Maybe it's just an issue that happened on this specific host. > > Hmm, what do you mean by "not starting"? Does "systemctl status" on > the service unit say it's up? Can you paste the output of this? > when using 'ps' there is no such process.. and also for status: systemctl status supervdsmd supervdsmd.service - "Auxiliary vdsm service for running helper functions as root" Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static) Active: inactive (dead) Sep 02 06:37:36 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 06:42:12 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 07:30:52 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 08:23:59 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 11:08:51 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 12:16:32 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 13:25:02 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 13:37:48 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root". Sep 02 13:41:10 green-vdsc.qa.lab.tlv.redhat.com systemd[1]: Stopped "Auxiliary vdsm service for running helper functions as root" > Did it start, but maybe just exit quickly? > If it would have started I'd expect to see it in our logs (/usr/share/vdsm/supervdsm.log) but there is no such indication.. > Lennart > > -- > Lennart Poettering, Red Hat > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Cannot start service due to 'systemd-tty-ask-password-agent --watch' not answering
- Original Message - > From: "Lennart Poettering" > To: "Yeela Kaplan" > Cc: systemd-devel@lists.freedesktop.org, "Oved Ourfali" , > "Dan Kenigsberg" > Sent: Tuesday, September 8, 2015 6:23:56 PM > Subject: Re: [systemd-devel] Cannot start service due to > 'systemd-tty-ask-password-agent --watch' not answering > > On Tue, 08.09.15 10:47, Yeela Kaplan (ykap...@redhat.com) wrote: > > > > > > > - Original Message - > > > From: "Lennart Poettering" > > > To: "Yeela Kaplan" > > > Cc: systemd-devel@lists.freedesktop.org, "Oved Ourfali" > > > , "Dan Kenigsberg" > > > Sent: Tuesday, September 8, 2015 4:48:31 PM > > > Subject: Re: [systemd-devel] Cannot start service due to > > > 'systemd-tty-ask-password-agent --watch' not answering > > > > > > On Tue, 08.09.15 07:56, Yeela Kaplan (ykap...@redhat.com) wrote: > > > > > > > > > Thank you for your response. > > > > > > I have tried running: strace -f systemctl --no-ask-password start > > > > > > supervdsmd.service > > > > > > and it still reproduces. > > > > > > but now intead of: > > > > > > bash(3448)─┬─pstree(3625) > > > > > >└─strace(3618)───systemctl(3623)───systemd-tty-ask(3624) > > > > > > > > > > > > I get: > > > > > > bash(3448)─┬─pstree(3586) > > > > > >└─strace(3564)───systemctl(3569) > > > > > > > > > > > > Do you know what systemctl is waiting for then? > > > > > > > > > > Most likely that supervdsmd.service finishes start-up. Maybe there's > > > > > a > > > > > bug in the unit file regarding notification when it finished > > > > > start-up? > > > > > > > > > > (You can also tell systemctl to just enqueue a start job, but not > > > > > wait > > > > > for it to complete. Use --no-block for that). > > > > > > > > This is the unit file: > > > > [Unit] > > > > Description="Auxiliary vdsm service for running helper functions as > > > > root" > > > > Requires=libvirtd.service > > > > > > > > [Service] > > > > Type=simple > > > > LimitCORE=infinity > > > > EnvironmentFile=-/etc/sysconfig/supervdsmd > > > > ExecStart=/usr/share/vdsm/daemonAdapter > > > > "/usr/share/vdsm/supervdsmServer" > > > > --sockfile "/var/run/vdsm/svdsm.sock" > > > > Restart=always > > > > > > > > > > > > The service works fine on other machines... this is the first time we > > > > see > > > > it.. > > > > and the service is able to start without systemd.. by just running: > > > > /usr/share/vdsm/daemonAdapter "/usr/share/vdsm/supervdsmServer" > > > > --sockfile > > > > "/var/run/vdsm/svdsm.sock" > > > > but hangs for a long time without starting even when using --no-block.. > > > > > > Ah, you are saying that even with --no-block it will block? If so, > > > that indicates a D-Bus problem of some kind. Could you paste the > > > strace output of "systemctl --no-block --no-ask-password start ..." > > > somewhere please? What distro is this? > > > > It's not blocking, but it also does not start the service so it is > > not a notification issue.. > > Hmm? You wrote earlier that systemctl was hanging. Is it hanging or > isn't it hanging? I'm sorry, I used the wrong word. I meant that it is not blocking. But it also does not start the service. So eventually, that's our problem, that the service does not start.. even though systemd is not blocking. Maybe it's just an issue that happened on this specific host. > > > That's the strace: > > https://paste.fedoraproject.org/264808/72290914/ > > That shows that systemctl exits cleanly with exit code 0... > > What precisely is the problem? You now say that systemctl wasn't > hanging and show me that it exits with exit code 0, so there's nothing > wrong from what I can see. Not sure what you are trying to say at all? > > Lennart > > -- > Lennart Poettering, Red Hat > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Cannot start service due to 'systemd-tty-ask-password-agent --watch' not answering
- Original Message - > From: "Lennart Poettering" > To: "Yeela Kaplan" > Cc: systemd-devel@lists.freedesktop.org, "Oved Ourfali" , > "Dan Kenigsberg" > Sent: Tuesday, September 8, 2015 4:48:31 PM > Subject: Re: [systemd-devel] Cannot start service due to > 'systemd-tty-ask-password-agent --watch' not answering > > On Tue, 08.09.15 07:56, Yeela Kaplan (ykap...@redhat.com) wrote: > > > > > Thank you for your response. > > > > I have tried running: strace -f systemctl --no-ask-password start > > > > supervdsmd.service > > > > and it still reproduces. > > > > but now intead of: > > > > bash(3448)─┬─pstree(3625) > > > >└─strace(3618)───systemctl(3623)───systemd-tty-ask(3624) > > > > > > > > I get: > > > > bash(3448)─┬─pstree(3586) > > > >└─strace(3564)───systemctl(3569) > > > > > > > > Do you know what systemctl is waiting for then? > > > > > > Most likely that supervdsmd.service finishes start-up. Maybe there's a > > > bug in the unit file regarding notification when it finished start-up? > > > > > > (You can also tell systemctl to just enqueue a start job, but not wait > > > for it to complete. Use --no-block for that). > > > > This is the unit file: > > [Unit] > > Description="Auxiliary vdsm service for running helper functions as root" > > Requires=libvirtd.service > > > > [Service] > > Type=simple > > LimitCORE=infinity > > EnvironmentFile=-/etc/sysconfig/supervdsmd > > ExecStart=/usr/share/vdsm/daemonAdapter "/usr/share/vdsm/supervdsmServer" > > --sockfile "/var/run/vdsm/svdsm.sock" > > Restart=always > > > > > > The service works fine on other machines... this is the first time we see > > it.. > > and the service is able to start without systemd.. by just running: > > /usr/share/vdsm/daemonAdapter "/usr/share/vdsm/supervdsmServer" --sockfile > > "/var/run/vdsm/svdsm.sock" > > but hangs for a long time without starting even when using --no-block.. > > Ah, you are saying that even with --no-block it will block? If so, > that indicates a D-Bus problem of some kind. Could you paste the > strace output of "systemctl --no-block --no-ask-password start ..." > somewhere please? What distro is this? It's not blocking, but it also does not start the service so it is not a notification issue.. That's the strace: https://paste.fedoraproject.org/264808/72290914/ It's RHEV-H 7.1. It is based on el7 with systemd-208-20.el7_1.5.x86_64. Thank you! > > Lennart > > -- > Lennart Poettering, Red Hat > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Cannot start service due to 'systemd-tty-ask-password-agent --watch' not answering
- Original Message - > From: "Lennart Poettering" > To: "Yeela Kaplan" > Cc: systemd-devel@lists.freedesktop.org, "Oved Ourfali" , > "Dan Kenigsberg" > Sent: Tuesday, September 8, 2015 1:45:54 PM > Subject: Re: [systemd-devel] Cannot start service due to > 'systemd-tty-ask-password-agent --watch' not answering > > On Tue, 08.09.15 05:51, Yeela Kaplan (ykap...@redhat.com) wrote: > > > > This is really a misconception about what > > > systemd-tty-ask-password-agent actually does. It's job is to bring > > > password queries from system components to the screen while you wait > > > for systemctl to finish. This is useful for things like LUKS hdd > > > encryption where the system might have to query the user for a > > > passphrase to proceed starting units. Other cases where this is used > > > is for passphrases of SSL certificates. > > > > > > Unless you actually use LUKS or SSL certificates with a passphrase > > > (with a web server that supports querying passwords via systemd's > > > password querying mechanism), the agent does pretty much nothing. > > > > > > The agent is forked off, waits for passwords to be queried, queries > > > them when there are any, and then exits when systemctl finishes. > > > > > > You can pass --no-ask-password to systemctl to turn off the agent. See > > > the man page for details. > > > > > > > Thank you for your response. > > I have tried running: strace -f systemctl --no-ask-password start > > supervdsmd.service > > and it still reproduces. > > but now intead of: > > bash(3448)─┬─pstree(3625) > >└─strace(3618)───systemctl(3623)───systemd-tty-ask(3624) > > > > I get: > > bash(3448)─┬─pstree(3586) > >└─strace(3564)───systemctl(3569) > > > > Do you know what systemctl is waiting for then? > > Most likely that supervdsmd.service finishes start-up. Maybe there's a > bug in the unit file regarding notification when it finished start-up? > > (You can also tell systemctl to just enqueue a start job, but not wait > for it to complete. Use --no-block for that). This is the unit file: [Unit] Description="Auxiliary vdsm service for running helper functions as root" Requires=libvirtd.service [Service] Type=simple LimitCORE=infinity EnvironmentFile=-/etc/sysconfig/supervdsmd ExecStart=/usr/share/vdsm/daemonAdapter "/usr/share/vdsm/supervdsmServer" --sockfile "/var/run/vdsm/svdsm.sock" Restart=always The service works fine on other machines... this is the first time we see it.. and the service is able to start without systemd.. by just running: /usr/share/vdsm/daemonAdapter "/usr/share/vdsm/supervdsmServer" --sockfile "/var/run/vdsm/svdsm.sock" but hangs for a long time without starting even when using --no-block.. > > Lennart > > -- > Lennart Poettering, Red Hat > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Cannot start service due to 'systemd-tty-ask-password-agent --watch' not answering
- Original Message -
> From: "Lennart Poettering"
> To: "Yeela Kaplan"
> Cc: systemd-devel@lists.freedesktop.org, "Oved Ourfali"
> Sent: Sunday, September 6, 2015 4:56:29 PM
> Subject: Re: [systemd-devel] Cannot start service due to
> 'systemd-tty-ask-password-agent --watch' not answering
>
> On Thu, 03.09.15 04:39, Yeela Kaplan (ykap...@redhat.com) wrote:
>
> > Hi all,
> > We have run into a problem starting a service on EL7.1.
> > We installed the same environment on two hosts, on the first one the
> > service started successfully and on the second one it failed to start.
> >
> > On the machine failing when running:
> > systemctl start supervdsmd.service
> >
> > we see that a child process `systemd-tty-ask-password-agent --watch`
> > is spawned by systemctl:
> >
> > [root@green-vdsc ~]# pstree -p $$
> > bash(13179)─┬─pstree(13494)
> > └─systemctl(13440)───systemd-tty-ask(13441)
> >
> > using strace -f we see that systemctl is waiting on a response from
> > systemd-tty-ask but never receives it and gets stuck forever polling:
> >
> > [pid 14884] execve("/usr/bin/systemd-tty-ask-password-agent",
> > ["/usr/bin/systemd-tty-ask-passwor"..., "--watch"], [/* 31 vars */]) = 0
> > .
> > .
> > .
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\242\233\270\f9\257\326\257_\n\30\264\263\17\32\272\0058\331\245\236f\207\202\321(\322`pS\324\376"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "G\262_$\210%*\275Q`G\240\361F\323\306\254.\334\323]\314:!u\343H\200\227\1\206&"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\272\177J\330D\252\367\2\367w\302\230-\221'\267\365\376\214\217\334\327\245a\311\377\tfc\177\273j"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\324\230x\231\222\210\271-\22a\2134\202\274\\\373\305\231;\177\244e\246=k\204\216\327\340iG\17"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "3\210\234X\0N\202G\354\22WFt^\331V\344\32\\A\36\323[\302\370\363\371\210\211\t\2129"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "+\235\26\274\373R\34\377Rs9\370\273\370\t),V\276\v`\233^E\256\257lX#\27\23W"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\346\237\ru5\311\372\362K?2\203\300\246C2K(\4+\20\341t\4S\370\35\25}>\265\240"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "F\25\257j\233q\315W\25\334\306\233\217!\306$\255Y\33\364\0039Qy\2\223@o\235\257hw"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\324\316\10k*\332A\253\344^\5y\372\10I\375\216\372\271\277\205\2264\0\35>?]`\263\374\275"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\34P\10w\314e\270\226X\t\212\30\"\17.\334\3471\342B\334\272\315\351\367\324k\273S\363\323f"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "\210\230;\227\31\261\364A\301#\374\311\206?\17\202\2749\340\246\323gm\315\325\361\335K\200\6h\275"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> > read(3,
> > "i\10Dl\5\312\222\376\10l\37\277X\341CIs\\\"\235{\315|\334\227\10>\246a\5,\36"...,
> > 120) = 120
> > poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
> >
> > systemctl is waiting for a response from its child process forever and does
> > not close it.
> > As a result, it never even tries to run the start command for our service
> > supervdsm.
> > So it looks like an issue either with systemd or the way our service
> > interacts with systemd.
> >
> > Is anyone familiar with such an issue or has an idea why it happens and how
> > to fix it?
>
> This is really a misconception about what
> systemd-tty-ask-password-agent actually does. It's job is to bring
> password queries from
[systemd-devel] Cannot start service due to 'systemd-tty-ask-password-agent --watch' not answering
Hi all,
We have run into a problem starting a service on EL7.1.
We installed the same environment on two hosts, on the first one the service
started successfully and on the second one it failed to start.
On the machine failing when running:
systemctl start supervdsmd.service
we see that a child process `systemd-tty-ask-password-agent --watch`
is spawned by systemctl:
[root@green-vdsc ~]# pstree -p $$
bash(13179)─┬─pstree(13494)
└─systemctl(13440)───systemd-tty-ask(13441)
using strace -f we see that systemctl is waiting on a response from
systemd-tty-ask but never receives it and gets stuck forever polling:
[pid 14884] execve("/usr/bin/systemd-tty-ask-password-agent",
["/usr/bin/systemd-tty-ask-passwor"..., "--watch"], [/* 31 vars */]) = 0
.
.
.
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\242\233\270\f9\257\326\257_\n\30\264\263\17\32\272\0058\331\245\236f\207\202\321(\322`pS\324\376"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"G\262_$\210%*\275Q`G\240\361F\323\306\254.\334\323]\314:!u\343H\200\227\1\206&"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\272\177J\330D\252\367\2\367w\302\230-\221'\267\365\376\214\217\334\327\245a\311\377\tfc\177\273j"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\324\230x\231\222\210\271-\22a\2134\202\274\\\373\305\231;\177\244e\246=k\204\216\327\340iG\17"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"3\210\234X\0N\202G\354\22WFt^\331V\344\32\\A\36\323[\302\370\363\371\210\211\t\2129"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"+\235\26\274\373R\34\377Rs9\370\273\370\t),V\276\v`\233^E\256\257lX#\27\23W"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\346\237\ru5\311\372\362K?2\203\300\246C2K(\4+\20\341t\4S\370\35\25}>\265\240"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"F\25\257j\233q\315W\25\334\306\233\217!\306$\255Y\33\364\0039Qy\2\223@o\235\257hw"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\324\316\10k*\332A\253\344^\5y\372\10I\375\216\372\271\277\205\2264\0\35>?]`\263\374\275"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\34P\10w\314e\270\226X\t\212\30\"\17.\334\3471\342B\334\272\315\351\367\324k\273S\363\323f"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"\210\230;\227\31\261\364A\301#\374\311\206?\17\202\2749\340\246\323gm\315\325\361\335K\200\6h\275"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
read(3,
"i\10Dl\5\312\222\376\10l\37\277X\341CIs\\\"\235{\315|\334\227\10>\246a\5,\36"...,
120) = 120
poll([{fd=3, events=POLLIN}], 1, 100) = 1 ([{fd=3, revents=POLLIN}])
systemctl is waiting for a response from its child process forever and does not
close it.
As a result, it never even tries to run the start command for our service
supervdsm.
So it looks like an issue either with systemd or the way our service interacts
with systemd.
Is anyone familiar with such an issue or has an idea why it happens and how to
fix it?
We still have the machine with this problem running if it helps,
can anybody maybe take a look and help figure out what's the problem?
Thanks,
Yeela.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
