Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On 01/15/2015 08:56 AM, Nathan Kinder wrote: On 01/15/2015 12:01 AM, Jan Pazdziora wrote: ... You need to use --cap-add=SYS_TIME when running the server container or ntpd will fail. Thanks for the tip. This works. It would be handy to add this to the README for your freeipa-server container. Nevermind. I just saw your reply to Lukas on this. If we can keep the client install from hanging forever, then I agree that it's best to have it be noticeable that time sync is not working in the client installer output vs. hiding that it's not working. Even if you do that, SELinux will likely prevent ntpd doing its job but at least it will stay around so that the client can connect to it. What is interesting though is the fact that the client hangs indefinitely instead of reporting that it cannot sync the time and proceeding. I think this is simply a behavior difference between ntpdate and ntpd (which we are using now during the client install on f21). This issue should not be specific to using IPA in a container. Hanging indefinitely is never a good thing, so I think it would be nice to add a timeout in ipa-client-install in case we can't reach the server for ntp. I have filed a ticket for this: https://fedorahosted.org/freeipa/ticket/4842 -NGK -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On 01/15/2015 12:01 AM, Jan Pazdziora wrote: On Wed, Jan 14, 2015 at 08:18:02PM -0800, Nathan Kinder wrote: Hi, I'm running into a strange problem related to ntpd when trying to use IPA in a container. I'm using the adelton/freeipa-server:fedora-21 and adelton/freeipa-client:fedora-21 docker images. Basically, the client install hangs when it runs ntpd. This is reproducible on two different docker hosts of mine, so it will probably easily reproduce for others as [...] The /sbin/ipa-server-configure-first entrypoint script for the server image does a 'systemctl start-enabled' to bring up all of the services, which results in this output in /var/log/systemctl.log: [start-enabled] [start ntpd.service] Running [export OPTIONS=-g -x; /usr/sbin/ntpd -u ntp:ntp $OPTIONS] Marked pid [15] for [ntpd.service] Marked process name [/usr/sbin/ntpd] for [ntpd.service] ... This is the same log output that is generated if I manually run 'systemctl start ntpd.service' from within the container, but the ntpd process stays around when I start it this way. It's hard to tell what might be happening to ntpd, as there is no journal in the container. I'm continuing to debug this, but I thought I'd share my findings thus far in case anyone else has seen this or has any ideas for tracking the problem down. Any ideas? You need to use --cap-add=SYS_TIME when running the server container or ntpd will fail. Thanks for the tip. This works. It would be handy to add this to the README for your freeipa-server container. Even if you do that, SELinux will likely prevent ntpd doing its job but at least it will stay around so that the client can connect to it. What is interesting though is the fact that the client hangs indefinitely instead of reporting that it cannot sync the time and proceeding. I think this is simply a behavior difference between ntpdate and ntpd (which we are using now during the client install on f21). This issue should not be specific to using IPA in a container. Hanging indefinitely is never a good thing, so I think it would be nice to add a timeout in ipa-client-install in case we can't reach the server for ntp. I have filed a ticket for this: https://fedorahosted.org/freeipa/ticket/4842 -NGK -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On Thu, Jan 15, 2015 at 08:56:29AM -0800, Nathan Kinder wrote: Even if you do that, SELinux will likely prevent ntpd doing its job but at least it will stay around so that the client can connect to it. What is interesting though is the fact that the client hangs indefinitely instead of reporting that it cannot sync the time and proceeding. I think this is simply a behavior difference between ntpdate and ntpd (which we are using now during the client install on f21). This issue should not be specific to using IPA in a container. The problem is, on Fedora 21 client which is not container and ntpd not running on the server, I was not able to reproduce the issue. -- Jan Pazdziora Principal Software Engineer, Identity Management Engineering, Red Hat -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On 01/15/2015 09:41 AM, Jan Pazdziora wrote: On Thu, Jan 15, 2015 at 08:56:29AM -0800, Nathan Kinder wrote: Even if you do that, SELinux will likely prevent ntpd doing its job but at least it will stay around so that the client can connect to it. What is interesting though is the fact that the client hangs indefinitely instead of reporting that it cannot sync the time and proceeding. I think this is simply a behavior difference between ntpdate and ntpd (which we are using now during the client install on f21). This issue should not be specific to using IPA in a container. The problem is, on Fedora 21 client which is not container and ntpd not running on the server, I was not able to reproduce the issue. Strange... Does it reproduce for you when using containers for both the server and the client? -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On Wed, Jan 14, 2015 at 08:18:02PM -0800, Nathan Kinder wrote: Hi, I'm running into a strange problem related to ntpd when trying to use IPA in a container. I'm using the adelton/freeipa-server:fedora-21 and adelton/freeipa-client:fedora-21 docker images. Basically, the client install hangs when it runs ntpd. This is reproducible on two different docker hosts of mine, so it will probably easily reproduce for others as [...] The /sbin/ipa-server-configure-first entrypoint script for the server image does a 'systemctl start-enabled' to bring up all of the services, which results in this output in /var/log/systemctl.log: [start-enabled] [start ntpd.service] Running [export OPTIONS=-g -x; /usr/sbin/ntpd -u ntp:ntp $OPTIONS] Marked pid [15] for [ntpd.service] Marked process name [/usr/sbin/ntpd] for [ntpd.service] ... This is the same log output that is generated if I manually run 'systemctl start ntpd.service' from within the container, but the ntpd process stays around when I start it this way. It's hard to tell what might be happening to ntpd, as there is no journal in the container. I'm continuing to debug this, but I thought I'd share my findings thus far in case anyone else has seen this or has any ideas for tracking the problem down. Any ideas? You need to use --cap-add=SYS_TIME when running the server container or ntpd will fail. Even if you do that, SELinux will likely prevent ntpd doing its job but at least it will stay around so that the client can connect to it. What is interesting though is the fact that the client hangs indefinitely instead of reporting that it cannot sync the time and proceeding. -- Jan Pazdziora Principal Software Engineer, Identity Management Engineering, Red Hat -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On (15/01/15 09:01), Jan Pazdziora wrote: On Wed, Jan 14, 2015 at 08:18:02PM -0800, Nathan Kinder wrote: Hi, I'm running into a strange problem related to ntpd when trying to use IPA in a container. I'm using the adelton/freeipa-server:fedora-21 and adelton/freeipa-client:fedora-21 docker images. Basically, the client install hangs when it runs ntpd. This is reproducible on two different docker hosts of mine, so it will probably easily reproduce for others as [...] The /sbin/ipa-server-configure-first entrypoint script for the server image does a 'systemctl start-enabled' to bring up all of the services, which results in this output in /var/log/systemctl.log: [start-enabled] [start ntpd.service] Running [export OPTIONS=-g -x; /usr/sbin/ntpd -u ntp:ntp $OPTIONS] Marked pid [15] for [ntpd.service] Marked process name [/usr/sbin/ntpd] for [ntpd.service] ... This is the same log output that is generated if I manually run 'systemctl start ntpd.service' from within the container, but the ntpd process stays around when I start it this way. It's hard to tell what might be happening to ntpd, as there is no journal in the container. I'm continuing to debug this, but I thought I'd share my findings thus far in case anyone else has seen this or has any ideas for tracking the problem down. Any ideas? You need to use --cap-add=SYS_TIME when running the server container or ntpd will fail. Could you add this important information to the https://registry.hub.docker.com/u/adelton/freeipa-server/? LS -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
Re: [Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
On Thu, Jan 15, 2015 at 09:06:54AM +0100, Lukas Slebodnik wrote: I'm continuing to debug this, but I thought I'd share my findings thus far in case anyone else has seen this or has any ideas for tracking the problem down. Any ideas? You need to use --cap-add=SYS_TIME when running the server container or ntpd will fail. Could you add this important information to the https://registry.hub.docker.com/u/adelton/freeipa-server/? As mentioned, it will not help you due to SELinux, so at this point I'd rather have people notified that the time sync does not happen than to have false assumptions. I'll update the git repo README / image documentation once we know what exactly the plan with SELinux and situation with Fedora 21 client blocking are. It is something I work on right now. -- Jan Pazdziora Principal Software Engineer, Identity Management Engineering, Red Hat -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go To http://freeipa.org for more info on the project
[Freeipa-users] Problems with ntpd when running FreeIPA in a Docker container
Hi, I'm running into a strange problem related to ntpd when trying to use IPA in a container. I'm using the adelton/freeipa-server:fedora-21 and adelton/freeipa-client:fedora-21 docker images. Basically, the client install hangs when it runs ntpd. This is reproducible on two different docker hosts of mine, so it will probably easily reproduce for others as well. Below are the steps I'm using. Install IPA server in F21 container: [root@localhost ~]# docker run --name freeipa-server-container -d -h ipa.example.test -e PASSWORD=Secret123 adelton/freeipa-server:fedora-21 875007ab561ff62ea45dde5e8a5e320a209c63b3c8fc52bd4ca7b22561d1bbf0 [root@localhost ~]# docker logs freeipa-server-container ... FreeIPA server configured. Go loop. Install IPA client in F21 container and link it to the IPA server container. This will hang indefinitely when it tries to run ntpd to sync the time before getting the admin ticket: [root@localhost ~]# docker run --name client -h client.example.test --link freeipa-server-container:ipa -e PASSWORD=Secret123 -e IPA_CLIENT_INSTALL=--debug -it adelton/freeipa-client:fedora-21 ... Synchronizing time with KDC... Search DNS for SRV record of _ntp._udp.example.test DNS record found: 0 100 123 ipa.example.test. Starting external process args='/usr/sbin/ntpd' '-qgc' '/tmp/tmpRhhyCz' If I use nsenter to go into the client container and kill ntpd, the install continues and completes. I also confirmed that the ntpd config file that we create in /tmp is correct. From within the client container (via nsenter), running 'ntpd -qgc' with a conf file that points to the IPA server just loops endlessly. I looked into the IPA server container, and ntpd is not running. The ipaserver-install.log shows that it attempts to start (which returns 0), but the service is not active afterwards: ... 2015-01-14T22:57:02Z DEBUG [4/4]: starting ntpd 2015-01-14T22:57:02Z DEBUG Starting external process 2015-01-14T22:57:02Z DEBUG args='/bin/systemctl' 'start' 'ntpd.service' 2015-01-14T22:57:03Z DEBUG Process finished, return code=0 2015-01-14T22:57:03Z DEBUG stdout= 2015-01-14T22:57:03Z DEBUG stderr= 2015-01-14T22:57:03Z DEBUG Starting external process 2015-01-14T22:57:03Z DEBUG args='/bin/systemctl' 'is-active' 'ntpd.service' 2015-01-14T22:57:04Z DEBUG Process finished, return code=3 2015-01-14T22:57:04Z DEBUG stdout=inactive 2015-01-14T22:57:04Z DEBUG stderr= 2015-01-14T22:57:04Z DEBUG duration: 1 seconds 2015-01-14T22:57:04Z DEBUG Done configuring NTP daemon (ntpd). ... It seems that this causes ntpd on the F21 client to just loop endlessly since it never sees a response. We use ntpdate on F20, which bails out and skips the time update on a F20 client when the server is unavailable: ... 2015-01-15T03:29:11Z DEBUG args=/usr/sbin/ntpdate -U ntp -s -b -v ipa.example.test 2015-01-15T03:29:11Z DEBUG Process finished, return code=1 2015-01-15T03:29:11Z DEBUG stdout= 2015-01-15T03:29:11Z DEBUG stderr= 2015-01-15T03:29:11Z DEBUG Starting external process 2015-01-15T03:29:11Z DEBUG args=/usr/sbin/ntpdate -U ntp -s -b -v ipa.example.test 2015-01-15T03:29:11Z DEBUG Process finished, return code=1 2015-01-15T03:29:11Z DEBUG stdout= 2015-01-15T03:29:11Z DEBUG stderr= 2015-01-15T03:29:11Z WARNING Unable to sync time with IPA NTP server, assuming the time is in sync. Please check that 123 UDP port is opened. ... I can do a 'systemctl start ntpd.service' on the IPA server container, and it does start up successfully. It never seems to automatically start though, even if I restart the IPA server docker container. I did confirm that ntpd.service is enabled with systemctl, yet it doesn't start automatically. The /sbin/ipa-server-configure-first entrypoint script for the server image does a 'systemctl start-enabled' to bring up all of the services, which results in this output in /var/log/systemctl.log: [start-enabled] [start ntpd.service] Running [export OPTIONS=-g -x; /usr/sbin/ntpd -u ntp:ntp $OPTIONS] Marked pid [15] for [ntpd.service] Marked process name [/usr/sbin/ntpd] for [ntpd.service] ... This is the same log output that is generated if I manually run 'systemctl start ntpd.service' from within the container, but the ntpd process stays around when I start it this way. It's hard to tell what might be happening to ntpd, as
