Re: [openstack-dev] [neutron] Is this a bug in metadata proxy...
> > I was wondering if the user/group should be (only) set in a common config, > like neutron.conf, if it should be duplicated in dhcp and metadata config > files, or if the metadata ini should be added to the list of ini files, > when starting up the DHCP agent. > Previously, metadata_proxy_user/group were documented in neutron.conf (when a neutron.conf sample was in github repo) in order to deduce metadata_proxy_socket_mode correctly. You can also define them in both l3/dhcp.ini and metadata-agent.ini config files or set explicitly metadata_proxy_socket_mode in metadata-agent.ini. But it's unrelated as your trouble seems to be linked to a metadata_proxy_watch_log misconfiguration and metadata_proxy_user/group/watch_log are all used by dhcp/l3-agents. With the wrong config, I hit the access denied issue and had no info > indicating that is what has happened. Was wondering if there was any > protection against that misconfiguration case, or way to get an indication > of it. > Before dropping privileges, we cannot detect such access deny to log file (because of features like GRsec,PaX, RBAC). After dropping privileges, we can only log to syslog or stdout if we catch an access deny to log file. Cedric/ZZelle@IRC __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Is this a bug in metadata proxy...
Yeah, the setting for user was neutron. The setting for group was different than the process that started up the proxy (which had user neutron and group neutron). The review 161494, was it applied to Liberty? I think in our case, DHCP agent only had dhcp ini file and not metadata ini file, so it didn't have the user/group setting. I was wondering if the user/group should be (only) set in a common config, like neutron.conf, if it should be duplicated in dhcp and metadata config files, or if the metadata ini should be added to the list of ini files, when starting up the DHCP agent. With the wrong config, I hit the access denied issue and had no info indicating that is what has happened. Was wondering if there was any protection against that misconfiguration case, or way to get an indication of it. P.S. Sorry, I didn't see your reply till now... PCM On Wed, Aug 31, 2016 at 10:36 AM ZZelle wrote: > Hi, > > Are you sure metadata_proxy_user==neutron? > > neutron-metadata-proxy must be able to connect to the metadata-agent > socket and watchs its log files and neutron user should be able to do both > with usual file permissions. > > Otherwise the metadata proxy is generally no more able to: > - watch log[1] so you should set metadata_proxy_watch_log=False > - connect to the metadata-agent because of socket permissions, so you > should set metadata_proxy_socket_mode option[2] in order to let the > metadata agent set the correct perms on metadata socket. > > If you provide metadata_proxy_user/group in l3/dhcp-agent and > metadata-agent config then neutron should be able to deduce both > metadata_proxy_watch_log and metadata_proxy_socket_mode values. > > > > [1] https://review.openstack.org/#/c/161494/ > [2] https://review.openstack.org/#/c/165115/ > > Cédric/ZZelle > > On Wed, Aug 31, 2016 at 2:16 PM, Paul Michali wrote: > >> Hi, >> >> I had seen something and was not sure if this was a subtle bug or not. >> >> I have a Liberty based openstack setup. The account that is setting up >> processes was user=neutron, group=neutron, however the metadata_agent.ini >> config file was set up for a different group. So there was a >> metadata_proxy_user=neutron, and metadata_proxy_group=foo config setting. >> >> This ini file was used by the metadata agent process, but it was not >> included in the DHCP agent process (not sure if I should have included the >> metadata_agent.ini in the startup of DHCP or should have added these two >> metadata proxy settings to neutron.conf, so that they were available to >> DHCP). >> >> In any case, here is what I saw happen... >> >> I created a subnet (not using a router in this setup). It looks like DHCP >> starts up the metadata agent proxy daemon) and the DHCP configuration is >> used, which does NOT include the metadata_proxy_user/group, so the current >> user's uid and gid are used (neutron/neutron) for the >> metadata_proxy_user/group settings. >> >> The proxy calls drop_privileges(), which because the group is different, >> the log file can no longer be accessed by the daemon. An OSError occurs >> with permission denied on the log file for this process, and the process >> exits without any indications. >> >> When I then try to use metadata services it fails (obviously). Looking, >> we see that the metadata service is running (but the proxy is not, and I >> don't see a way for an end user to check that - is there a way?). >> >> Looking in the proxy log, the initial startup messages are seen, showing >> all the configuration settings, and then there is nothing more. No >> indication that it is lowering privileges to run under some other >> user/group, that there was a fatal error, or that it is working and ready >> to process requests. Nothing more appears in the log, as it was working and >> there were no metadata proxy requests occurring. >> >> I was only able to figure it out, by first checking to see if the proxy >> was running, and then manually trying to start the proxy, using the command >> line in the log, under a debugger, to find out that there was a permission >> denied error. >> >> So, it is likely a misconfiguration error on the user's part, but it was >> really hard to figure that out. >> >> Should/could we somehow indicate if there is an error lowering privs? >> >> Is there a (user) way to tell if proxy is running? >> >> Is there some documentation indicating that the proxy user/group settings >> need to be available for both the metadata agent and for other agents that >> may spawn the proxy (DHCP, L3)? >> >> Regards, >> >> PCM >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > __ > OpenStack Development Mailing List (not for usage questions
Re: [openstack-dev] [neutron] Is this a bug in metadata proxy...
Hi, Are you sure metadata_proxy_user==neutron? neutron-metadata-proxy must be able to connect to the metadata-agent socket and watchs its log files and neutron user should be able to do both with usual file permissions. Otherwise the metadata proxy is generally no more able to: - watch log[1] so you should set metadata_proxy_watch_log=False - connect to the metadata-agent because of socket permissions, so you should set metadata_proxy_socket_mode option[2] in order to let the metadata agent set the correct perms on metadata socket. If you provide metadata_proxy_user/group in l3/dhcp-agent and metadata-agent config then neutron should be able to deduce both metadata_proxy_watch_log and metadata_proxy_socket_mode values. [1] https://review.openstack.org/#/c/161494/ [2] https://review.openstack.org/#/c/165115/ Cédric/ZZelle On Wed, Aug 31, 2016 at 2:16 PM, Paul Michali wrote: > Hi, > > I had seen something and was not sure if this was a subtle bug or not. > > I have a Liberty based openstack setup. The account that is setting up > processes was user=neutron, group=neutron, however the metadata_agent.ini > config file was set up for a different group. So there was a > metadata_proxy_user=neutron, and metadata_proxy_group=foo config setting. > > This ini file was used by the metadata agent process, but it was not > included in the DHCP agent process (not sure if I should have included the > metadata_agent.ini in the startup of DHCP or should have added these two > metadata proxy settings to neutron.conf, so that they were available to > DHCP). > > In any case, here is what I saw happen... > > I created a subnet (not using a router in this setup). It looks like DHCP > starts up the metadata agent proxy daemon) and the DHCP configuration is > used, which does NOT include the metadata_proxy_user/group, so the current > user's uid and gid are used (neutron/neutron) for the > metadata_proxy_user/group settings. > > The proxy calls drop_privileges(), which because the group is different, > the log file can no longer be accessed by the daemon. An OSError occurs > with permission denied on the log file for this process, and the process > exits without any indications. > > When I then try to use metadata services it fails (obviously). Looking, we > see that the metadata service is running (but the proxy is not, and I don't > see a way for an end user to check that - is there a way?). > > Looking in the proxy log, the initial startup messages are seen, showing > all the configuration settings, and then there is nothing more. No > indication that it is lowering privileges to run under some other > user/group, that there was a fatal error, or that it is working and ready > to process requests. Nothing more appears in the log, as it was working and > there were no metadata proxy requests occurring. > > I was only able to figure it out, by first checking to see if the proxy > was running, and then manually trying to start the proxy, using the command > line in the log, under a debugger, to find out that there was a permission > denied error. > > So, it is likely a misconfiguration error on the user's part, but it was > really hard to figure that out. > > Should/could we somehow indicate if there is an error lowering privs? > > Is there a (user) way to tell if proxy is running? > > Is there some documentation indicating that the proxy user/group settings > need to be available for both the metadata agent and for other agents that > may spawn the proxy (DHCP, L3)? > > Regards, > > PCM > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] Is this a bug in metadata proxy...
Hi, I had seen something and was not sure if this was a subtle bug or not. I have a Liberty based openstack setup. The account that is setting up processes was user=neutron, group=neutron, however the metadata_agent.ini config file was set up for a different group. So there was a metadata_proxy_user=neutron, and metadata_proxy_group=foo config setting. This ini file was used by the metadata agent process, but it was not included in the DHCP agent process (not sure if I should have included the metadata_agent.ini in the startup of DHCP or should have added these two metadata proxy settings to neutron.conf, so that they were available to DHCP). In any case, here is what I saw happen... I created a subnet (not using a router in this setup). It looks like DHCP starts up the metadata agent proxy daemon) and the DHCP configuration is used, which does NOT include the metadata_proxy_user/group, so the current user's uid and gid are used (neutron/neutron) for the metadata_proxy_user/group settings. The proxy calls drop_privileges(), which because the group is different, the log file can no longer be accessed by the daemon. An OSError occurs with permission denied on the log file for this process, and the process exits without any indications. When I then try to use metadata services it fails (obviously). Looking, we see that the metadata service is running (but the proxy is not, and I don't see a way for an end user to check that - is there a way?). Looking in the proxy log, the initial startup messages are seen, showing all the configuration settings, and then there is nothing more. No indication that it is lowering privileges to run under some other user/group, that there was a fatal error, or that it is working and ready to process requests. Nothing more appears in the log, as it was working and there were no metadata proxy requests occurring. I was only able to figure it out, by first checking to see if the proxy was running, and then manually trying to start the proxy, using the command line in the log, under a debugger, to find out that there was a permission denied error. So, it is likely a misconfiguration error on the user's part, but it was really hard to figure that out. Should/could we somehow indicate if there is an error lowering privs? Is there a (user) way to tell if proxy is running? Is there some documentation indicating that the proxy user/group settings need to be available for both the metadata agent and for other agents that may spawn the proxy (DHCP, L3)? Regards, PCM __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
