Re: Announcing Dwarf: OpenStack API on top of libvirt/kvm
Thanks for testing! On Fri, Apr 11, 2014 at 11:21 AM, Sandro red Mathys r...@fedoraproject.org wrote: On Fri, Apr 11, 2014 at 2:53 PM, Sandro red Mathys r...@fedoraproject.org wrote: On Fri, Apr 11, 2014 at 12:43 AM, Mats Wichmann m...@wichmann.us wrote: On 04/10/14 02:09, Juerg Haefliger wrote: In a nutshell, dwarf is OpenStack API on top of local libvirt/KVM. I suppose it's too late to suggest a name change? dwarf already has long-time usage as the debugging format for various compilers, and there's a matching libdwarf with an api for examining the format. http://dwarfstd.org/ FWIW: It's also not very ideal because one of the OpenStack components (I think it was Trove) used to be called Red Dwarf... Anyway, thanks Juerg for packaging this and putting it into a copr repo as requested. Will give it a try. -- Sandro Let me start by saying I did not see any SELinux issues, before I get to the bad (possibly TL;DR) news. So, on the only system I could try this out, a few things are changed from the defaults. I hope that didn't cause any of the issues I've found. I hope I didn't either. ;) If dwarf is started through systemd, it will log sudo: sorry, you must have a tty to run sudo whenever it is: 1) trying to chown the console-log - and therefore, 'nova console-log server' won't work: 2014-04-11 16:53:23,077 - INFO - dwarf.utils : execute(cmd=['chown', 991, '/var/lib/dwarf/instances/997adba9-ba7c-4f2a-9fff-94e4cfe5d37c/console.log'], check_exit_code=None, shell=False, run_as_root=True) 2014-04-11 16:53:23,106 - WARNING - dwarf.exception : Exception caught: Failed to run command: ['sudo', 'chown', '991', '/var/lib/dwarf/instances/997adba9-ba7c-4f2a-9fff-94e4cfe5d37c/console.log'] exit_code: 1 stdout: stderr: sudo: sorry, you must have a tty to run sudo (500) 2) trying to set up the nat prerouting rule for the ec2 metadata service: 2014-04-11 16:27:22,717 - INFO - dwarf.utils : execute(cmd=['iptables', '-t', 'nat', '-D', 'PREROUTING', '-s', u'', '-d', '169.254.169.254/32', '-p', 'tcp', '-m', 'tcp', '--dport', 80, '-j', 'REDIRECT', '--to-port', 8080], check_exit_code=False, shell=False, run_as_root=True) 2014-04-11 16:27:22,729 - INFO - dwarf.utils : execute(stderr=sudo: sorry, you must have a tty to run sudo ) ...now, working around the issue by starting dwarf by hand instead of through systemd should solve both based on the given sudo errors. I admit I don't understand this. Starting it through systemd works just fine in my env (Fedora cloud image). But it seems the iptables still isn't put into place (but the log shows no issues). At least iptables -L -t nat shows no rules at all (yes, I cleared them beforehand). Now, I noticed that the above log entry shows -s u and wonder if u is ever replaced but since it's not logged anymore, I can't tell. -s is the source IP, i.e., the IP that your instance should receive from dnsmasq. Did it get an IP at all? You should see something like the following in the log: 2014-03-13 13:38:29,201 - INFO - dwarf.compute.servers : _get_server_ip(mac=52:54:00:f7:db:92) : None 2014-03-13 13:38:31,203 - INFO - dwarf.compute.servers : _get_server_ip(mac=52:54:00:f7:db:92) : None 2014-03-13 13:38:33,204 - INFO - dwarf.compute.servers : _get_server_ip(mac=52:54:00:f7:db:92) : None 2014-03-13 13:38:35,205 - INFO - dwarf.compute.servers : _get_server_ip(mac=52:54:00:f7:db:92) : None 2014-03-13 13:38:37,207 - INFO - dwarf.compute.servers : _get_server_ip(mac=52:54:00:f7:db:92) : 192.168.122.157 Since that would result in an error I figure it would turn up in the log if it was an issue. Anyway, because of this cloud-init obviously can't reach the ec2 metadata. I tried to execute the iptables command myself but didn't figure out what I should replace u by. My guesses were either the instance's IP or virbr0's IP or the network address but all failed like: $ sudo iptables -t nat -D PREROUTING -s 192.168.111.1/24 -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-port 8080 iptables: No chain/target/match by that name. (Yes, my libvirt is configured to use non-default IP ranges and yes, I updated dwarf.conf accordingly). That's where my poor network-fu ends. The work around successfully enables the console-log, though. Also, when trying to work around this issue by configuring the ec2 metadata to listen on port 80 directly (and opening that port in the firewall - or just shut it down as I did) and waiting for cloud-init to fall back to the gateway IP instead of using 169.254.169.254, it will be able to connect but get a 404 not found. I'd expect this to work but either you really expect access through the special IP or you don't expect a request for /latest/meta-data/instance-id but only for /2009-04-04/meta-data-instance-id. I removed the workaround to try other things before I came up with this idea and therefore couldn't verify it right now. I'm not a networking expert,
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd
--+
Reporter: mattdm| Owner: janeznemanic
Type: task | Status: new
Priority: normal| Milestone: Fedora 21 (Alpha)
Component: Cloud Base Image | Resolution:
Keywords:|
--+
Comment (by gospo):
Replying to [comment:3 janeznemanic]:
Report on my work.
[...]
I did some research on scripts in /etc/sysconfig/network-scripts. All
those scripts depend on ifcfg-* files in the same directory. But systemd-
networkd doesn't use ifcfg-* files. I removed ifcfg-* files and systemd-
netword worked fine. I was also able to ping google. But if I tried to use
for example ifup or ifdown script then that script failed. Because it
couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and
ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove
/usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I
think we also don't need to use /etc/rc.d/init.d/network. I removed that
file and networking worked fine with systemd-networkd.
So to sum up. If systemd-networkd service is used it is possible in my
opinion to remove scripts in /etc/sysconfig/network-scripts and
/etc/rc.d/init.d/network script. Besides that it is possible to remove
NetworkManager.
I took a quick look at the files from initscripts that are likely
networking related. It seems possible to remote these from initscripts
core and (as Johannes suggested above) possibly break initscripts out into
a new package. If there was a desire to add the missing functionality to
networkd that is currently in initscripts then the new initscripts package
could be slowly whittled away to nothing when ppp and netconsole support
are added to networkd (that appears to be the only missing functionality).
/etc/NetworkManager
/etc/NetworkManager/dispatcher.d
/etc/NetworkManager/dispatcher.d/00-netreport
/etc/networks
/etc/ppp
/etc/ppp/ip-down
/etc/ppp/ip-down.ipv6to4
/etc/ppp/ip-up
/etc/ppp/ip-up.ipv6to4
/etc/ppp/ipv6-down
/etc/ppp/ipv6-up
/etc/ppp/peers
/etc/rc.d/init.d/netconsole
/etc/rc.d/init.d/network
/etc/sysconfig/netconsole
/etc/sysconfig/network-scripts
/etc/sysconfig/network-scripts/*
/usr/bin/ipcalc
/usr/lib/udev/rename_device
/usr/lib/udev/rules.d/60-net.rules
/usr/libexec/initscripts
/usr/libexec/initscripts/legacy-actions
/usr/sbin/ifdown
/usr/sbin/ifup
/usr/sbin/netreport
/usr/sbin/ppp-watch
/usr/sbin/usernetctl
/usr/share/doc/initscripts/changes.ipv6
/usr/share/doc/initscripts/ipv6-6to4.howto
/usr/share/doc/initscripts/ipv6-tunnel.howto
/usr/share/doc/initscripts/static-routes-ipv6
/usr/share/man/man1/genhostid.1.gz
/usr/share/man/man1/ipcalc.1.gz
/usr/share/man/man1/netreport.1.gz
/usr/share/man/man8/ifdown.8.gz
/usr/share/man/man8/ifup.8.gz
/usr/share/man/man8/ppp-watch.8.gz}}}
'''Thoughts? Suggestions? Ideas?'''
One more thing that's been bothering me for a few days now. How does
systemd name network interfaces when Fedora is run on top of different
hypervisors? If xen or kvm is used how does systemd name network
interfaces?
Systemd will name the device based on the bus/device/function of the
interfaces, so there could be some differences in the name chosen
depending on the hypervisor used (since each hypervisor may chose a
difference bus/dev/function for a given emulated interface.
--
Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:5
cloud https://fedorahosted.org/cloud
Fedora Cloud Working Group Ticketing System
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by mattdm): Replying to [comment:5 gospo]: One more thing that's been bothering me for a few days now. How does systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces? Systemd will name the device based on the bus/device/function of the interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface. It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a *single* device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one). -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:7 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by mattdm): Also, note that we are not using NetworkManager -- we're using the traditional network initscripts. It would be nice to eventually move away from these, and I don't have a strong view over which one should win (once NetworkManager gains the ability to configure a dhcp interface and then stop running). From my (cloud SIG) point of view, NetworkManager has the disadvantage of being dependency heavy (and the run-once mode is not yet done -- see https://bugzilla.redhat.com/show_bug.cgi?id=863515), but the significant advantage of parsing the existing config files. If there were a generator for systemd-networkd which could read the basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case) -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:8 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by gospo): Replying to [comment:7 mattdm]: It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a *single* device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one). I was thinking about the fact that the slot-naming-rules could be modified so that the first NIC found was named em1, second em2, etc, but disabling the rules completely seems fine. -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:9 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by johannbg): As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people *wait* until the networkd code in systemd stabilizes before proceeding any further with this -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:10 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd
--+
Reporter: mattdm| Owner: janeznemanic
Type: task | Status: new
Priority: normal| Milestone: Fedora 21 (Alpha)
Component: Cloud Base Image | Resolution:
Keywords:|
--+
Comment (by janeznemanic):
Replying to [comment:6 gospo]:
Replying to [comment:3 janeznemanic]:
But if I tried to use for example ifup or ifdown script then that
script failed. Because it couldn't source ifcfg-*. I think so, I might be
wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might
be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that
so I'm not sure. I think we also don't need to use
/etc/rc.d/init.d/network. I removed that file and networking worked fine
with systemd-networkd.
Sorry, I forgot to address this in my last post. As was mentioned above
there are some who will want to preserve some of the ability to use
ifup/ifdown in some manner. I know that controlling the network
interfaces is a current TODO for networkd, but it would seem reasonable
that the networkd unit can be stopped/started or restarted if one wanted
to reset the network config, correct?
You can do ''systemctl {start,stop,restart} systemd-networkd.service'' to
control networking.
I'm not fedora networking expert, but I think it is possible to use ip
command instead of ifup/ifdown to bring interface up or down.
--
Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:11
cloud https://fedorahosted.org/cloud
Fedora Cloud Working Group Ticketing System
___
cloud mailing list
cloud@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/cloud
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by mattdm): Replying to [comment:10 johannbg]: As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people *wait* until the networkd code in systemd stabilizes before proceeding any further with this Let's start collecting a list of basic requirements. Just to brainstorm a bit * Single eth0 interface with DHCP * Possible other interfaces, also with DHCP (common in OpenNebula) * Static ipv4 configuration * ipv6? * Nice-to-have: ifcfg compatibility (basic config files, ifup/ifdown commands) Some use cases for basic use and for more advanced functionality like tunneling would help make a prioritized list. -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:13 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by janeznemanic): Replying to [comment:8 mattdm]: Also, note that we are not using NetworkManager -- we're using the traditional network initscripts. It would be nice to eventually move away from these, and I don't have a strong view over which one should win (once NetworkManager gains the ability to configure a dhcp interface and then stop running). From my (cloud SIG) point of view, NetworkManager has the disadvantage of being dependency heavy (and the run-once mode is not yet done -- see https://bugzilla.redhat.com/show_bug.cgi?id=863515), but the significant advantage of parsing the existing config files. If there were a generator for systemd-networkd which could read the basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case) Found out today on mailing list that cloud image is not using NetworkManager?. To me the idea of having both NetworkManager? and systemd-networkd in the cloud image is a little bit strange since both are network managers. I assume that NetworkManager? is superior to systemd- networkd, but I think systemd-networkd does it's job good in virtualized environment. Anyway I'm not an expert so that's just my opinion. -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:14 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: [cloud] #14: Investigate systemd-networkd
#14: Investigate systemd-networkd --+ Reporter: mattdm| Owner: janeznemanic Type: task | Status: new Priority: normal| Milestone: Fedora 21 (Alpha) Component: Cloud Base Image | Resolution: Keywords:| --+ Comment (by janeznemanic): Replying to [comment:13 mattdm]: Replying to [comment:10 johannbg]: As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people *wait* until the networkd code in systemd stabilizes before proceeding any further with this Let's start collecting a list of basic requirements. Just to brainstorm a bit * Single eth0 interface with DHCP * Possible other interfaces, also with DHCP (common in OpenNebula) * Static ipv4 configuration * ipv6? * Nice-to-have: ifcfg compatibility (basic config files, ifup/ifdown commands) Some use cases for basic use and for more advanced functionality like tunneling would help make a prioritized list. Which other interfaces do you have in mind? Bridge? -- Ticket URL: https://fedorahosted.org/cloud/ticket/14#comment:16 cloud https://fedorahosted.org/cloud Fedora Cloud Working Group Ticketing System ___ cloud mailing list cloud@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/cloud Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
