Re: [Lxc-users] lxc setup on a grsec enabled kernel
On Sun, 9 Jan 2011 13:25:53 +0100, Patrick Winnertz win...@der-winnie.de wrote: Hello, I've tried the last days hard to set up working lxc containers on a grsec enabled kernel. However I failed everytime with several error msgs and/or kernel oopses. After booting in the grsec kernel I've verified with gradm that RBAC is disabled to start the containers first: gradm -D lxc-start -n example however I get then first an error that /dev/pts can't be mounted and afterwards a kernel oops, which you can find attached to this mail - it seems to be some troubles with veth networking. I've straced the process and this is the output (strace-lxc1): 335:read(16, lxc-start: Operation not permitted - failed to mount a new instance of '/dev/pts' 336:lxc-start: failed to setup the new pts instance 337:lxc-start: failed to setup the container 344:write(2, failed to spawn 'web', 21failed to spawn 'web') = 21 358:write(2, Device or resource busy - failed..., 63Device or resource busy - failed to remove cgroup '/cgroup/web') = 63 After a reboot I tried again, but this time I switched into the learning mode of grsec.. now the kernel oops is gone, however I'm getting now this error msg (output from strace (strace-lxc2)): failed to create vethde3FDA-veth..., 64failed to create vethde3FDA-vethelGBjP : Operation not permitted) = 64 295:write(2, failed to create netdev, 23failed to create netdev) = 23 299:write(2, failed to create the network, 28failed to create the network) = 28 305:write(2, failed to spawn 'web', 21failed to spawn 'web') = 21 319:write(2, No such file or directory - fail..., 65No such file or directory - failed to remove cgroup '/cgroup/web') = 65 It would be nice if someone could give me hints or advices what is going wrong here and how to fix it. Full strace output of both lxc-start runs is also attached to the mail Greetings Patrick I can tell you I ran into similar oopses, haven't tested with learning mode though. What I did was disable CONFIG_PAX_KERNEXEC, which conflicts with CONFIG_PARAVIRT_GUEST and/or CONFIG_KVM_GUEST anyway (I was running the kernel under KVM; wish this conflict would be documented anywhere). After that, I could successfully start LXC guests without crashes. It was on 2.6.32.2-grsec if it matters. Yes, it is a workaround, and it does not help security of the system, but it's the best I can suggest. -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] how to use routing with LXC?
On Fri, Jan 7, 2011 at 5:39 PM, Mike deb...@good-with-numbers.com wrote: The instructions that I've seen for LXC suggest creating a bridge in the host, placing its name in lxc.network.link. On a diskless system I have eth0 eth1, and create the bridge on eth1. I can't put eth0 in a bridge, because it's the port for the NFS root. But when I want traffic to go from the container's port to (the host's) eth0, I don't see how to direct that--I don't think that's even possible. It instead goes out eth1 to the next hop, where the eth0 address isn't even routeable. So it seems that a router configuration for LXC is what I want. I've done this in Xen, using their vif-route script. How would that work with LXC? -- Gaining the trust of online customers is vital for the success of any company that requires sensitive data to be transmitted over the Web. Learn how to best implement a security strategy that keeps consumers' information secure and instills the confidence they need to proceed with transactions. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users Did you try macvlan instead of veth? ~Nirmal -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] how to use routing with LXC?
Nirmal Guhan wrote: Did you try macvlan instead of veth? No, it looked bleeding edge, and it didn't occur to me how I could use it. How would one? What is it? -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] note on using rsyslog in a container
Mike deb...@good-with-numbers.com writes: This has sort of been mentioned earlier on this list. I noticed netfilter messages getting trashed in the various /var/log/messages on a system with two containers, netfilter rules on the host, and each container and the host running rsyslog. On closer inspection, I realized that only every other character or so of the message was appearing in a given log file. Disabling kernel logging in the containers, by commenting out $ModLoad imklog in /etc/rsyslog.conf, straightened out the log files. Huh. I disabled that because (containers') rsyslog reported it couldn't read from the kernel log. Maybe you've accidentally left containers with read-access to arbitrary devices? # These are: null zero random urandom tty* tty console # ptmx pts/* rtc0 lxc.cgroup.devices.deny = a lxc.cgroup.devices.allow = c 1:3 rw lxc.cgroup.devices.allow = c 1:5 rw lxc.cgroup.devices.allow = c 1:8 rw lxc.cgroup.devices.allow = c 1:9 rw lxc.cgroup.devices.allow = c 4:* rw lxc.cgroup.devices.allow = c 5:0 rw lxc.cgroup.devices.allow = c 5:1 rw lxc.cgroup.devices.allow = c 5:2 rw lxc.cgroup.devices.allow = c 136:* rw lxc.cgroup.devices.allow = c 254:0 r And in the container, I configure rsyslog thusly: cat /etc/dhcp3/dhclient-exit-hooks.d/lxc-postinst EOF case \$reason in BOUND|RENEW|REBIND|REBOOT) :;; *) return;; esac exec /dev/console 21 /dev/null rm -f /etc/dhcp3/dhclient-exit-hooks.d/lxc-postinst # delete self # [...] perl -p0 -i -e 's:(.|\\\n)*/dev/xconsole\$::' /etc/rsyslog.d/50-default.conf sed -i '/ModLoad imklog\|KLogPath/ s/^/#/' /etc/rsyslog.conf etckeeper commit Suppress noise in logs. sed -i '/RSYSLOG_TraditionalFileFormat/ s/^/#/' /etc/rsyslog.conf cat /etc/rsyslog.d/20-to-logserv.conf -EOT # Enable RFC 3339 (ns granularity) timestamps in CC'd logs. \\\$ActionForwardDefaultTemplate RSYSLOG_ForwardFormat # CC all logs to logserv. *.* @logserv EOT etckeeper commit Enable RFC 3339 timestamps in logs. restart rsyslog # [...] EOF -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] concurrent aptitude/dpkg runs in separate containers -- bork bork bork
I can provision a new LXC container, which includes running a few
aptitude install foo lines (inside the containers), and it Just Works.
If I try to provision two containers at the same time, both containers
appear to hang with a dpkg process in the D state[0].
Has anybody run into this before?
I'm invoking aptitude in the post-install hook with triggered when
ISC dhclient acquires a DHCP address. Where /etc/lxc/$name.setup
contains lines like aptitude install -yq foo,
chroot $target_dir tee /dev/null /root/lxc-setup /etc/lxc/$name.setup
chroot $target_dir chmod +x /root/lxc-setup
[...]
chroot $target_dir tee /dev/null
/etc/dhcp3/dhclient-exit-hooks.d/lxc-postinst EOF
case \$reason in
BOUND|RENEW|REBIND|REBOOT) :;;
*) return;;
esac
exec /dev/console 21 /dev/null
rm -f /etc/dhcp3/dhclient-exit-hooks.d/lxc-postinst # delete self
if $debug_p
then ${TERM+export TERM=$TERM} # colour during debug build
PS4=\`tput setaf 2||:\`\$PS4\`tput sgr0||:\`
set -x
fi
export DEBIAN_FRONTEND=noninteractive
export HOME=\`mktemp -d\` # needed for apt-etckeeper-git
git config --global user.name root
git config --global user.email r...@\`hostname --fqdn\`
etckeeper commit Generate new SSL host key.
# Generate new SSH host keys.
rm /etc/ssh/ssh_host_???_key*
dpkg-reconfigure openssh-server
etckeeper commit Generate new SSH host keys.
test -x /root/lxc-setup /root/lxc-setup
etckeeper commit Ran container-specific posthook.
EOF
I'm using ubuntu 10.04 for both dom0 and containers, with its stock
2.6.32 kernel, and an lxc 0.7.3 from natty. My config for each
container looks like this:
# Created 2011-01-12 11:31:11.151135457+11:00
# Created 2011-01-12 01:05:41.488025425+11:00 (template)
lxc.utsname = proud
lxc.console = /var/log/lxc/proud.console
lxc.rootfs = /srv/lxc/proud
lxc.tty = 1
lxc.pts = 1024
lxc.network.type = veth
lxc.network.link = br-managed
lxc.network.name = managed
lxc.network.flags = up
# Note: memsw is memory+swap; it CANNOT be less than memory alone.
lxc.cgroup.memory.soft_limit_in_bytes = 128M
lxc.cgroup.memory.limit_in_bytes = 256M
lxc.cgroup.memory.memsw.limit_in_bytes = 512M
# These are: null zero random urandom tty* tty console ptmx pts/* rtc0
lxc.cgroup.devices.deny = a
lxc.cgroup.devices.allow = c 1:3 rw
lxc.cgroup.devices.allow = c 1:5 rw
lxc.cgroup.devices.allow = c 1:8 rw
lxc.cgroup.devices.allow = c 1:9 rw
lxc.cgroup.devices.allow = c 4:* rw
lxc.cgroup.devices.allow = c 5:0 rw
lxc.cgroup.devices.allow = c 5:1 rw
lxc.cgroup.devices.allow = c 5:2 rw
lxc.cgroup.devices.allow = c 136:* rw
lxc.cgroup.devices.allow = c 254:0 r
# Prevent container from using mount(8), esp. remounting its root filesystem
-oro.
# This necessitates mounting *at least* /proc outside.
lxc.cap.drop = sys_admin
lxc.mount.entry = none /srv/lxc/proud/dev/shm tmpfs nosuid,nodev
lxc.mount.entry = none /srv/lxc/proud/lib/init/rw tmpfs
mode=0755,nosuid,size=8m
lxc.mount.entry = none /srv/lxc/proud/proc proc nodev,noexec,nosuid
#lxc.mount.entry = none /srv/lxc/proud/proc/sys/fs/binfmt_misc binfmt_misc
nodev,noexec,nosuid
#lxc.mount.entry = none /srv/lxc/proud/sys sysfs nodev,noexec,nosuid
#lxc.mount.entry = none /srv/lxc/proud/sys/fs/fuse/connections fusectl defaults
#lxc.mount.entry = none /srv/lxc/proud/sys/kernel/debug debugfs defaults
#lxc.mount.entry = none /srv/lxc/proud/sys/kernel/security securityfs defaults
lxc.mount.entry = none /srv/lxc/proud/tmp tmpfs defaults
lxc.mount.entry = none /srv/lxc/proud/var/lock tmpfs
nodev,noexec,nosuid,size=8m
# This mount would break lxc-start's halt/reboot autodetection (in lxc 0.7.x).
#lxc.mount.entry = none /srv/lxc/proud/var/run tmpfs mode=0755,nosuid,size=8m
# Data mountpoints
lxc.mount.entry = /srv/mirror /srv/lxc/proud/srv/mirror none bind,ro
lxc.mount.entry = /home /srv/lxc/proud/home none bind
lxc.mount.entry = /srv/squid /srv/lxc/proud/var/spool/squid none bind
# Disabled because their absence causes problems:
#chown net_admin setgid # getty or login
#net_bind_service net_raw net_broadcast # dhclient
#setuid # rsyslog
#sys_chroot # openssh-server
#fowner dac_override dac_read_search# lots of things (like root_squash)
#kill # needed by default to stop
rsyslogd/slapd
# Disabled because I *think* they're harmless:
#fsetid ipc_lock ipc_owner lease sys_nice sys_ptrace
lxc.cap.drop = audit_control audit_write linux_immutable mac_admin
lxc.cap.drop = mac_override mknod setfcap setpcap sys_admin sys_boot
lxc.cap.drop = sys_module sys_pacct sys_rawio sys_resource sys_time
lxc.cap.drop = sys_tty_config
[0] To make matters worse, attempting to cleanly shut down those
containers causes them to run /bin/sync, which also hangs in D!
I've workaround that part by replacing /bin/sync with a link to
/bin/true in
[Lxc-users] Forwarding packets from host to container
Hi, How do I forward packets (ethernet frames included) from host to container. I plan to run a packet capture program (tcpdump for instance) within container that will capture the packets coming to host eth1 interface. I tried both using bridge and iptables but they do not seem to help. iptables -A FORWARD -i eth1 -o br1 -j ACCEPT and/or iptables -A FORWARD -i eth1 -o vethZtPPol -j ACCEPT Instead of the above, I also tried adding host eth1 to br1 but still tcpdump from container cannot see the packets sent to eth1 from external world. I use fedora 12 for both host and container. xc.network.type = veth lxc.network.link = br1 lxc.network.name = eth1 lxc.network.flags = up lxc.network.mtu = 1500 -Nirmal -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Forwarding packets from host to container
On Tue, Jan 11, 2011 at 5:25 PM, Nirmal Guhan vavat...@gmail.com wrote: Hi, How do I forward packets (ethernet frames included) from host to container. I plan to run a packet capture program (tcpdump for instance) within container that will capture the packets coming to host eth1 interface. I tried both using bridge and iptables but they do not seem to help. iptables -A FORWARD -i eth1 -o br1 -j ACCEPT and/or iptables -A FORWARD -i eth1 -o vethZtPPol -j ACCEPT Instead of the above, I also tried adding host eth1 to br1 but still tcpdump from container cannot see the packets sent to eth1 from external world. I use fedora 12 for both host and container. xc.network.type = veth lxc.network.link = br1 lxc.network.name = eth1 lxc.network.flags = up lxc.network.mtu = 1500 -Nirmal An update : If I connect host eth1 to a bridge br2 and add lxc.network.type = veth lxc.network.link = br2 lxc.network.name = eth2 lxc.network.flags = up lxc.network.mtu = 1500 I can then see packets coming into eth2 (basically echo reply from external machine) but not the ones going out. Kindly help. -Nirmal -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Forwarding packets from host to container
On Tue, Jan 11, 2011 at 5:34 PM, Nirmal Guhan vavat...@gmail.com wrote: On Tue, Jan 11, 2011 at 5:25 PM, Nirmal Guhan vavat...@gmail.com wrote: Hi, How do I forward packets (ethernet frames included) from host to container. I plan to run a packet capture program (tcpdump for instance) within container that will capture the packets coming to host eth1 interface. I tried both using bridge and iptables but they do not seem to help. iptables -A FORWARD -i eth1 -o br1 -j ACCEPT and/or iptables -A FORWARD -i eth1 -o vethZtPPol -j ACCEPT Instead of the above, I also tried adding host eth1 to br1 but still tcpdump from container cannot see the packets sent to eth1 from external world. I use fedora 12 for both host and container. xc.network.type = veth lxc.network.link = br1 lxc.network.name = eth1 lxc.network.flags = up lxc.network.mtu = 1500 -Nirmal An update : If I connect host eth1 to a bridge br2 and add lxc.network.type = veth lxc.network.link = br2 lxc.network.name = eth2 lxc.network.flags = up lxc.network.mtu = 1500 I can then see packets coming into eth2 (basically echo reply from external machine) but not the ones going out. Kindly help. -Nirmal A typo : packets coming into eth1 of the host... -- Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
