[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 Mark Linimon changed: What|Removed |Added See Also||https://bugs.freebsd.org/bu ||gzilla/show_bug.cgi?id=2769 ||36 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 --- Comment #9 from Mina Galić --- (In reply to Thomas Hurst from comment #8) you wanna prepare a patch? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 --- Comment #8 from Thomas Hurst --- A small note in mac_portacl(4) that the reservedlow/high oids are per-vnet - in contrast with the global mac_portacl policy - would probably have made all the difference here. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 Zhenlei Huang changed: What|Removed |Added Status|Open|Closed Resolution|--- |Works As Intended --- Comment #7 from Zhenlei Huang --- Close as that is expected behavior. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 --- Comment #6 from Zhenlei Huang --- (In reply to uros from comment #1) > Anyone have any idea what could be checked here? For vnet jails, adjust vnet sysctl `net.inet.ip.portrange.reservedhigh` appropriately. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 --- Comment #5 from Zhenlei Huang --- (In reply to Zhenlei Huang from comment #4) > Tree indeed. Sorry for the typo :( It was `True indeed`. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 Zhenlei Huang changed: What|Removed |Added CC||z...@freebsd.org --- Comment #4 from Zhenlei Huang --- (In reply to Thomas Hurst from comment #3) > Investigating this finds 'net.inet.ip.portrange.reservedhigh=1023' within a > vnet jail, so mac_portacl doesn't get a chance to do anything. The sysctl variable `net.inet.ip.portrange.reservedhigh` is a per-vnet one. So it is default to 1023 for vnet jail. That is expected. ``` /* * Reserved ports accessible only to root. There are significant * security considerations that must be accounted for when changing these, * but the security benefits can be great. Please be careful. */ VNET_DEFINE(int, ipport_reservedhigh) = IPPORT_RESERVED - 1;/* 1023 */ VNET_DEFINE(int, ipport_reservedlow); SYSCTL_INT(_net_inet_ip_portrange, OID_AUTO, reservedhigh, CTLFLAG_VNET | CTLFLAG_RW | CTLFLAG_SECURE, &VNET_NAME(ipport_reservedhigh), 0, ""); ``` > As on the host this needs to be set to 0 to allow mac_portacl to operate, > though to change this I had to drop the initial jail securelevel. Tree indeed. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 Thomas Hurst changed: What|Removed |Added CC||t...@hur.st --- Comment #3 from Thomas Hurst --- Investigating this finds 'net.inet.ip.portrange.reservedhigh=1023' within a vnet jail, so mac_portacl doesn't get a chance to do anything. As on the host this needs to be set to 0 to allow mac_portacl to operate, though to change this I had to drop the initial jail securelevel. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 259149] mac_portacl not in affect when running VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 Mina Galić changed: What|Removed |Added Status|New |Open Assignee|b...@freebsd.org|j...@freebsd.org CC||free...@igalic.co --- Comment #2 from Mina Galić --- assigning to jail@ for now, maybe someone there can make sense of this -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #8 from Alex Seitsinger --- (In reply to Antranig Vartanian from comment #1) ping -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #7 from Alex Seitsinger --- 1. Start a jail 2. Using exec.created/exec.poststart, successfully move the epair interface into the jail via ifconfig vnet. 3. Run another script via exec.created/exec.poststart, but have it fail, causing the jail's creation to fail and exit. 4. Observe that the first epair interface still exists on the host via ifconfig. 5. Some seconds to minutes following this, the kernel will panic and reboot the system. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #6 from Alex Seitsinger --- Created attachment 243906 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=243906&action=edit create-jail-epair.sh -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #5 from Alex Seitsinger --- Created attachment 243905 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=243905&action=edit bridge-jail-epair.sh -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #4 from Alex Seitsinger --- Created attachment 243904 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=243904&action=edit create-bridge.sh -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #3 from Alex Seitsinger --- Created attachment 243903 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=243903&action=edit logs0/scripts/created -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 --- Comment #2 from Alex Seitsinger --- Created attachment 243902 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=243902&action=edit /etc/jail.conf bastille jail -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 Antranig Vartanian changed: What|Removed |Added CC||antran...@freebsd.am --- Comment #1 from Antranig Vartanian --- Can you send a jail.conf and commands that you use for replication? I just tried this on my machine and it worked fine. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 272958] jail creation failure with vnet epair leaves host interface lingering
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272958 Mark Linimon changed: What|Removed |Added Assignee|b...@freebsd.org|j...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug.
vnet & mac_portacl
Hi, Would virtualizing mac_portacl be a low hanging fruit? It seems rather interesting for rootless vnet jails Thanks, Nikos
[Bug 251624] [jls] is unable to list IP(s) of vnet jails.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251624 Alexander V. Chernikov changed: What|Removed |Added Summary|[jls] [ifconfig] is unable |[jls] is unable to list |to list IP(s) of vnet |IP(s) of vnet jails. |jails. | CC||melif...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #31 from Zhenlei Huang --- As a workaround, if such setup as (comment #11) is mandatary: em0 -- vm-sw1 -- epair0b -- epair0a(connected to host) epair0a.2 -- vm-sw2 -- jails vlan 2 epair0a.4 -- vm-sw4 -- jails vlan 4 epair0a.6 -- vm-sw6 -- jails vlan 6 epair0a.8 -- vm-sw8 -- jails vlan 8 Let em0, vm-sw1 and epair0b be pure layer 2 interfaces. Set IP/IPv6 addresses on epair0a as required. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #30 from Zhenlei Huang --- (In reply to Kristof Provost from comment #13) Let bridge(4) ignore all packets with vlan tag might be too aggressive. All tagged packets are ignored. I'd propose to make bridge(4) decide by configuration. That is something similar with hardware switches. Some syntax like this: ``` # ifconfig bridge0 vlan 10,20,100-200 # ifconfig bridge0 addm em0 link-type trunk # ifconfig bridge0 addm em1 link-type hybrid # ifconfig bridge0 addm em2 link-type access # ifconfig bridge0 addm em0 trunk vlan 10,100-110 # ifconfig bridge0 addm em1 hybrid vlan all # ifconfig bridge0 addm em2 access vlan 20 ``` Then bridge(4) determines to accept tagged / untagged packets by checking the configuration of port member. For example, as the syntax above, bridge0 is interested in vlan 10,20,100-200, any packets received on em1 without vlan tag 10,20,100-200 will be ignored and returned for local processing. As for em2, tagged packets are ignored, and untagged packets will be add vlan tag 20 and processed normally (by bridge0). -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #29 from Zhenlei Huang --- (In reply to kvs from comment #28) I think your should open a separate PR, as you have different setup with that of the original PR by John Westbrook. He has SR-IOV configured. I managed to repeat with cxl / lagg / bridge / epair (vnet) on 13.2-RC3. Also tried re / ue . > tcpdump -i cc0: > 10:00:17.981050 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 42 > tcpdump -i cc1: > 10:00:17.981041 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 > 10:00:17.981282 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui > Unknown), length 46 You might want to tcpdump on cc0 with `--direction=in` to filter ARP request send out from cc1 and then come back to cc0 (the switch forwarded it). The IF_BRIDGE(4) seems to hide some thing to protect itself get confused. If you can confirm, then please config you switch properly. The two ports cc0 and cc1 connected should be in same link aggregation group. I'll see if I can teach IF_BRIDGE(4) to emit warnings in case it get ARP request packet sent from it self. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #28 from kvs --- (In reply to Slawomir Wojciech Wojtczak from comment #27) I have some headway on my end, though I don't know how much it's related to the earlier bugs at this point. After further testing, vlans apparently aren't related to my problem. The problem occurs on lagg without vlan interfaces. When a jail+VNET (on bridge) sends an ARP request it traverses the bridge and exits both interfaces in the host lagg group. When the ARP reply comes back, it appears it will only ever enter the host bridge if it comes in on the primary lagg member. I'm not certain this is exclusive to vnets, also possibly this is normal operation for laggs using lacp? Lab test: lagg0 (ports cc0 + cc1), bridge2020 (members epair0a & lagg0) ping from jail+VNET to switch (10.20.20.254), using source epair0b (10.20.20.77) (epair0b -> epair0a -> bridge2020 -> lagg0 -> cc0/cc1 -> switch) tcpdump -i epair0a 10:00:17.981011 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 tcpdump -i bridge2020 10:00:17.981051 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 tcpdump -i lagg0 10:00:17.981030 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 10:00:17.981282 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 tcpdump -i cc0: 10:00:17.981050 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 42 tcpdump -i cc1: 10:00:17.981041 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 10:00:17.981282 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 Arp table is not populated on VM, as bridge2020 and epair0a/b never sees ARP reply come in over cc1. I believe in my case specifically the switch is seeing cc1 as the primary lagg member while the FreeBSD server sees cc0 as the primary lagg member. When ARP replies manage to come in over cc0, the ARP replies make it to the vnet interface and the jail populates its ARP table. I can force this event by downing cc1 or shutting down the cc1 switch port (in both cases it appears the switch then identifies cc0 as the primary lagg member over which it sends ARP replies). Alternatively, if both cc0 and cc1 are up, and the switch sends an ARP reply over cc0 (has happened randomly), the ARP reply does makes it through the bridge/epair and populates the ARP cache on the VM. Example after ifconfig cc1 down: tcpdump -i epair0a 10:48:18.949695 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 10:48:18.950041 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 tcpdump -i bridge2020 10:48:18.949731 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 10:48:18.950041 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 tcpdump -i lagg0 10:48:18.949711 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 10:48:18.950041 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 tcpdump -i cc0 10:48:18.949722 ARP, Request who-has 10.20.20.254 tell 10.20.20.77, length 28 10:48:18.950041 ARP, Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 ARP table on VM is now populated with switch address, and everything appears to work as normal over lagg0 (with cc0 up / cc1 down). In the mean time I've managed to get the switch configured to send L2/ARP over both lagg members which has fixed the immediate problem. Though I do think it's strange that FreeBSD populates the ARP table just fine on the host over cc1, but just wont send that ARP reply over the bridge interface unless it comes in on cc0. That *feels* like a bug, as it only seems to affect the second interface on a lagg that's in a bridge, and quite possibly only for layer 2 (L2/3 needs further testing - I've not lost packets once the arp table is populated, but it's possible the switch was handling layer 3 differently and always using the cc0 port, in which case FreeBSD would probably send over the bridge without trouble). Testing has been performed on 14-CURRENT and 13-STABLE with identical results. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 Slawomir Wojciech Wojtczak changed: What|Removed |Added CC||verma...@interia.pl --- Comment #27 from Slawomir Wojciech Wojtczak --- Hi, any progress on this one? Will it be fixed in 13.2-RELEASE or the little later upcoming 14.0-RELEASE? I ask because my buddy just hit it again with 13.1-RELEASE today ... Regards, vermaden -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
oui Unknown), ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 0x: 0001 0800 0604 0002 0211 2233 4455 0a14 .."3DU.. 0x0010: 14fe 0207 f080 de0b 0a14 1406 0x0020: .. 15:23:10.623924 02:11:22:33:44:55 (oui Unknown) > 02:07:f0:80:de:0b (oui Unknown), ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 0x: 0001 0800 0604 0002 0211 2233 4455 0a14 .."3DU.. 0x0010: 14fe 0207 f080 de0b 0a14 1406 0x0020: .. 15:23:10.623926 02:11:22:33:44:55 (oui Unknown) > 02:07:f0:80:de:0b (oui Unknown), ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 10.20.20.254 is-at 02:11:22:33:44:55 (oui Unknown), length 46 0x: 0001 0800 0604 0002 0211 2233 4455 0a14 .."3DU.. 0x0010: 14fe 0207 f080 de0b 0a14 1406 0x0020: .. 15:23:10.623943 02:07:f0:80:de:0b (oui Unknown) > 02:11:22:33:44:55 (oui Unknown), ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 56841, offset 0, flags [none], proto ICMP (1), length 84) 10.20.20.6 > 10.20.20.254: ICMP echo request, id 22927, seq 0, length 64 0x: 4500 0054 de09 4001 5f74 0a14 1406 E..T@._t 0x0010: 0a14 14fe 0800 8750 598f 0006 2ec0 ...PY... 0x0020: 15c1 e795 0809 0a0b 0c0d 0e0f 1011 1213 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 36374567 15:23:10.624147 02:11:22:33:44:55 (oui Unknown) > 02:07:f0:80:de:0b (oui Unknown), ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 54016, offset 0, flags [none], proto ICMP (1), length 84) 10.20.20.254 > 10.20.20.6: ICMP echo reply, id 22927, seq 0, length 64 0x: 4500 0054 d300 4001 6a7d 0a14 14fe E..T@.j} 0x0010: 0a14 1406 8f50 598f 0006 2ec0 ...PY... 0x0020: 15c1 e795 0809 0a0b 0c0d 0e0f 1011 1213 0x0030: 1415 1617 1819 1a1b 1c1d 1e1f 2021 2223 .!"# 0x0040: 2425 2627 2829 2a2b 2c2d 2e2f 3031 3233 $%&'()*+,-./0123 0x0050: 3435 36374567 (arp cache seems valid as well) jail-10-20-20-6# arp -na ? (10.20.20.6) at 02:07:f0:80:de:0b on epair0b permanent [ethernet] ? (10.20.20.254) at 02:11:22:33:44:55 on epair0b expires in 1085 seconds [ethernet] Additional thoughts: 1) With lagg0, cc0, and cc1 up, I created a second jail on host25 using 10.20.20.7 (epair1). I add epair1a to bridge2020 (now including epair0a, epair1a and lagg0.2020). When I attempt to ping from jail-10-20-20-6 to .254 I get a timeout as previously experienced. Pinging from .6 to .7 appears to work without any trouble, if lagg0 has any cc0/1 members up or down. This was expected, as packets should never traverse lagg0.2020, but I did want to test/confirm. 2) I did run some ping tests with untagged lagg0 in the bridge, and it does appear it's working without trouble. I removed lagg0.2020 from bridge2020, then added lagg0 to bridge2020, and set the switch ports as untagged in the switch. The packets appear to move without trouble even with both cc0+cc1 up. I need to further test this to be conclusive, but this felt less important to perform at this time as it doesn't solve the requirement I need of tagged ports. 3) I have a few bhyve vm's that I've added as tests, tap0, tap1, etc to the bridge2020. The results seem to be largely consistent with jails. You could replace jail-10-20-20-6, with vm-10-20-20-11 (tested freebsd / openbsd / windows) for instance, and these same results appear. Packets fail when originating from tap/vnet and traversing lagg0.2020. (again, lagg0/lacp is up, includes cc0+cc1, bridge2020 includes lagg0.2020, tap0, and epair0a devices) host25# ping 10.20.20.254 success! vm-10-20-20-11# arp -da (attempt traverse lagg0.2020) vm-10-20-20-11# ping 10.20.20.254 ping: sendto: Host is down (try tap0 -> epair0) vm-10-20-20-11# ping 10.20.20.6 success! (try tests again with lagg0 member cc1 down) host25# cc1 down (tap0 -> lagg0.2020 -> 10.20.20.254) vm-10-20-20-11# ping 10.20.20.254 success! (again tap0 -> epair0, works as expected) vm-10-20-20-11# ping 10.20.20.6 success! (turn cc1 back up, wait about 10 seconds for both laggports to be distributing) host25# cc1 up vm-10-20-20-11# arp -da vm-10-20-20-11# ping 10.20.20
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #25 from Andriy Gapon --- (In reply to Kristof Provost from comment #24) I think I will need to look at the code. I thought that a bridge would see packets only from a bridged virtual/vlan interface (such as the proposed igb0.0), but it looks that the actual ethernet input processing has a different flow. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #24 from Kristof Provost --- (In reply to Andriy Gapon from comment #23) Ah, I see. I did indeed misunderstand. However, I don't think that'd fix the issue of VLAN on if_bridge interfaces. The problem is that the bridge checks if it needs to grab the packet before vlan_input() gets its turn. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #23 from Andriy Gapon --- (In reply to Kristof Provost from comment #22) Just to be sure that we talk about the same thing (and I feel like we are not), I am not suggesting any modification to what's going on the wire. Just a new virtual interface that captures only untagged packets. To be more clear: - igb0: receives all arriving packets, sends packets without inserting any VLAN tag - igb0.1: receives arriving packets with VLAN tag 1, adds VLAN tag 1 when sending - igb0.0: [proposed] receives only packets without any VLAN tag, sends packets without inserting any VLAN tag -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #22 from Kristof Provost --- (In reply to Andriy Gapon from comment #20) I'm not sure how that'd fix this issue, but I believe that's something we already support. It's possible to set Priority code point (PCP) on a regular interface, which should insert a vlan header with VID 0. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 Nikita Olenets changed: What|Removed |Added CC||z...@zeon.kiev.ua --- Comment #21 from Nikita Olenets --- In the mean time you can try "workaround" to create ng_bridge interface to your parent and then use than newly created interface as a member to your management bridge. Assuming you have em0 as your parent interface em0: flags=8963 metric 0 mtu 1500 options=4e527bb ether 58:9c:fc:10:f1:16 media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 /bin/sh /usr/share/examples/jails/jng bridge main em0 Will create ng0_main interface: ng0_main: flags=8943 metric 0 mtu 1500 options=28 ether 02:60:c8:08:84:9b hwaddr 58:9c:fc:10:ff:ff media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 Now you can add ng0_main interface instead your em0. Will work like a charm. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #20 from Andriy Gapon --- (In reply to Kristof Provost from comment #13) Perhaps we could create a special vlan "sub-interface" that sees only untagged traffic on input and does not add any tag on output (just like the parent interface). We could use some reserved VLAN ID to mark the special interface. E.g., the currently prohibited VLAN ID 0. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 --- Comment #10 from Joshua Kinard --- (In reply to Peter Much from comment #9) I'd give the jailed VNET router another try. It has been working fine on my end for a few months now (since ~13.1-BETA2). Just give the jail the external WAN interface and internal LAN interface and configure as normal. The host should listen on a third interface with a different address than the jail (my router has six gigabit interfaces to play with). Only bug I've still got to hunt down is when the jail starts at boot, it sometimes fails to pull an address from my ISP even though I have SYNCDHCP set. Easy fix to just restart netif em0 in the jail when it happens. I should debug it, though and figure out how to really fix it. I even have my wlan0 interface assigned to the jail and run hostapd out of it to run the wifi on a separate subnet. Needs a small tweak to hostapd's rc script to let it run into a jail (see #263359). -- You are receiving this mail because: You are the assignee for the bug.
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 Peter Much changed: What|Removed |Added CC||p...@citylink.dinoex.sub.org --- Comment #9 from Peter Much --- I am seeing these messages too, and no, Joshua, You're not the only one trying such things with dummynet. A few comments: * dummynet did never malfunction, it did simply reinsert the shaped packets into the host's ipfw instead of the jail's one - because there was only one dummynet on the system. The issue was more serios: every jail could manipulate the hosts pipes parameters, so it was necessary to run jails in securelevel=3. * This appears to be fixed in 13.1. Kudos to the author! * What has sadly not been fixed alongside is the same issue with ng_ipfw, (Should be simpler, because netgraph itself is already VIMAGE-able) I once tried the bufferbloat thing, ran into the very same issue, considered it too difficult to fix on my own, and then moved the router into a bhyve instead. There timing issues were more challenging, but the shaping apparently did work. Currently it doesn't anymore. Thanks for Your configuration data, that might help me when looking into that issue occasionally - maybe the bhyve has to leave again. Now I tried to do the ng_ipfw fix myself, but with horror then noticed the beforementioned error-messages, and another one that looks worse: Apr 22 15:34:06 edge kernel: Freed UMA keg (IPFW counters) was not empty (1 items). Lost 20 pages of memory. Jun 15 20:41:45 edge kernel: Freed UMA keg (IPFW counters) was not empty (2 items). Lost 40 pages of memory. Jun 26 17:06:23 edge kernel: [55226] Freed UMA keg (IPFW counters) was not empty (3 items). Lost 60 pages of memory. Jun 28 22:42:20 edge kernel: [248183] Freed UMA keg (IPFW counters) was not empty (3 items). Lost 40 pages of memory. But this one also obviousely exists since upgrade to 13.1 (RC2), and seems to happen at shutdown after some uptime; it cannot be related to my recent hacking. (I do massive and frequent ruleset updates during runtime.) -- You are receiving this mail because: You are the assignee for the bug.
Re: FreeBSD 12.3-p5: problems vnet on if_bridge
On Tue, 24 May 2022 09:52:46 + Ole wrote: Hello, > Hello, > > could you solve the problem? I think I ran into the same problem. > I opened a Ticket. I couldn't solve the problem. > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264198 > > I seems to be related to IPFW and effects vnet-Jails and also bhyve VMs. There is also a PR regarding vnet/if_bridge/routing issues, at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 but I can not guarantee this PR is in any way similaror adjacent to the problem of mine (and probably yours). For reasons not to be discussed here I'm bound to FreeBSD 12.3-RELENG (XigmaNAS base) and running several boxes of different hardware grades with the very same XigmaNAS version gives different results. The host in question in my case has Intel NICs, the box with a very similar setup and vnet jails also on the same NIC the LAN resides on, has a Realtek NIC (it is a very small home NAS). Another has a dedicated NIC into another network (diffrent IP of the NIC and the vnet jails bound to that NIC compared to the LAN/host itself - Intel i350 NICs all over the place, no problems. Another case, our build platform, is running CURRENT and hosts about almost 15 jails on a NIC, which is part of the same network as the main host's NIC - no problem. All of our FreeBSD boxes uses IPFW as their IP filtering facility. Have you checked the MAC addresses of you vnet jails / vnet hosts? I have had on 12-CURRENT a serious issue with the very same MAC address on all jail-belonging parts of the epair vnet interface. I haven't checked this on our 12.3 (XigmaNAS) installations, I remember this issue as I writet this email, it could be a hint to look after ... > > regards > Ole Kind regarads, O. Hartmann > > > Wed, 11 May 2022 20:47:55 +0200 - FreeBSD User : > > > On Tue, 10 May 2022 21:21:29 +0200 > > FreeBSD User wrote: > > > > > Hello, > > > > > > I ran into serious trouble setting up a FreeBSD 12.3-RELEASE-p5 > > > host having a second NIC and vnt jails attached to that second NIC > > > (basically, the host is a recent Xigmanas with Bastille jails, but > > > the issue also occurs on a vanilla FreeBSD 12.3). > > > > > > The host is compromised of two NICs, em0 (management only) and igb0 > > > (service/jails). Both, the server and the jails as well as the igb0 > > > interface are residing on the same network, but both NICs are > > > connected to two different ports on a switch, to which we do not > > > have access (part of the campus infrastructure). > > > > > > Both NICs are attached with a IPv4 of the same network, the host is > > > listening on both NICs for services, i.e. port 22 for ssh. No > > > problem to connect to both(!) addresses via ssh. igb0 is member of > > > an if_bridge. The box also hosts a bunch of vnet jails, each jail > > > does have an if_epair created via "jib" and these vnet epairs are > > > members of the bridge, to which ifb0 is also member. > > > > > > Problem: while any service bound to NIC igb0/IPv4 residing on igb0 > > > is accessible flawlessly, accessing an jail is almost impossible. > > > Pinging a jail does work after a while the ping initiating host has > > > been waiting, in ery rare situations someone can access the sshd of > > > the jail, but any access of that kind is highly erratic. From 5 > > > jails, at most two are responding to pings, the other don't and it > > > is non-deterministic which host will respond. > > > > > > Following some advices found on the web, the following sysctl > > > settings are provided to if_bridge: > > > > > > deviceif_bridge > > > net.link.bridge.ipfw: 0 > > > net.link.bridge.allow_llz_overlap: 0 > > > net.link.bridge.inherit_mac: 0 > > > net.link.bridge.log_stp: 0 > > > net.link.bridge.pfil_local_phys: 0 > > > net.link.bridge.pfil_member: 0 > > > net.link.bridge.ipfw_arp: 0 > > > net.link.bridge.pfil_bridge: 0 > > > net.link.bridge.pfil_onlyip: 0 > > > > > > We do not have access to the switch the box is connected to, so I > > > don't have access to any logs revealing a problem either to a > > > conceptual misunderstanding of networking of mine and so a > > > misconfiguration or a probelm with Layer 2 or the switches > > > themselfes. > > > > > > I'd like to ask whether someone has a similar setup up and running > > > and could report this > > > - or give a hint of the problem I possibly made (igb0 is attach
Re: FreeBSD 12.3-p5: problems vnet on if_bridge
Hello, could you solve the problem? I think I ran into the same problem. I opened a Ticket. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264198 I seems to be related to IPFW and effects vnet-Jails and also bhyve VMs. regards Ole Wed, 11 May 2022 20:47:55 +0200 - FreeBSD User : > On Tue, 10 May 2022 21:21:29 +0200 > FreeBSD User wrote: > > > Hello, > > > > I ran into serious trouble setting up a FreeBSD 12.3-RELEASE-p5 > > host having a second NIC and vnt jails attached to that second NIC > > (basically, the host is a recent Xigmanas with Bastille jails, but > > the issue also occurs on a vanilla FreeBSD 12.3). > > > > The host is compromised of two NICs, em0 (management only) and igb0 > > (service/jails). Both, the server and the jails as well as the igb0 > > interface are residing on the same network, but both NICs are > > connected to two different ports on a switch, to which we do not > > have access (part of the campus infrastructure). > > > > Both NICs are attached with a IPv4 of the same network, the host is > > listening on both NICs for services, i.e. port 22 for ssh. No > > problem to connect to both(!) addresses via ssh. igb0 is member of > > an if_bridge. The box also hosts a bunch of vnet jails, each jail > > does have an if_epair created via "jib" and these vnet epairs are > > members of the bridge, to which ifb0 is also member. > > > > Problem: while any service bound to NIC igb0/IPv4 residing on igb0 > > is accessible flawlessly, accessing an jail is almost impossible. > > Pinging a jail does work after a while the ping initiating host has > > been waiting, in ery rare situations someone can access the sshd of > > the jail, but any access of that kind is highly erratic. From 5 > > jails, at most two are responding to pings, the other don't and it > > is non-deterministic which host will respond. > > > > Following some advices found on the web, the following sysctl > > settings are provided to if_bridge: > > > > device if_bridge > > net.link.bridge.ipfw: 0 > > net.link.bridge.allow_llz_overlap: 0 > > net.link.bridge.inherit_mac: 0 > > net.link.bridge.log_stp: 0 > > net.link.bridge.pfil_local_phys: 0 > > net.link.bridge.pfil_member: 0 > > net.link.bridge.ipfw_arp: 0 > > net.link.bridge.pfil_bridge: 0 > > net.link.bridge.pfil_onlyip: 0 > > > > We do not have access to the switch the box is connected to, so I > > don't have access to any logs revealing a problem either to a > > conceptual misunderstanding of networking of mine and so a > > misconfiguration or a probelm with Layer 2 or the switches > > themselfes. > > > > I'd like to ask whether someone has a similar setup up and running > > and could report this > > - or give a hint of the problem I possibly made (igb0 is attached > > to an IPv4 AND is member of an if_brige on which IPv4 attached vnet > > jails are residing). > > > > We have also already setup another "similar" scenarion with the > > same FreeBSD 12.3-p5 version and also two NICs, but our > > "service/jail" NIC is part of a different IPv4 network and the NIC > > is attached to a different switch (to which we have full access). > > > > Thanks in advance, > > > > O. Hartmann > > > > On FreeBSD 12.3-p5, em0 seems to suffer from a bug regarding hardware > chesum support, I see a lot of : > [...] > Flags [.], cksum 0xe826 (incorrect -> 0x606b), seq > 101269476:10127, ack 5077, win 257, options [nop,nop,TS val > 2618589801 ecr 3610923914], length 524 > > Disabling TXCSUM via "ifconfig em0 -txcsum" renders incorrect -> > correct. > > em0 is: > > em0@pci0:0:25:0: class=0x02 card=0x20528086 > chip=0x153b8086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' > device = 'Ethernet Connection I217-V' > class = network > subclass = ethernet > bar [10] = type Memory, range 32, base 0xf7d0, size 131072, > enabled bar [14] = type Memory, range 32, base 0xf7d35000, size > 4096, enabled bar [18] = type I/O Port, range 32, base 0xf080, size > 32, enabled cap 01[c8] = powerspec 2 supports D0 D3 current D0 > cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message > cap 13[e0] = PCI Advanced Features: FLR TP > > > I remember faintly that there was an issue when I used to use FBSD 12 > > > > pgp32k8_HX17U.pgp Description: Digitale Signatur von OpenPGP
Re: FreeBSD 12.3-p5: problems vnet on if_bridge
On Tue, 10 May 2022 21:21:29 +0200 FreeBSD User wrote: > Hello, > > I ran into serious trouble setting up a FreeBSD 12.3-RELEASE-p5 host having a > second NIC > and vnt jails attached to that second NIC (basically, the host is a recent > Xigmanas with > Bastille jails, but the issue also occurs on a vanilla FreeBSD 12.3). > > The host is compromised of two NICs, em0 (management only) and igb0 > (service/jails). > Both, the server and the jails as well as the igb0 interface are residing on > the same > network, but both NICs are connected to two different ports on a switch, to > which we do > not have access (part of the campus infrastructure). > > Both NICs are attached with a IPv4 of the same network, the host is listening > on both > NICs for services, i.e. port 22 for ssh. No problem to connect to both(!) > addresses via > ssh. igb0 is member of an if_bridge. The box also hosts a bunch of vnet > jails, each jail > does have an if_epair created via "jib" and these vnet epairs are members of > the bridge, > to which ifb0 is also member. > > Problem: while any service bound to NIC igb0/IPv4 residing on igb0 is > accessible > flawlessly, accessing an jail is almost impossible. Pinging a jail does work > after a > while the ping initiating host has been waiting, in ery rare situations > someone can > access the sshd of the jail, but any access of that kind is highly erratic. > From 5 > jails, at most two are responding to pings, the other don't and it is > non-deterministic > which host will respond. > > Following some advices found on the web, the following sysctl settings are > provided to > if_bridge: > > deviceif_bridge > net.link.bridge.ipfw: 0 > net.link.bridge.allow_llz_overlap: 0 > net.link.bridge.inherit_mac: 0 > net.link.bridge.log_stp: 0 > net.link.bridge.pfil_local_phys: 0 > net.link.bridge.pfil_member: 0 > net.link.bridge.ipfw_arp: 0 > net.link.bridge.pfil_bridge: 0 > net.link.bridge.pfil_onlyip: 0 > > We do not have access to the switch the box is connected to, so I don't have > access to > any logs revealing a problem either to a conceptual misunderstanding of > networking of > mine and so a misconfiguration or a probelm with Layer 2 or the switches > themselfes. > > I'd like to ask whether someone has a similar setup up and running and could > report this > - or give a hint of the problem I possibly made (igb0 is attached to an IPv4 > AND is > member of an if_brige on which IPv4 attached vnet jails are residing). > > We have also already setup another "similar" scenarion with the same FreeBSD > 12.3-p5 > version and also two NICs, but our "service/jail" NIC is part of a different > IPv4 > network and the NIC is attached to a different switch (to which we have full > access). > > Thanks in advance, > > O. Hartmann > On FreeBSD 12.3-p5, em0 seems to suffer from a bug regarding hardware chesum support, I see a lot of : [...] Flags [.], cksum 0xe826 (incorrect -> 0x606b), seq 101269476:10127, ack 5077, win 257, options [nop,nop,TS val 2618589801 ecr 3610923914], length 524 Disabling TXCSUM via "ifconfig em0 -txcsum" renders incorrect -> correct. em0 is: em0@pci0:0:25:0:class=0x02 card=0x20528086 chip=0x153b8086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Connection I217-V' class = network subclass = ethernet bar [10] = type Memory, range 32, base 0xf7d0, size 131072, enabled bar [14] = type Memory, range 32, base 0xf7d35000, size 4096, enabled bar [18] = type I/O Port, range 32, base 0xf080, size 32, enabled cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 13[e0] = PCI Advanced Features: FLR TP I remember faintly that there was an issue when I used to use FBSD 12
FreeBSD 12.3-p5: problems vnet on if_bridge
Hello, I ran into serious trouble setting up a FreeBSD 12.3-RELEASE-p5 host having a second NIC and vnt jails attached to that second NIC (basically, the host is a recent Xigmanas with Bastille jails, but the issue also occurs on a vanilla FreeBSD 12.3). The host is compromised of two NICs, em0 (management only) and igb0 (service/jails). Both, the server and the jails as well as the igb0 interface are residing on the same network, but both NICs are connected to two different ports on a switch, to which we do not have access (part of the campus infrastructure). Both NICs are attached with a IPv4 of the same network, the host is listening on both NICs for services, i.e. port 22 for ssh. No problem to connect to both(!) addresses via ssh. igb0 is member of an if_bridge. The box also hosts a bunch of vnet jails, each jail does have an if_epair created via "jib" and these vnet epairs are members of the bridge, to which ifb0 is also member. Problem: while any service bound to NIC igb0/IPv4 residing on igb0 is accessible flawlessly, accessing an jail is almost impossible. Pinging a jail does work after a while the ping initiating host has been waiting, in ery rare situations someone can access the sshd of the jail, but any access of that kind is highly erratic. From 5 jails, at most two are responding to pings, the other don't and it is non-deterministic which host will respond. Following some advices found on the web, the following sysctl settings are provided to if_bridge: device if_bridge net.link.bridge.ipfw: 0 net.link.bridge.allow_llz_overlap: 0 net.link.bridge.inherit_mac: 0 net.link.bridge.log_stp: 0 net.link.bridge.pfil_local_phys: 0 net.link.bridge.pfil_member: 0 net.link.bridge.ipfw_arp: 0 net.link.bridge.pfil_bridge: 0 net.link.bridge.pfil_onlyip: 0 We do not have access to the switch the box is connected to, so I don't have access to any logs revealing a problem either to a conceptual misunderstanding of networking of mine and so a misconfiguration or a probelm with Layer 2 or the switches themselfes. I'd like to ask whether someone has a similar setup up and running and could report this - or give a hint of the problem I possibly made (igb0 is attached to an IPv4 AND is member of an if_brige on which IPv4 attached vnet jails are residing). We have also already setup another "similar" scenarion with the same FreeBSD 12.3-p5 version and also two NICs, but our "service/jail" NIC is part of a different IPv4 network and the NIC is attached to a different switch (to which we have full access). Thanks in advance, O. Hartmann
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 O. Hartmann changed: What|Removed |Added CC||ohartm...@walstatt.org --- Comment #19 from O. Hartmann --- Hello. We also have an similar issue on FreeBSD 12.3-RELEASE-p2 (XigmaNAS, stuck at -p2 for the moment) as described. The boxes in question do have two NICs, one is supposed for the management (em0) access and the other one is supposed to be bound to offered services. Additionally, the second NIC (igb0) is accessible via an IP AND serves as the physical NIC as member of a bridge for vnet jails, which do have epair interfaces (in Xigmanas created via the FreeBSD in-tree tool "jib"). Binding provided services as SAMBA and NFS to the second NIC (igb0) works as expected, also ping and ssh is no problem. Base host's IP (both NICs) and those of the jails are within the same network. When it comes to the vnet jails on the bridge, of which the igb0 NIC is member of, trouble begins. We use several jails on those boxes. Pinging those jails from outside the campus network does work sporadically with some IPs, it takes a long time until the jail starts repsonding. Same behaviour is within the LAN. We also already disabled pfil on the bridges as suggested: device if_bridge net.link.bridge.ipfw: 0 net.link.bridge.allow_llz_overlap: 0 net.link.bridge.inherit_mac: 0 net.link.bridge.log_stp: 0 net.link.bridge.pfil_local_phys: 0 net.link.bridge.pfil_member: 0 net.link.bridge.ipfw_arp: 0 net.link.bridge.pfil_bridge: 0 net.link.bridge.pfil_onlyip: 0 A curiosity is that if one can ping one or two out of the five jails on the host, in another attempt to do so one, at most two different hosts would answer the ping then and the former working pinged hosts do not anymore. It is like gambling. We also run another host with the very same XigmaNAS version, in that case, he second NIC is configured to be part of another network and attached to another switch - not problem there! In the problematic cases described above, we do not have direct access to the switches of the backend of the department, so I can't see whether I'm the culprit (misconfiguration, misunderstanding et cetera of network technology). Hope the problem could be solved anyway within FreeBSD 12.3. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #18 from Peter Eriksson --- > root@filur00:/etc # ifconfig bridge0 deletem lagg0 Easy solution... Remove from rc.conf: ifconfig_bridge0="addm vlan1601 up" and then tell iocage to not add lagg0 automatically to the jail's bridge: # iocage set vnet_default_interface=vlan1601 test -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 Peter Eriksson changed: What|Removed |Added CC||p...@lysator.liu.se --- Comment #17 from Peter Eriksson --- I'm seeing the same (or similar issue) on 12.3-RELEASE-p5 when trying to bridge a vlan interface into a jail: # egrep 'ifconfig|cloned' rc.conf ifconfig_ixl0="up" ifconfig_ixl2="up" cloned_interfaces="lagg0 vlan1601 bridge0" ifconfig_lagg0="laggproto lacp laggport ixl0 laggport ixl2 130.236.8.40 netmask 255.255.255.224 lacp_fast_timeout" ifconfig_lagg0_ipv6="inet6 2001:6b0:17:2400::8:40/64 lacp_fast_timeout" ifconfig_vlan1601="vlandev lagg0 vlan 1601 up" ifconfig_bridge0="addm vlan1601 up" # ifconfig bridge0 bridge0: flags=8843 metric 0 mtu 1500 ether 02:90:7b:7b:f5:00 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vnet0.1 flags=143 ifmaxaddr 0 port 14 priority 128 path cost 2000 member: lagg0 flags=143 ifmaxaddr 0 port 10 priority 128 path cost 1000 member: vlan1601 flags=143 ifmaxaddr 0 port 11 priority 128 path cost 200 groups: bridge nd6 options=9 root@filur00:/etc # iocage console test Last login: Thu Apr 21 14:27:18 on pts/0 FreeBSD 12.3-RELEASE-p5 GENERIC -- ... root@test:~ # ping 130.236.8.65 PING 130.236.8.65 (130.236.8.65): 56 data bytes ^C --- 130.236.8.65 ping statistics --- 2 packets transmitted, 0 packets received, 100.0% packet loss If I now manually remove the "lagg0" member from the bridge0 interface then things start to work fine. It would be nice if it didn't add it automatically :-) root@filur00:/etc # ifconfig bridge0 deletem lagg0 root@filur00:/etc # iocage console test Last login: Thu Apr 21 14:38:34 on pts/0 FreeBSD 12.3-RELEASE-p5 GENERIC root@test:~ # ping 130.236.8.65 PING 130.236.8.65 (130.236.8.65): 56 data bytes 64 bytes from 130.236.8.65: icmp_seq=0 ttl=255 time=0.249 ms ^C --- 130.236.8.65 ping statistics --- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.249/0.249/0.249/0.000 ms -- You are receiving this mail because: You are the assignee for the bug.
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 --- Comment #8 from Joshua Kinard --- (In reply to Kristof Provost from comment #7) FWIW, I've revisited this in 13.1-RC1 on my router and was able to successfully put my router configuration into a jail by allowing the jail to have direct access to the WAN and LAN ports (the jail host has six GbE ports total, so it runs management services on a third port). I was also able to get the separate wireless subnet into the same jail after putting wlan0 into the jail and tweaking hostapd's rc.d script to allow running in a jail. So far, the only issue noticed, which appears to be benign, are these messages in dmesg when the jail is shutdown or restarted: [nhop_ctl] inet.0 nhop_free: failed to unlink nh#3/inet/em0/resolve [nhop_ctl] inet.0 nhop_free: failed to unlink nh#2/inet/em0/ [nhop_ctl] inet.0 nhop_free: failed to unlink nh#5/inet/em1/resolve [nhop_ctl] inet6.0 nhop_free: failed to unlink nh#5/inet6/em1/resolve If those messages are not an issue, I'd say this bug can be closed. Thanks! -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #16 from Gabor ADORJANI --- Thanks, I'll try - I've been doing this on Linux for many years. Ironically, some years ago I read the opposite in some FreeBSD-related doc: assigning the addresses to the parent interface was the recommended way. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 cr...@rlwinm.de changed: What|Removed |Added CC||cr...@rlwinm.de --- Comment #15 from cr...@rlwinm.de --- You probably want to create the vlan interfaces on the physical interface and add them as member interfaces to the bridges and all IP interfaces belong on the bridge interface. Don't put IP addresses on bridge member interfaces. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 --- Comment #14 from Gabor ADORJANI --- (In reply to Kristof Provost from comment #13) Thanks Kristof, you are right, I didn't see the forest for the trees. It's not a bug, but a feature. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106
--- Comment #13 from Kristof Provost ---
(In reply to Bjoern A. Zeeb from comment #12)
Note that the issue described in #10 is a configuration problem more than a
bug.
In this configuration the bridge will grab all packets, including those with a
vlan tag and nothing will be passed to the vlan interfaces. That's expected.
After all, the system has been configured to bridge all packets arriving on em0
to the members of vm-sw1, and that includes those with ETHERTYPE_VLAN.
This patch should make it do what the user wants, but I'm not convinced that's
actually appropriate:
diff --git a/sys/net/if_bridge.c b/sys/net/if_bridge.c
index 12c807fe2009..98c79764bc69 100644
--- a/sys/net/if_bridge.c
+++ b/sys/net/if_bridge.c
@@ -2467,6 +2467,11 @@ bridge_input(struct ifnet *ifp, struct mbuf *m)
eh = mtod(m, struct ether_header *);
+ if (ntohs(eh->ether_type) == ETHERTYPE_VLAN ||
+ ntohs(eh->ether_type) == ETHERTYPE_QINQ) {
+ return (m);
+ }
+
bridge_span(sc, m);
if (m->m_flags & (M_BCAST|M_MCAST)) {
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 Bjoern A. Zeeb changed: What|Removed |Added CC||b...@freebsd.org, ||k...@freebsd.org --- Comment #12 from Bjoern A. Zeeb --- I beleive I ran into similar problems with main last year and I am still seeing occasional "blackouts"; I believe back then I could trigger traffic by sending packets from one specific part of jail / host / remote to another and that would hold until the entry expired but I have no more notes on this. I am also adding kp@ given bridge ... -- You are receiving this mail because: You are the assignee for the bug.
[Bug 240106] VNET issue with ARP and routing sockets in jails
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106 Gabor ADORJANI changed: What|Removed |Added CC||bugs.freebsd@gabor.ador ||jani.net --- Comment #11 from Gabor ADORJANI --- I believe I ran into the same issue today on 13.1-BETA3. Setup: I use a NUC for virtualisation host with a single NIC: em0. It has vPro (poor man's service processor), which shares the NIC with the OS and communicates on the native VLAN (VLAN1). Because of this I put the OS to a tagged one. I set up several tagged VLANs: 2, 4, 6, 8. The host OS uses em0.2 on VLAN2. I set up a bridge for each VLAN interface, as well as for the physical: em0 -> vm-sw1 em0.2 -> vm-sw2 em0.4 -> vm-sw4 em0.6 -> vm-sw6 em0.8 -> vm-sw8 Then I created a jail with Bastille, assigning it to VLAN2/vm-sw2 using VNET, with an IP from the subnet also used on the host. I could ping the host from the jail and vice versa, but could not reach the external world from the jail, nor could ping the jail from the router in the same subnet. After 'ifconfig vm-sw1 destroy' it suddenly started working and the jail now has full IP4/6 connectivity. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 262471] jail utility crashes when supplied -m and vnet parameter
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262471 Jamie Gritton changed: What|Removed |Added Assignee|j...@freebsd.org|ja...@freebsd.org --- Comment #1 from Jamie Gritton --- I see the problem. In the check for non-changeable parameters, it's trying to dereference the value of vnet. There's code to catch boolean parameters with implicit values, but it misses vnet which isn't a true boolean. Working on making a hideous-looking if statement even worse... -- You are receiving this mail because: You are the assignee for the bug.
[Bug 262471] jail utility crashes when supplied -m and vnet parameter
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262471 Mark Johnston changed: What|Removed |Added CC||ja...@freebsd.org, ||ma...@freebsd.org Assignee|b...@freebsd.org|j...@freebsd.org Status|New |Open -- You are receiving this mail because: You are the assignee for the bug.
Re: iocage, vnet jail does not go outside
Le samedi 24 juil. 2021 à 23:48:26 (+0200), infoomatic à écrit:
Hello,
> Hi,
>
> sorry to hear that.
That's life :-) and I learn a lot form my mistake and your help.
>
> I use the tools from the FreeBSD base system, they work great, and I
> encourage all newbies to use the tools from the base systems - and
> recommend reading the parts of the handbook and the man pages of jail
> and jail.conf
I use to. I alose read Michael W. Lucas Mastery Jails bookṡ
>
> Here are the relevant parts of my config:
>
> rc.conf:
>
> cloned_interfaces="bridge0"
>
> ifconfig_bridge0="inet 192.168.1.1 netmask 255.255.255.0 up"
>
> pf.conf:
>
> nat pass on em0 proto tcp from {192.168.1.201} to any -> pu.bl.ic.ip
>
> and the jail.conf:
>
> example {
> host.hostname = example;
> vnet;
> vnet.interface = "epair201b";
> path ="/jails/$name";
> exec.prestart += "ifconfig epair201 create";
> exec.prestart += "ifconfig epair201a up";
> exec.prestart += "ifconfig bridge0 addm epair201a";
> exec.prestop += "ifconfig epair201b -vnet $name";
> exec.poststop += "ifconfig epair201a destroy";
>
> }
>
> and the /jails/example/etc/rc.conf:
>
> ifconfig_epair201b="inet 192.168.1.201 netmask 255.255.255.0"
> defaultrouter="192.168.1.1"
> hope this helps,
Of course it helps. And as I understood about إaving or not em0 into the
bridge. Without your are sure that your jail CANNOT communicate with the
external world (useful for a database jail for example), and with your jail
CAN communicate with the external world (useful for a weⅺsie ou mail jail).
In my case, I would like to have a VNET jail that can dialog with the World.
So, from your sample I add em0 the bridge and give it an IPv4 address, but it
did not work.
In any case, thanks for your help and the time you spent on my stupid problem.
Btw I read all the other answer and try to make a mixupo on my brain with all
this informations. Thanks to all.
> >
> >> iocage autoatically creates a bridge with your physical interface and
> >> the vnet interface. Imho this is wrong behaviour so I quit using iocage,
> >> however, there is a workaround, for more info see [1]
> >
> > I read carfully the issue your pointed and it appears that the
> > vnet_default_interface parameter set to auto, em0 is added to the bridge,
> > set
> > to none, em0 is not added to the bridge.
> >
> > So I stopped my jail, destroy bridge0 interface, set vnet_default_interface
> > to
> > none and restart the jail.
> >
> > As exepected em0 is not in the bridge any more:
> >
> > bridge0: flags=8843 metric 0 mtu
> > 1500
> > description: jails-bridge
> > ether 58:9c:fc:10:ed:66
> > inet 10.0.10.1 netmask 0xff00 broadcast 10.0.10.255
> > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> > member: vnet0.657 flags=143
> > ifmaxaddr 0 port 6 priority 128 path cost 2000
> > groups: bridge
> > nd6 options=9
> >
> > Since from the jail I cannot ping anything, from outside I cannot connect to
> > the jail and from the jail I cannot connect to outside host.
> >
> > In fact, see quickly, the situation is worst.
> >
> > I did not look at the routing tables yet (too many other things to do).
> >
> > As I understood your did not use iocage any more. Did you use the "raw"
> > method (ie /etc/jail.conf)? If yes, I am really interested of "picture" of
> > your configurætion.
> >
> > To be honest, I used to try the "raw" method whithout success before tring
> > iocage.
> >
> > Thanks for your time and advices.
>
--
Jacques Foucry
Re: iocage, vnet jail does not go outside
Hi,
sorry to hear that.
I use the tools from the FreeBSD base system, they work great, and I
encourage all newbies to use the tools from the base systems - and
recommend reading the parts of the handbook and the man pages of jail
and jail.conf
Here are the relevant parts of my config:
rc.conf:
cloned_interfaces="bridge0"
ifconfig_bridge0="inet 192.168.1.1 netmask 255.255.255.0 up"
pf.conf:
nat pass on em0 proto tcp from {192.168.1.201} to any -> pu.bl.ic.ip
and the jail.conf:
example {
host.hostname = example;
vnet;
vnet.interface = "epair201b";
path ="/jails/$name";
exec.prestart += "ifconfig epair201 create";
exec.prestart += "ifconfig epair201a up";
exec.prestart += "ifconfig bridge0 addm epair201a";
exec.prestop += "ifconfig epair201b -vnet $name";
exec.poststop += "ifconfig epair201a destroy";
}
and the /jails/example/etc/rc.conf:
ifconfig_epair201b="inet 192.168.1.201 netmask 255.255.255.0"
defaultrouter="192.168.1.1"
hope this helps,
Robert
On 24.07.21 13:38, Jacques Foucry wrote:
> Le vendredi 23 juil. 2021 à 23:06:41 (+0200), infoomatic à écrit:
>
> Hello Robert,
>
> Thanks for your answer.
>
>> iocage autoatically creates a bridge with your physical interface and
>> the vnet interface. Imho this is wrong behaviour so I quit using iocage,
>> however, there is a workaround, for more info see [1]
>
> I read carfully the issue your pointed and it appears that the
> vnet_default_interface parameter set to auto, em0 is added to the bridge, set
> to none, em0 is not added to the bridge.
>
> So I stopped my jail, destroy bridge0 interface, set vnet_default_interface to
> none and restart the jail.
>
> As exepected em0 is not in the bridge any more:
>
> bridge0: flags=8843 metric 0 mtu 1500
> description: jails-bridge
> ether 58:9c:fc:10:ed:66
> inet 10.0.10.1 netmask 0xff00 broadcast 10.0.10.255
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> member: vnet0.657 flags=143
> ifmaxaddr 0 port 6 priority 128 path cost 2000
> groups: bridge
> nd6 options=9
>
> Since from the jail I cannot ping anything, from outside I cannot connect to
> the jail and from the jail I cannot connect to outside host.
>
> In fact, see quickly, the situation is worst.
>
> I did not look at the routing tables yet (too many other things to do).
>
> As I understood your did not use iocage any more. Did you use the "raw"
> method (ie /etc/jail.conf)? If yes, I am really interested of "picture" of
> your configurætion.
>
> To be honest, I used to try the "raw" method whithout success before tring
> iocage.
>
> Thanks for your time and advices.
Re: iocage, vnet jail does not go outside
I use qjail for my vnet jails because iocage just did not work for me.
Re: iocage, vnet jail does not go outside
Le vendredi 23 juil. 2021 à 23:06:41 (+0200), infoomatic à écrit: Hello Robert, Thanks for your answer. > iocage autoatically creates a bridge with your physical interface and > the vnet interface. Imho this is wrong behaviour so I quit using iocage, > however, there is a workaround, for more info see [1] I read carfully the issue your pointed and it appears that the vnet_default_interface parameter set to auto, em0 is added to the bridge, set to none, em0 is not added to the bridge. So I stopped my jail, destroy bridge0 interface, set vnet_default_interface to none and restart the jail. As exepected em0 is not in the bridge any more: bridge0: flags=8843 metric 0 mtu 1500 description: jails-bridge ether 58:9c:fc:10:ed:66 inet 10.0.10.1 netmask 0xff00 broadcast 10.0.10.255 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vnet0.657 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000 groups: bridge nd6 options=9 Since from the jail I cannot ping anything, from outside I cannot connect to the jail and from the jail I cannot connect to outside host. In fact, see quickly, the situation is worst. I did not look at the routing tables yet (too many other things to do). As I understood your did not use iocage any more. Did you use the "raw" method (ie /etc/jail.conf)? If yes, I am really interested of "picture" of your configurætion. To be honest, I used to try the "raw" method whithout success before tring iocage. Thanks for your time and advices. -- Jacques Foucry
Re: iocage, vnet jail does not go outside
iocage autoatically creates a bridge with your physical interface and
the vnet interface. Imho this is wrong behaviour so I quit using iocage,
however, there is a workaround, for more info see [1]
Regards,
Robert
[1] https://github.com/iocage/iocage/issues/521
On 23.07.21 18:36, Jacques Foucry wrote:
> Hello friends,
>
> I'm turing crazy.
>
> I made a new jail ,on my hosted system using iocage.
>
> Here is the config.json file:
>
> more config.json
> {
> "allow_mount": 1,
> "allow_mount_devfs": 1,
> "allow_mount_nullfs": 1,
> "allow_mount_procfs": 1,
> "allow_mount_tmpfs": 1,
> "allow_mount_zfs": 1,
> "allow_raw_sockets": 1,
> "allow_socket_af": 1,
> "allow_sysvipc": 1,
> "bpf": 1,
> "cloned_release": "13.0-RELEASE",
> "defaultrouter": "10.0.10.1",
> "defaultrouter6": "auto",
> "dhcp": 0,
> "host_hostname": "examplejail",
> "host_hostuuid": "examplejail",
> "ip4_addr": "vnet0|10.0.10.23/24",
> "ip6_addr": "vnet0|2a01:4f9:4a:1fd8::23",
> "jail_zfs_dataset": "iocage/jails/examplejail/data",
> "last_started": "2021-07-23 15:11:28",
> "nat": 0,
> "release": "13.0-RELEASE-p3",
> "vnet": 1,
> "vnet0_mac": "b42e999c5bca b42e999c5bcb",
> "vnet_default_interface": "auto"
> }
>
> The jail's ifconfig:
>
> ifconfig
> lo0: flags=8049 metric 0 mtu 16384
> options=680003
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
> inet 127.0.0.1 netmask 0xff00
> groups: lo
> nd6 options=21
> pflog0: flags=0<> metric 0 mtu 33160
> groups: pflog
> epair0b: flags=8843 metric 0 mtu 1500
> options=8
> ether b4:2e:99:9c:5b:cb
> hwaddr 02:ae:46:07:62:0b
> inet 10.0.10.23 netmask 0xff00 broadcast 10.0.10.255
> inet6 2a01:4f9:4a:1fd8::23 prefixlen 64
> inet6 fe80::b62e:99ff:fe9c:5bcb%epair0b prefixlen 64 scopeid 0x3
> groups: epair
> media: Ethernet 10Gbase-T (10Gbase-T )
> status: active
> nd6 options=21
>
> The jail's netstat:
>
> netstat -rn
> Routing tables
>
> Internet:
> DestinationGatewayFlags Netif Expire
> default10.0.10.1 UGS epair0b
> 10.0.10.0/24 link#3 U epair0b
> 10.0.10.23 link#3 UHS lo0
> 127.0.0.1 link#1 UH lo0
>
> Internet6:
> Destination Gateway Flags
> Netif Expire
> ::/96 ::1 UGRS
> lo0
> default fe80::1%epair0b UGS
> epair0b
> ::1 link#1UHS
> lo0
> :::0.0.0.0/96 ::1 UGRS
> lo0
> 2a01:4f9:4a:1fd8::/64 link#3U
> epair0b
> 2a01:4f9:4a:1fd8::23 link#3UHS
> lo0
> fe80::/10 ::1 UGRS
> lo0
> fe80::%lo0/64 link#1U
> lo0
> fe80::1%lo0 link#1UHS
> lo0
> fe80::%epair0b/64 link#3U
> epair0b
> fe80::b62e:99ff:fe9c:5bcb%epair0b link#3UHS
> lo0
> ff02::/16
>
> On the host, the ifconfig (note thereis a lot of old fashion jails):
>
> ifconfig
> em0: flags=8963 metric 0 mtu
> 1500
>
> options=4810099
> ether b4:2e:99:6a:80:9d
> inet6 2a01:4f9:4a:1fd8::2 prefixlen 64
> inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1
> inet6 2a01:4f9:4a:1fd8::5 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::11 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::12 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::15 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::16 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::18 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::19 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::21 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::22 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::25 prefixlen 64
>
Re: iocage, vnet jail does not go outside
On Fri, 23 Jul 2021 20:04:41 +0200
Jacques Foucry wrote:
> Le vendredi 23 juil. 2021 à 19:51:42 (+0200), Michael Gmelin à écrit:
>
> Hello Michael,
>
> > You need to enable some sort of NAT at your end, e.g. using pf.
> > Traffic is leaving your host on a private IP.
>
>
> I forgot to post the part of my pf.conf your right. I enabled a NAT
> (may be in a wroing way):
>
> ext_if = em0
> int_if = "{lo0 lo1}"
> bridge_if = bridge0
>
> icmp_types="{ echoreq, unreach }"
>
> # ok loopback
> set skip on lo0
> set skip on lo1
> #set skip on bridge0
>
> # define jails
> jails_net = "{192.168.12.0/24 10.0.10.0/24 2a01:4f9:4a:1fd8::/64}"
>
> …
>
> # nat
> nat on $ext_if from $jails_net to any -> $ext_if
>
> …
>
> # ExampleJail
> rdr on $ext_if inet proto tcp from any to $ext_if port
> $examplejail_ports -> $examplejail_v4 rdr on $ext_if inet6 proto tcp
> from any to $ext_if port $examplejail_ports -> $examplejail_v6
>
> …
>
> pass in log quick on $ext_if proto tcp from any to $examplejail_v4
> port $examplejail_ports flags S/SA keep state pass in log quick on
> $ext_if proto tcp from any to $examplejail_v6 port $examplejail_ports
>
> …
>
> # Allow icmp
> pass in inet proto icmp all icmp-type $icmp_types
> #IPv6 - pass in/out all IPv6 ICMP traffic
> pass in quick proto icmp6 Allow
>
>
>
> Is there something wrong of missing? I was guessing that the NAT is
> correct because I can connect from outside (IPv4 and IPv6) to this
> jail.
>
>
> Thanks again for your time.
There's one thing on your bridge that looks wrong:
> bridge0: flags=8843 metric 0
> mtu 1500 description: jails-bridge
> ether 58:9c:fc:10:ed:66
> inet 10.0.10.1 netmask 0xff00 broadcast 10.0.10.255
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> member: vnet0.655 flags=143
> ifmaxaddr 0 port 6 priority 128 path cost 2000
> member: em0 flags=143
> ifmaxaddr 0 port 1 priority 128 path cost 2
> groups: bridge
> nd6 options=9
>
em0 shouldn't be part of the bridge, as you don't want to bridge with
your uplink, but NAT to it.
So try
ifconfig bridge0 deletem em0
Once done, you might need to enable ip forwarding (if it isn't enabled
already).
service gateway enable
sysctl net.inet.ip.forwarding=1
-m
--
Michael Gmelin
Re: iocage, vnet jail does not go outside
Le vendredi 23 juil. 2021 à 19:51:42 (+0200), Michael Gmelin à écrit:
Hello Michael,
> You need to enable some sort of NAT at your end, e.g. using pf. Traffic
> is leaving your host on a private IP.
I forgot to post the part of my pf.conf your right. I enabled a NAT (may be in
a wroing way):
ext_if = em0
int_if = "{lo0 lo1}"
bridge_if = bridge0
icmp_types="{ echoreq, unreach }"
# ok loopback
set skip on lo0
set skip on lo1
#set skip on bridge0
# define jails
jails_net = "{192.168.12.0/24 10.0.10.0/24 2a01:4f9:4a:1fd8::/64}"
…
# nat
nat on $ext_if from $jails_net to any -> $ext_if
…
# ExampleJail
rdr on $ext_if inet proto tcp from any to $ext_if port $examplejail_ports ->
$examplejail_v4
rdr on $ext_if inet6 proto tcp from any to $ext_if port $examplejail_ports ->
$examplejail_v6
…
pass in log quick on $ext_if proto tcp from any to $examplejail_v4 port
$examplejail_ports flags S/SA keep state
pass in log quick on $ext_if proto tcp from any to $examplejail_v6 port
$examplejail_ports
…
# Allow icmp
pass in inet proto icmp all icmp-type $icmp_types
#IPv6 - pass in/out all IPv6 ICMP traffic
pass in quick proto icmp6 Allow
Is there something wrong of missing? I was guessing that the NAT is correct
because I can connect from outside (IPv4 and IPv6) to this jail.
Thanks again for your time.
--
Jacques Foucry
Re: iocage, vnet jail does not go outside
On Fri, 23 Jul 2021 18:36:25 +0200
Jacques Foucry wrote:
> Hello friends,
>
> I'm turing crazy.
>
> I made a new jail ,on my hosted system using iocage.
>
> Here is the config.json file:
>
> more config.json
> {
> "allow_mount": 1,
> "allow_mount_devfs": 1,
> "allow_mount_nullfs": 1,
> "allow_mount_procfs": 1,
> "allow_mount_tmpfs": 1,
> "allow_mount_zfs": 1,
> "allow_raw_sockets": 1,
> "allow_socket_af": 1,
> "allow_sysvipc": 1,
> "bpf": 1,
> "cloned_release": "13.0-RELEASE",
> "defaultrouter": "10.0.10.1",
> "defaultrouter6": "auto",
> "dhcp": 0,
> "host_hostname": "examplejail",
> "host_hostuuid": "examplejail",
> "ip4_addr": "vnet0|10.0.10.23/24",
> "ip6_addr": "vnet0|2a01:4f9:4a:1fd8::23",
> "jail_zfs_dataset": "iocage/jails/examplejail/data",
> "last_started": "2021-07-23 15:11:28",
> "nat": 0,
> "release": "13.0-RELEASE-p3",
> "vnet": 1,
> "vnet0_mac": "b42e999c5bca b42e999c5bcb",
> "vnet_default_interface": "auto"
> }
>
> The jail's ifconfig:
>
> ifconfig
> lo0: flags=8049 metric 0 mtu 16384
> options=680003
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
> inet 127.0.0.1 netmask 0xff00
> groups: lo
> nd6 options=21
> pflog0: flags=0<> metric 0 mtu 33160
> groups: pflog
> epair0b: flags=8843 metric 0
> mtu 1500 options=8
> ether b4:2e:99:9c:5b:cb
> hwaddr 02:ae:46:07:62:0b
> inet 10.0.10.23 netmask 0xff00 broadcast 10.0.10.255
> inet6 2a01:4f9:4a:1fd8::23 prefixlen 64
> inet6 fe80::b62e:99ff:fe9c:5bcb%epair0b prefixlen 64 scopeid
> 0x3 groups: epair
> media: Ethernet 10Gbase-T (10Gbase-T )
> status: active
> nd6 options=21
>
> The jail's netstat:
>
> netstat -rn
> Routing tables
>
> Internet:
> DestinationGatewayFlags Netif Expire
> default10.0.10.1 UGS epair0b
> 10.0.10.0/24 link#3 U epair0b
> 10.0.10.23 link#3 UHS lo0
> 127.0.0.1 link#1 UH lo0
>
> Internet6:
> Destination Gateway Flags
> Netif Expire ::/96 ::1
>UGRSlo0 default
> fe80::1%epair0b UGS epair0b ::1
> link#1UHS lo0
> :::0.0.0.0/96 ::1 UGRS
> lo0 2a01:4f9:4a:1fd8::/64 link#3
> U epair0b 2a01:4f9:4a:1fd8::23 link#3
> UHS lo0 fe80::/10 ::1
> UGRSlo0 fe80::%lo0/64
> link#1U lo0 fe80::1%lo0
> link#1UHS lo0
> fe80::%epair0b/64 link#3U
> epair0b fe80::b62e:99ff:fe9c:5bcb%epair0b link#3
> UHS lo0 ff02::/16
>
> On the host, the ifconfig (note thereis a lot of old fashion jails):
>
> ifconfig
> em0: flags=8963
> metric 0 mtu 1500
> options=4810099
> ether b4:2e:99:6a:80:9d inet6 2a01:4f9:4a:1fd8::2 prefixlen 64
> inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1
> inet6 2a01:4f9:4a:1fd8::5 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::11 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::12 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::15 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::16 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::18 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::19 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::21 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::22 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::25 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::14 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::29 prefixlen 64
> inet6 2a01:4f9:4a:1fd8::17 prefixlen 64
> inet 95.217.83.231 netmask 0xffc0 broadcast 95.217.83.255
> media: Ethernet autoselect (1000baseT )
> status: active
> nd6 options=21
> lo0: flags=8049 metric 0 mtu 16384
> options=680003
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid
iocage, vnet jail does not go outside
Hello friends,
I'm turing crazy.
I made a new jail ,on my hosted system using iocage.
Here is the config.json file:
more config.json
{
"allow_mount": 1,
"allow_mount_devfs": 1,
"allow_mount_nullfs": 1,
"allow_mount_procfs": 1,
"allow_mount_tmpfs": 1,
"allow_mount_zfs": 1,
"allow_raw_sockets": 1,
"allow_socket_af": 1,
"allow_sysvipc": 1,
"bpf": 1,
"cloned_release": "13.0-RELEASE",
"defaultrouter": "10.0.10.1",
"defaultrouter6": "auto",
"dhcp": 0,
"host_hostname": "examplejail",
"host_hostuuid": "examplejail",
"ip4_addr": "vnet0|10.0.10.23/24",
"ip6_addr": "vnet0|2a01:4f9:4a:1fd8::23",
"jail_zfs_dataset": "iocage/jails/examplejail/data",
"last_started": "2021-07-23 15:11:28",
"nat": 0,
"release": "13.0-RELEASE-p3",
"vnet": 1,
"vnet0_mac": "b42e999c5bca b42e999c5bcb",
"vnet_default_interface": "auto"
}
The jail's ifconfig:
ifconfig
lo0: flags=8049 metric 0 mtu 16384
options=680003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff00
groups: lo
nd6 options=21
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
epair0b: flags=8843 metric 0 mtu 1500
options=8
ether b4:2e:99:9c:5b:cb
hwaddr 02:ae:46:07:62:0b
inet 10.0.10.23 netmask 0xff00 broadcast 10.0.10.255
inet6 2a01:4f9:4a:1fd8::23 prefixlen 64
inet6 fe80::b62e:99ff:fe9c:5bcb%epair0b prefixlen 64 scopeid 0x3
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T )
status: active
nd6 options=21
The jail's netstat:
netstat -rn
Routing tables
Internet:
DestinationGatewayFlags Netif Expire
default10.0.10.1 UGS epair0b
10.0.10.0/24 link#3 U epair0b
10.0.10.23 link#3 UHS lo0
127.0.0.1 link#1 UH lo0
Internet6:
Destination Gateway Flags Netif
Expire
::/96 ::1 UGRSlo0
default fe80::1%epair0b UGS epair0b
::1 link#1UHS lo0
:::0.0.0.0/96 ::1 UGRSlo0
2a01:4f9:4a:1fd8::/64 link#3U epair0b
2a01:4f9:4a:1fd8::23 link#3UHS lo0
fe80::/10 ::1 UGRSlo0
fe80::%lo0/64 link#1U lo0
fe80::1%lo0 link#1UHS lo0
fe80::%epair0b/64 link#3U epair0b
fe80::b62e:99ff:fe9c:5bcb%epair0b link#3UHS lo0
ff02::/16
On the host, the ifconfig (note thereis a lot of old fashion jails):
ifconfig
em0: flags=8963 metric 0 mtu
1500
options=4810099
ether b4:2e:99:6a:80:9d
inet6 2a01:4f9:4a:1fd8::2 prefixlen 64
inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1
inet6 2a01:4f9:4a:1fd8::5 prefixlen 64
inet6 2a01:4f9:4a:1fd8::11 prefixlen 64
inet6 2a01:4f9:4a:1fd8::12 prefixlen 64
inet6 2a01:4f9:4a:1fd8::15 prefixlen 64
inet6 2a01:4f9:4a:1fd8::16 prefixlen 64
inet6 2a01:4f9:4a:1fd8::18 prefixlen 64
inet6 2a01:4f9:4a:1fd8::19 prefixlen 64
inet6 2a01:4f9:4a:1fd8::21 prefixlen 64
inet6 2a01:4f9:4a:1fd8::22 prefixlen 64
inet6 2a01:4f9:4a:1fd8::25 prefixlen 64
inet6 2a01:4f9:4a:1fd8::14 prefixlen 64
inet6 2a01:4f9:4a:1fd8::29 prefixlen 64
inet6 2a01:4f9:4a:1fd8::17 prefixlen 64
inet 95.217.83.231 netmask 0xffc0 broadcast 95.217.83.255
media: Ethernet autoselect (1000baseT )
status: active
nd6 options=21
lo0: flags=8049 metric 0 mtu 16384
options=680003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff00
inet 127.0.12.1 netmask 0xff00
inet 127.0.1.5 netmask 0x
inet 127.0.1.11 netmask 0x
inet 127.0.1.12 netmask 0x
inet 127.0.1.15 netmask 0x
inet 127.0.1.16 netmask 0x
inet 127.0.1.18 netmask 0x
inet 127.0.1.19 netmask 0x
inet 127.0.1.21
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326 Xin LI changed: What|Removed |Added Assignee|j...@freebsd.org|k...@freebsd.org Version|12.1-RELEASE|12.2-RELEASE Keywords||vimage CC||delp...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug.
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 --- Comment #7 from Kristof Provost --- (In reply to Mark Johnston from comment #6) I do hope to MFC this work, but it's currently incomplete. It's part of an effort to make pf support dummynet, and as part of that I'm also writing a few test cases (which requires the dummynet support). That's revealed at least one bug (when an interface goes away dummynet does not purge queued packets for that interface. When it goes to send them bad things happen.). I'm also having issues with an IPv6 queue test, but I'm still debugging. tl;dr: I hope to eventually MFC this, but right now vnet support in dummynet should be considered experimental. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 Mark Johnston changed: What|Removed |Added Status|New |Open CC||ma...@freebsd.org --- Comment #6 from Mark Johnston --- It looks like the patches to make dummynet work in vnets have been committed: https://cgit.freebsd.org/src/commit/?id=fe3bcfbda30e763a3ec56083b3a19cebbeaf8952 Based on lack of a MFC tag, I guess this will be 14.0 only? -- You are receiving this mail because: You are the assignee for the bug.
Network in VNET jail does not work on my FreeBSD current bhyve vm
Hello everybody,
since a few weeks, my jails on a bhyve-vm, running current are not reachable
via network, when configured with VNET. They can't even access the gateway. I
don't remember when this problem started, but it's a few weeks.
The same jail.conf works on a 13.0 host, on a current system the network does
not work. A configuration without VNET on the same jail works. Are there any
changes, that i missed? Here is the configuration, maybe someone spots an
error, or has an idea what's going on:
--
Martin
uname on bhyve vm:
--
root@fbsd14:~ # uname -a
FreeBSD fbsd14.fritz.box 14.0-CURRENT FreeBSD 14.0-CURRENT
main-n247020-e0fa04e257c GENERIC-NODEBUG amd64
root@fbsd14:~ # freebsd-version -kru
14.0-CURRENT
14.0-CURRENT
14.0-CURRENT
jail.conf on bhyve vm:
--
# set default configuration values
mount.devfs = true;
exec.clean = true;
allow.chflags = 1;
allow.raw_sockets = 1;
devfs_ruleset = 5;
exec.system_user = "root";
exec.jail_user= "root";
exec.timeout = 30;
stop.timeout = 30;
#
# Jails #
#
j1 {
# Hostname
host.hostname = "j1.fritz.box";
host.domainname = "fritz.box";
host.hostuuid = "68c2ad9b-b582-11eb-a925-589cfc0ac350";
osrelease = "14.0-CURRENT";
osreldate = "1400013";
# Network
vnet = 1;
vnet.interface = "epair2b";
exec.prestart += "ifconfig epair2 create up";
exec.prestart += "ifconfig epair2a description 'IFID=2 JAIL=j1'";
exec.prestart += "ifconfig bridge0 addm epair2a";
command = "ifconfig epair2b inet 192.168.1.101/22";
command += "route -n add -inet default 192.168.0.1";
exec.prestop = "ifconfig epair2b -vnet j1";
exec.poststop += "ifconfig bridge0 deletem epair2a";
exec.poststop += "ifconfig epair2a destroy";
sysvmsg = new;
sysvsem = new;
sysvshm = new;
path = "/jails/j1";
allow.mount.zfs = 1;
## Script execution
exec.timeout = 90;
# Pre-/Post-Scripts
exec.prestart += "logger trying to start jail j1 ...";
exec.poststart += "logger jail j1 has started";
exec.prestop += "logger shutting down jail j1";
exec.poststop += "logger jail j1 has shut down";
# Start Script
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
}
---
/etc/rc.conf on bhyve vm:
-
syslogd_flags="-ss"
sendmail_enable="NONE"
hostname="fbsd14.fritz.box"
ifconfig_vtnet0="inet 192.168.1.100 netmask 255.255.252.0"
defaultrouter="192.168.0.1"
local_unbound_enable="YES"
sshd_enable="YES"
ntpd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
jail_enable="YES"
keymap="de"
cloned_interfaces="bridge0"
ifconfig_bridge0="addm vtnet0 up"
# NFS
rpc_lockd_enable="YES"
rpc_statd_enable="YES"
nfs_client_enable="YES"
nfsuserd_enable="YES"
-
ifconfig on bhyve vm:
-
root@fbsd14:~ # ifconfig -f inet:cidr
vtnet0: flags=8863 metric 0 mtu 1500
options=80028
ether 58:9c:fc:0a:c3:50
inet 192.168.1.100/22 broadcast 192.168.3.255
media: Ethernet autoselect (10Gbase-T )
status: active
nd6 options=29
lo0: flags=8049 metric 0 mtu 16384
options=680003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1/8
groups: lo
nd6 options=21
bridge0: flags=8843 metric 0 mtu 1500
ether 58:9c:fc:10:ff:bf
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair2a flags=143
ifmaxaddr 0 port 4 priority 128 path cost 2000
groups: bridge
nd6 options=9
epair2a: flags=8943 metric 0
mtu 1500
description: IFID=2 JAIL=j1
options=8
ether 02:b4:ee:59:b3:0a
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T )
status: active
nd6 options=29
---
/etc/rc.conf in jail:
-
syslogd_flags="-ss"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
sshd_enable="YES"
---
ifconfig in jail:
-
root@j1:~ # ifconfig -f inet:cidr
lo0: flags=8049 metr
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 --- Comment #5 from Joshua Kinard --- (In reply to Kristof Provost from comment #4) Noted, thanks. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 --- Comment #4 from Kristof Provost --- (In reply to Joshua Kinard from comment #3) It'll be done when it's done. There is no plan. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 --- Comment #3 from Joshua Kinard --- (In reply to Kristof Provost from comment #2) > There's ongoing work here: https://reviews.freebsd.org/D29274 , but right > now dummynet cannot be used inside vnet jails. Ah ha, thanks for the info and the link to current effort. Is this work in any way slated for the 13.1 (or 13.2) window, or more for 14.0? -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 Kristof Provost changed: What|Removed |Added CC||k...@freebsd.org --- Comment #2 from Kristof Provost --- There's ongoing work here: https://reviews.freebsd.org/D29274 , but right now dummynet cannot be used inside vnet jails. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 255830] dummynet(4) queues/pipes do not work inside of a VNET jail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255830 Mark Linimon changed: What|Removed |Added Assignee|b...@freebsd.org|j...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326 --- Comment #23 from Zhenlei Huang --- Created attachment 222062 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=222062&action=edit Kernel panic core text dump -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326
--- Comment #22 from Zhenlei Huang ---
(In reply to Kyle Evans from comment #20)
Steps to reproduce the kernel panic:
Host environment:
FreeBSD 12.2 Guest fresh install with kernel debug symbols, VMware Fusion
12.1.0, hardware configured with 4 Processor cores and 1G memory, system
updated to 12.2-RELEASE-p3.
Host and jail's /etc/rc.conf:
- rc.conf --
# The jails share this rc.conf, let's disable the syslog service
syslogd_enable="NO"
#syslogd_flags="-ss"
sendmail_enable="NONE"
hostname=""
ifconfig_em0="DHCP"
dumpdev="AUTO"
zfs_enable="YES"
Host's /etc/jail.conf:
jail.conf -
# template for all test jails
# it is convenient to share host's filesystem
path = "/";
exec.clean;
vnet = new;
vnet.interface = "epair${ifnum}b";
exec.prepare = "/sbin/ifconfig epair${ifnum} create";
exec.prepare += "/sbin/ifconfig epair${ifnum}a inet 192.168.${ifnum}.1/24 up";
exec.start = "/bin/sh /etc/rc";
# I've no ideas why opening and binding a socket would trigger the kernel panic
more likely :(
exec.start += "/usr/sbin/daemon /usr/bin/nc -l 0.0.0.0 ";
exec.start += "/sbin/ifconfig epair${ifnum}b inet 192.168.${ifnum}.2/24";
exec.start += "/sbin/route add default 192.168.${ifnum}.1";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.poststop += "/sbin/ifconfig epair${ifnum}a destroy";
test1 {
$ifnum = 10;
}
# with more jails it seems crash the host more likely
test2 {
$ifnum = 20;
}
Then repeat stopping and starting jail service, the host crashes about once in
2 or 3 times.
# service jail onestart && service jail onestop
...
The kernel panic message:
Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 04
fault virtual address = 0x410
fault code = supervisor read data, page not present
instruction pointer = 0x20:0x80b9f237
stack pointer = 0x28:0xfe0015b55370
frame pointer = 0x28:0xfe0015b553f0
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 93087 (ifconfig)
trap number = 12
panic: page fault
cpuid = 2
time = 1612193992
KDB: stack backtrace:
#0 0x80c0aa85 at kdb_backtrace+0x65
#1 0x80bbed3b at vpanic+0x17b
#2 0x80bbebb3 at panic+0x43
#3 0x8108e911 at trap_fatal+0x391
#4 0x8108e96f at trap_pfault+0x4f
#5 0x8108dfb6 at trap+0x286
#6 0x81066938 at calltrap+0x8
#7 0x80bb9591 at _rm_rlock_hard+0x3c1
#8 0x80ce5ce6 at rtinit+0x2a6
#9 0x80d3873e at in_scrubprefix+0x29e
#10 0x80d5001d at rip_ctlinput+0x8d
#11 0x80c4922c at pfctlinput+0x5c
#12 0x80cbb4fa at if_down+0x12a
#13 0x80cb90d0 at if_detach_internal+0x150
#14 0x80cb8df0 at if_detach+0x50
#15 0x82b1ebb1 at epair_clone_destroy+0x81
#16 0x80cc0c4d at if_clone_destroyif+0xdd
#17 0x80cc0b12 at if_clone_destroy+0x1a2
Uptime: 1m22s
Dumping 160 out of 982 MB:..10%..20%..30%..40%..50%..60%..70%..80%..90%..100%
To be clear, after update to 12.2-RELEASE-p3, it's difficult to crash the host
without the below line in jail.conf:
exec.start += "/usr/sbin/daemon /usr/bin/nc -l 0.0.0.0 ";
I'll attach full core text dump later.
--
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326
--- Comment #21 from Michael Grimm ---
I am experiencing those crashes for a while now, and they continue to happen
even after migrating from 12-STABLE to 13-STABLE recently.
Note: I did try every recommendation regarding jail shutdown in /etc/jail.conf,
and whether removing vnet before final shutdown or not, doesen't prevent those
random crashes.
Here my relevant part of /etc/jail.conf regarding the the panic message
following below.
- /etc/rc.conf --
#
# host dependent global settings
#
$ip4prefixLOCAL = "10.10.10“;
$ip6prefixLOCAL = "fd00:e:e:e“;
#
# global jail settings
#
$MTU= "mtu 1490";
host.hostname = "${name}";
path= "/usr/home/jails/${name}";
mount.fstab = "/etc/fstab.${name}";
exec.consolelog = "/var/log/jail_${name}_console.log";
vnet= "new";
vnet.interface = "epair${jailID}b";
exec.clean;
mount.devfs;
persist;
#
# network settings to apply/destroy during start/stop of every jail
#
exec.prestart= "sleep 2";
exec.prestart += "/sbin/ifconfig epair${jailID} create up ${MTU}";
exec.prestart += "/sbin/ifconfig bridge0 addm epair${jailID}a";
exec.prestart += "/sbin/ifconfig epair${jailID}a";
exec.start = "/sbin/sysctl net.inet6.ip6.dad_count=0";
exec.start += "/sbin/ifconfig lo0 127.0.0.1 up";
exec.start += "/sbin/ifconfig epair${jailID}b inet ${ip4_addr}
${MTU}";
exec.start += "/sbin/ifconfig epair${jailID}b inet6 ${ip6_addr}
${MTU}";
exec.start += "/sbin/route add default -gateway
${ip4prefixLOCAL}.254";
exec.start += "/sbin/route add -inet6 default -gateway
${ip6prefixLOCAL}::254";
exec.stop= "/sbin/route del default";
exec.stop += "/sbin/route del -inet6 default";
exec.stop += "/bin/sh /etc/rc.shutdown";
# testing: reported to prevent from crashing (BUT: will crash as well!)
#exec.poststop = "/sbin/ifconfig epair${jailID}a -vnet ${jailID}";
exec.poststop += "/sbin/ifconfig epair${jailID}a destroy";
#
# individual jail settings
#
[snip]
jail5 {
$jailID = 5;
$ip4_addr= ${ip4prefixLOCAL}.5;
$ip6_addr= ${ip6prefixLOCAL}::5/64;
exec.start += "/bin/sh /etc/rc";
}
jail6 {
$jailID = 6;
$ip4_addr= ${ip4prefixLOCAL}.6;
$ip6_addr= ${ip6prefixLOCAL}::6/64;
exec.start += "/bin/sh /etc/rc";
}
- /var/log/messages ---
Jan 30 20:02:42 mer-waases kernel: epair5a: link state changed to
DOWN
Jan 30 20:02:42 mer-waases kernel: epair5b: link state changed to
DOWN
Jan 30 20:02:42 mer-waases kernel: in6_purgeaddr: err=65,
destination address delete failed
Jan 30 20:02:42 mer-waases kernel: Freed UMA keg (rtentry) was not
empty (1 items). Lost 1 pages of memory.
Jan 30 20:02:47 mer-waases kernel: epair6a: link state changed to
DOWN
Jan 30 20:02:47 mer-waases kernel: epair6b: link state changed to
DOWN
Jan 30 20:02:48 mer-waases kernel: in6_purgeaddr: err=65,
destination address delete failed
Jan 30 20:02:48 mer-waases kernel: Freed UMA keg (rtentry) was not
empty (1 items). Lost 1 pages of memory.
Jan 30 20:03:33 mer-waases syslogd: restart
Jan 30 20:03:33 mer-waases syslogd: kernel boot file is
/boot/kernel/kernel
Jan 30 20:03:33 mer-waases kernel:
Jan 30 20:03:33 mer-waases syslogd: last message repeated 1 times
Jan 30 20:03:33 mer-waases kernel: Fatal trap 12: page fault while
in kernel mode
Jan 30 20:03:33 mer-waases kernel: cpuid = 0; apic id = 00
Jan 30 20:03:33 mer-waases kernel: fault virtual address= 0x0
Jan 30 20:03:33 mer-waases kernel: fault code =
supervisor write data, page not present
Jan 30 20:03:33 mer-waases kernel: instruction pointer =
0x20:0x80c668be
Jan 30 20:03:33 mer-waases kernel: stack pointer=
0x28:0xfe000e9e86c0
Jan 30 20:03:33 mer-waases kernel: frame pointer=
0x28:0xfe000e9e8700
Jan 30 20:03:33 mer-waases kernel: code segment = base
0x0, limit 0xf, type 0x1b
Jan 30 20:03:33 mer-waases kernel: = DPL 0, pres
1, long 1, def32 0, gran 1
Jan 30 20:03:33 mer-waases kernel: processor eflags = interrupt
enabled, resume, IOPL = 0
Jan 30 20:03:33 mer-waases kernel: current process = 12
(swi1: netisr 0)
Jan 30 20:03:33 mer-waases kernel: trap number = 12
Jan 30 20:03:33 mer-waases kernel: panic: page fault
Jan 30 20:03:33 mer-waases kernel: cpuid = 0
Jan 30 20:03:33 mer-waases ker
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326 Kyle Evans changed: What|Removed |Added CC||kev...@freebsd.org --- Comment #20 from Kyle Evans --- (In reply to Zhenlei Huang from comment #19) A panic message would be helpful; some folks have noted a tangentially related use-after-free in similar circumstances. It'd be good to note if you're hitting the primary issue that kp fixed or a second UAF. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326 --- Comment #19 from Zhenlei Huang --- The problem is still present on 12.2-RELEASE-p3. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Jails - vnet- netgraph
On 28 Jan 2021, at 2:54, petru garstea wrote: Greetings, Finally, I sorted out. I have copied an example from the following path /usr/share/examples/netgraph/ether.bridge and found out that I needed to load modules into the kernel ng_socket.ko ng_bridge.ko ng_ether.ko ng_eiface.ko and jng script start working, I spawned a jail and confirmed that vnet netgraph interface was properly configured However I have another question, if I run ifconfig on the host I dont see in the list the netgraph bridge interface and the jails ether interface, I can see these interfaces only if run Vnet jails own the interfaces, so it’s entirely expected that you wouldn’t see them in on the host. ngctl list Name: ng0_vnetjail Type: eiface ID: 0010 Num hooks: 1 Name: re0 Type: ether ID: 0005 Num hooks: 2 Name: ngctl1544 Type: socket ID: 0017 Num hooks: 0 Name: re0bridge Type: bridge ID: 000b Num hooks: 3 If you can see those from the host that seems like it’d be a bug in the netgraph code. Regards, Kristof ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Jails - vnet- netgraph
Greetings, Finally, I sorted out. I have copied an example from the following path /usr/share/examples/netgraph/ether.bridge and found out that I needed to load modules into the kernel ng_socket.ko ng_bridge.ko ng_ether.ko ng_eiface.ko and jng script start working, I spawned a jail and confirmed that vnet netgraph interface was properly configured However I have another question, if I run ifconfig on the host I dont see in the list the netgraph bridge interface and the jails ether interface, I can see these interfaces only if run ngctl list Name: ng0_vnetjail Type: eiface ID: 0010 Num hooks: 1 Name: re0 Type: ether ID: 0005 Num hooks: 2 Name: ngctl1544 Type: socket ID: 0017 Num hooks: 0 Name: re0bridge Type: bridge ID: 000b Num hooks: 3 Please advise Cheers, Petru Garstea On 1/27/21 5:36 AM, Kristof Provost wrote: On 27 Jan 2021, at 3:59, petru garstea wrote: In the recent FreeBSD magazines it was mentioned that "bridging" was refactored and I would like to know if that might be impacted the netgraph bridge. No, the if_bridge changes do not affect the netgraph code. Best regards, Kristof ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Jails - vnet- netgraph
On 27 Jan 2021, at 3:59, petru garstea wrote: In the recent FreeBSD magazines it was mentioned that "bridging" was refactored and I would like to know if that might be impacted the netgraph bridge. No, the if_bridge changes do not affect the netgraph code. Best regards, Kristof ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Jails - vnet- netgraph
Hi Ernie,
jib script is working fine, however in my current setup I need to
emulate bridge interface with netgraph subsystem, I tried to manage that
part with jng script with no luck then I decided to create the netgraph
bridge manually using ngctl client and in the end the result was the same.
In the recent FreeBSD magazines it was mentioned that "bridging" was
refactored and I would like to know if that might be impacted the
netgraph bridge.
Please advise
Cheers,
Petru Garstea
On 1/26/21 12:53 PM, Ernie Luzar wrote:
petru garstea wrote:
Greetings FreeBSD community,
   OS: FreeBSD sun 12.2-RELEASE-p1 FreeBSD 12.2-RELEASE-p1
GENERICÂ amd64
I am trying to build a netgraph vnet jail with support of official
jng script that comes with FreeBSD and developed by Devin Teske.
jail.conf file
netgraph {
 devfs_ruleset = 13;
 enforce_statfs = 2;
 exec.clean;
 exec.consolelog = /var/log/bastille/netgraph_console.log;
 exec.start = '/bin/sh /etc/rc';
 exec.stop = '/bin/sh /etc/rc.shutdown';
 host.hostname = netgraph;
 mount.devfs;
 mount.fstab = /usr/local/bastille/jails/netgraph/fstab;
 path = /usr/local/bastille/jails/netgraph/root;
 securelevel = 2;
 vnet;
 vnet.interface = e0b_bastille0;
# exec.prestart += "jib addm bastille0 re0";
# exec.poststop += "jib destroy bastille0";
 exec.prestart += "jng bridge netgraph re0";
 exec.poststop += "jng shutdown netgraph" ;
}
When I start the jail, netgraph subsystem raise the following exception
ngctl: send msg: No such file or directory
jail: netgraph: jng bridge netgraph re0: failed
I tried also to create the netgraph bridge with not using jng script
ngctl mkpeer re0: bridge lower link0
ngctl: send msg: No such file or directory
From what I found it looks it used to work on FreeBSD 11.x and
stopped working in version 12.
Any thoughts ?
Please advise
Cheers,
Petru Garstea
Don't see any reply so I will try to help you.
If I remember correctly the jib and jng was added as documentation
back around freebsd 10.00. I have tried to get it to work 10+, 11+
,12+ with no joy. There is something missing but can not tell what it
is. The jail environment has gone through many changes over time so no
wonder jib/jng don't work now.
Netgraph is a complete subsystem for network configuration that has
it's own syntax and commands. The learning curve is pretty great.
There is a outstanding bug and Devin Teske & (she) has taken up the
bug. Hopping 13 holds the bug fix.
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Jails - vnet- netgraph
petru garstea wrote:
Greetings FreeBSD community,
   OS: FreeBSD sun 12.2-RELEASE-p1 FreeBSD 12.2-RELEASE-p1 GENERICÂ
amd64
I am trying to build a netgraph vnet jail with support of official jng
script that comes with FreeBSD and developed by Devin Teske.
jail.conf file
netgraph {
 devfs_ruleset = 13;
 enforce_statfs = 2;
 exec.clean;
 exec.consolelog = /var/log/bastille/netgraph_console.log;
 exec.start = '/bin/sh /etc/rc';
 exec.stop = '/bin/sh /etc/rc.shutdown';
 host.hostname = netgraph;
 mount.devfs;
 mount.fstab = /usr/local/bastille/jails/netgraph/fstab;
 path = /usr/local/bastille/jails/netgraph/root;
 securelevel = 2;
 vnet;
 vnet.interface = e0b_bastille0;
# exec.prestart += "jib addm bastille0 re0";
# exec.poststop += "jib destroy bastille0";
 exec.prestart += "jng bridge netgraph re0";
 exec.poststop += "jng shutdown netgraph" ;
}
When I start the jail, netgraph subsystem raise the following exception
ngctl: send msg: No such file or directory
jail: netgraph: jng bridge netgraph re0: failed
I tried also to create the netgraph bridge with not using jng script
ngctl mkpeer re0: bridge lower link0
ngctl: send msg: No such file or directory
From what I found it looks it used to work on FreeBSD 11.x and stopped
working in version 12.
Any thoughts ?
Please advise
Cheers,
Petru Garstea
Don't see any reply so I will try to help you.
If I remember correctly the jib and jng was added as documentation back
around freebsd 10.00. I have tried to get it to work 10+, 11+ ,12+ with
no joy. There is something missing but can not tell what it is. The jail
environment has gone through many changes over time so no wonder jib/jng
don't work now.
Netgraph is a complete subsystem for network configuration that has it's
own syntax and commands. The learning curve is pretty great. There is a
outstanding bug and Devin Teske & (she) has taken up the bug. Hopping 13
holds the bug fix.
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Jails - vnet- netgraph
Greetings FreeBSD community,
OS: FreeBSD sun 12.2-RELEASE-p1 FreeBSD 12.2-RELEASE-p1 GENERIC amd64
I am trying to build a netgraph vnet jail with support of official jng
script that comes with FreeBSD and developed by Devin Teske.
jail.conf file
netgraph {
devfs_ruleset = 13;
enforce_statfs = 2;
exec.clean;
exec.consolelog = /var/log/bastille/netgraph_console.log;
exec.start = '/bin/sh /etc/rc';
exec.stop = '/bin/sh /etc/rc.shutdown';
host.hostname = netgraph;
mount.devfs;
mount.fstab = /usr/local/bastille/jails/netgraph/fstab;
path = /usr/local/bastille/jails/netgraph/root;
securelevel = 2;
vnet;
vnet.interface = e0b_bastille0;
# exec.prestart += "jib addm bastille0 re0";
# exec.poststop += "jib destroy bastille0";
exec.prestart += "jng bridge netgraph re0";
exec.poststop += "jng shutdown netgraph" ;
}
When I start the jail, netgraph subsystem raise the following exception
ngctl: send msg: No such file or directory
jail: netgraph: jng bridge netgraph re0: failed
I tried also to create the netgraph bridge with not using jng script
ngctl mkpeer re0: bridge lower link0
ngctl: send msg: No such file or directory
From what I found it looks it used to work on FreeBSD 11.x and stopped
working in version 12.
Any thoughts ?
Please advise
Cheers,
Petru Garstea
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Need help with VNET, Jail and IPv6
Le mardi 05 janv. 2021 à 21:39:27 (+), Mina Galić à écrit:
>
> > Hello all,
>
> Hi Jacques,
>
>
> > On my hosted machine I already have many "classical" jails.
> >
> > But I would like to switch to modern schema with Bridge and vnet.
> >
> > With IPv4 I have no problem. In fact is almost like without Bridge/VNET:
>
> For:
>
> https://alpha.pkgbase.live/
>
> instead of libioc I just used jail.conf. With:
>
> https://antranigv.am/weblog_en/posts/vnet-jail-howto/
I already read this and succesfully made a IPv4 jail with this tuto.
>
> as basis for the IPv4 setup.
>
> > My goal is first to have on jail (myjail) working with IPv4 and IPv6 then,
> >
> > slowly migrate the old jail to the new way.
> >
> > So, I need help to configure myjail to have IPv6 working:
> >
> > - configure a IPv6 on e0b_myjail is easy, but which defaultrouter6 did I
> > use?
> > - did the bridge have an IPv6 to be the defaultrouter6? I try with no
> > luck.
> > - did I need some configuration on PF?
> >
> > Thanks for reading me (I sure I not really clear) and for your advice.
> >
> > Btw, after I successfully configure myjail (and the other one) I will
> > wrote a how-to.
> >
>
> Okay, let's see if I can hit all beats:
>
> Here's the paste of webserver.jail.conf, rc.conf (highlights) and pf.conf
>
> https://gist.github.com/87ba10c1c5611ed32367d5d48ef5f402
Thanks, that really clear.
>
> I'll explain some of the important bits:
>
> my ISP binds the IPv4 to the MAC, but not the IPv6, go figure.
> That's why I leave the IPv4 address on the main interface, instead of
> fiddling with MAC addresses and moving it to the bridge.
>
> On the bridge, we have the IPv6 and the IPv4 NAT; That's handy, as it also
> means we only need one interface for both IPv4 and IPv6.
>
> cloned_interfaces="bridge0"
> # jail NAT and Network access
> ifconfig_bridge0="inet 192.168.17.1/24"
> gateway_enable="YES"
>
> note that we explicitly enable link-local addresses, because, as per spec,
> they are needed to make IPv6 work:
>
> # working IPv6 setup needs link-local addresses (according to the spec)
> ipv6_activate_all_interfaces="YES"
> ifconfig_bridge0_ipv6="inet6 2a01:4f9:c010:c64c::1/64 auto_linklocal"
> ipv6_defaultrouter="fe80::1%vtnet0"
Why vtnet instead of vnet ? Is there a difference that I did saw?
> # enable IPv6 gateway
> ipv6_gateway_enable="YES"
>
> and in the jail.conf it's really just about adding the IPv6 addresses to the
> interfaces, too!
>
> vnet.interface = "$jepair";
>
> exec.prestart = "ifconfig epair${id} create up";
> exec.prestart += "ifconfig epair${id}a up descr vnet-${name}";
> exec.prestart += "ifconfig $bridge addm epair${id}a up";
>
> exec.start = "/sbin/ifconfig lo0 127.0.0.1 up";
> exec.start += "/sbin/ifconfig epair${id}b ${ipaddr}";
> exec.start += "/sbin/ifconfig epair${id}b inet6 ${ip6addr}";
> exec.start += "/sbin/route add default ${gw}";
> exec.start += "/sbin/route add -inet6 default ${gw6}";
> exec.start += "/bin/sh /etc/rc";
>
> I also highly recommend adding IPv6 nameservers to your resolv.conf; that
> way, if you broke your IPv4 setup, you still have working IPv6!
That a good advice too :-)
> Being NAT, IPv4 routing is obviously happening via the host.
> And, given that my ISP uses fe80::1 as the default gateway, the only way
> to make jails' IPv6 routing work was by routing it thru the host.
> as for pf, it's only used for NAT.
> No firewalling, and I'm not doing anything to IPv6.
>
> That's all from me, i hope it helps.
Sure it'a help, thanks for your advices, your time and expertise.
--
Jacques Foucry
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: Need help with VNET, Jail and IPv6
> Hello all,
Hi Jacques,
> On my hosted machine I already have many "classical" jails.
>
> But I would like to switch to modern schema with Bridge and vnet.
>
> With IPv4 I have no problem. In fact is almost like without Bridge/VNET:
For:
https://alpha.pkgbase.live/
instead of libioc I just used jail.conf. With:
https://antranigv.am/weblog_en/posts/vnet-jail-howto/
as basis for the IPv4 setup.
> My goal is first to have on jail (myjail) working with IPv4 and IPv6 then,
>
> slowly migrate the old jail to the new way.
>
> So, I need help to configure myjail to have IPv6 working:
>
> - configure a IPv6 on e0b_myjail is easy, but which defaultrouter6 did I
> use?
> - did the bridge have an IPv6 to be the defaultrouter6? I try with no luck.
> - did I need some configuration on PF?
>
> Thanks for reading me (I sure I not really clear) and for your advice.
>
> Btw, after I successfully configure myjail (and the other one) I will
> wrote a how-to.
>
Okay, let's see if I can hit all beats:
Here's the paste of webserver.jail.conf, rc.conf (highlights) and pf.conf
https://gist.github.com/87ba10c1c5611ed32367d5d48ef5f402
I'll explain some of the important bits:
my ISP binds the IPv4 to the MAC, but not the IPv6, go figure.
That's why I leave the IPv4 address on the main interface, instead of fiddling
with MAC addresses and moving it to the bridge.
On the bridge, we have the IPv6 and the IPv4 NAT; That's handy, as it also
means we only need one interface for both IPv4 and IPv6.
cloned_interfaces="bridge0"
# jail NAT and Network access
ifconfig_bridge0="inet 192.168.17.1/24"
gateway_enable="YES"
note that we explicitly enable link-local addresses, because, as per spec, they
are needed to make IPv6 work:
# working IPv6 setup needs link-local addresses (according to the spec)
ipv6_activate_all_interfaces="YES"
ifconfig_bridge0_ipv6="inet6 2a01:4f9:c010:c64c::1/64 auto_linklocal"
ipv6_defaultrouter="fe80::1%vtnet0"
# enable IPv6 gateway
ipv6_gateway_enable="YES"
and in the jail.conf it's really just about adding the IPv6 addresses to the
interfaces, too!
vnet.interface = "$jepair";
exec.prestart = "ifconfig epair${id} create up";
exec.prestart += "ifconfig epair${id}a up descr vnet-${name}";
exec.prestart += "ifconfig $bridge addm epair${id}a up";
exec.start = "/sbin/ifconfig lo0 127.0.0.1 up";
exec.start += "/sbin/ifconfig epair${id}b ${ipaddr}";
exec.start += "/sbin/ifconfig epair${id}b inet6 ${ip6addr}";
exec.start += "/sbin/route add default ${gw}";
exec.start += "/sbin/route add -inet6 default ${gw6}";
exec.start += "/bin/sh /etc/rc";
I also highly recommend adding IPv6 nameservers to your resolv.conf; that way,
if you broke your IPv4 setup, you still have working IPv6!
Being NAT, IPv4 routing is obviously happening via the host.
And, given that my ISP uses fe80::1 as the default gateway, the only way to
make jails' IPv6 routing work was by routing it thru the host.
as for pf, it's only used for NAT.
No firewalling, and I'm not doing anything to IPv6.
That's all from me, i hope it helps.
> --
>
> Jacques Foucry
best of luck,
Mina
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Need help with VNET, Jail and IPv6
Hello all,
On my hosted machine I already have many "classical" jails.
But I would like to switch to modern schema with Bridge and vnet.
With IPv4 I have no problem. In fact is almost like without Bridge/VNET:
ifconfig em0bridge
em0bridge: flags=8843 metric 0 mtu 1500
ether 02:36:b3:c1:8a:00
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: em0 flags=143
ifmaxaddr 0 port 1 priority 128 path cost 2
groups: bridge
nd6 options=9
nyjail{
host.hostname="mywebite.fr";
path="/jails/mywebsite";
allow.mount=true;
allow.raw_sockets;
vnet;
vnet.interface = "e0b_${name}";
exec.prestart += "jib addm ${name} em0";
exec.poststop += "jib destroy ${name}";
exec.start = "/sbin/ifconfig e0b_${name} 10.1.1.28/24";
exec.start += "/sbin/route add default 10.1.1.254";
exec.poststart += "/sbin/ifconfig e0a_${name} 10.1.1.254/24";
exec.poststop += "/sbin/ifconfig e0b_${name} -vnet ${name}";
exec.poststop += "/sbin/ifconfig deletem e0b_${name}";
exec.poststop += "sleep 2";
exec.poststop += "/sbin/ifconfig e0b_${name} destroy";
persist=true;
mount.fstab="/etc/fstab.${name}";
}
With pf I could connect througt ssh to my jail:
jails_net = "{192.168.12.0/24 10.1.1.0/24}"
nat on $ext_if from $jails_net to any -> ($ext_if)
myjail_v4="10.1.1.28"
myjail_v6="2a01:4f9:4a:1fd8::28"
myjail_ports = 2228
rdr on $ext_if inet proto tcp from any to $ext_if port $myjail_ports ->
$myjail_v4
rdr on $ext_if inet6 proto tcp from any to $ext_if port $myjail_ports ->
$myjail_v6
pass in log quick on $ext_if proto tcp from any to $myjail_v4 port
pass in log quick on $ext_if proto tcp from any to $myjail_v6 port
The old fashion mail use em0_alias for IPv6
em0: flags=8943 metric 0 mtu
1500
options=81009b
ether b4:2e:99:6a:80:9d
inet6 2a01:4f9:4a:1fd8::2 prefixlen 64
inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1
inet6 2a01:4f9:4a:1fd8::5 prefixlen 64
inet6 2a01:4f9:4a:1fd8::16 prefixlen 64
inet6 2a01:4f9:4a:1fd8::14 prefixlen 64
inet6 2a01:4f9:4a:1fd8::15 prefixlen 64
inet6 2a01:4f9:4a:1fd8::21 prefixlen 64
inet6 2a01:4f9:4a:1fd8::25 prefixlen 64
inet6 2a01:4f9:4a:1fd8::29 prefixlen 64
inet6 2a01:4f9:4a:1fd8::17 prefixlen 64
inet6 2a01:4f9:4a:1fd8::11 prefixlen 64
inet6 2a01:4f9:4a:1fd8::12 prefixlen 64
inet6 2a01:4f9:4a:1fd8::18 prefixlen 64
inet6 2a01:4f9:4a:1fd8::22 prefixlen 64
inet6 2a01:4f9:4a:1fd8::19 prefixlen 64
inet6 2a01:4f9:4a:1fd8::28 prefixlen 64
inet 95.217.83.231 netmask 0xffc0 broadcast 95.217.83.255
media: Ethernet autoselect (1000baseT )
status: active
nd6 options=21
My goal is first to have on jail (myjail) working with IPv4 and IPv6 then,
slowly migrate the old jail to the new way.
So, I need help to configure myjail to have IPv6 working:
- configure a IPv6 on e0b_myjail is easy, but which defaultrouter6 did I use?
- did the bridge have an IPv6 to be the defaultrouter6? I try with no luck.
- did I need some configuration on PF?
Thanks for reading me (I sure I not really clear) and for your advice.
Btw, after I successfully configure myjail (and the other one) I will wrote a
how-to.
--
Jacques Foucry
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: VNET jails + VLAN over LAGG
A little followup on this, in an attempt to virtualize my FreeBSD router/firewall, it almost works with https://gist.github.com/silenius/5f556a036330f1595e2e6fcdd5e5e18e The only thing the doesn't work is the vhid (CARP) on the epairxb interface: as long as the jail is running it works, if I'm stopping the jail the other side switch from BACKUP to MASTER, which is OK, but when I'm starting the jail again afterwards the epairxb never goes to MASTER mode, (it stays in BACKUP mode) although it should... any idea? is CARP supposed to work with epair interfaces? Thanks, Julien On Thu, Oct 22, 2020 at 01:18:08PM +0200, Julien Cigar wrote: > Hello, > > I've a lagg0 interface with three ports: igb0,igb1,igb2 (with LACP). On > top of that I've several VLAN interfaces: vlan10, vlan11, vlan12 with > vlandev lagg0. All those vlans have ip addresses and one of them shares > also a vhid (through CARP). > Translated in ifconfig/rc.conf it gives (1) > > Currently I've several jails, all non-VNET, and I'd like to add a bunch > of VNET jails through epair and bridge. I'm wondering how should it be > done regarding the VLAN/LAGG interface(s).. (given that non-VNET jails > should continue to work too)? > > Some things I wonder: > - If I'm adding a vlan interface to a bridge, I guess the IP addresses > should be moved to the bridge, right? How will behave the non-VNET > jails..? How will behave the vhid on the HOST? Should I add a tap > interface in the HOST on top of the bridge too? > > - From what I can read the best is to create one bridge per vlan, adding > the corresponding HOST vlan and the epairxa, is this correct? > > Thanks, > Julien > > (1) > https://gist.githubusercontent.com/silenius/6066696fe78c95177548319f125d9c44/raw/0319e4d1cad33201ea66e2258a74f8349116fbc9/gistfile1.txt > > -- > Julien Cigar > Belgian Biodiversity Platform (http://www.biodiversity.be) > PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 > No trees were killed in the creation of this message. > However, many electrons were terribly inconvenienced. > ___ > freebsd-jail@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-jail > To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org" -- Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
VNET jails + VLAN over LAGG
Hello, I've a lagg0 interface with three ports: igb0,igb1,igb2 (with LACP). On top of that I've several VLAN interfaces: vlan10, vlan11, vlan12 with vlandev lagg0. All those vlans have ip addresses and one of them shares also a vhid (through CARP). Translated in ifconfig/rc.conf it gives (1) Currently I've several jails, all non-VNET, and I'd like to add a bunch of VNET jails through epair and bridge. I'm wondering how should it be done regarding the VLAN/LAGG interface(s).. (given that non-VNET jails should continue to work too)? Some things I wonder: - If I'm adding a vlan interface to a bridge, I guess the IP addresses should be moved to the bridge, right? How will behave the non-VNET jails..? How will behave the vhid on the HOST? Should I add a tap interface in the HOST on top of the bridge too? - From what I can read the best is to create one bridge per vlan, adding the corresponding HOST vlan and the epairxa, is this correct? Thanks, Julien (1) https://gist.githubusercontent.com/silenius/6066696fe78c95177548319f125d9c44/raw/0319e4d1cad33201ea66e2258a74f8349116fbc9/gistfile1.txt -- Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
Hi Ernie, please consider the last block in my second email, that is the jail.conf for the non-dedicated interface. The host runs all "normal" IP networking on em0. On Wed, 14 Oct 2020 at 17:14, Ernie Luzar wrote: > Arsenij Solovjev wrote: > > On Wed, 14 Oct 2020 at 15:41, Kristof Provost wrote: > > > >> On 14 Oct 2020, at 15:36, Arsenij Solovjev wrote: > >>> On Wed, 14 Oct 2020 at 14:42, Kristof Provost wrote: > >>> > >>>> On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote: > >>>>> Hi all! > >>>>> Does anybody know if it's possible to run a vnet jail on a > >>>>> non-dedicated > >>>>> interface? I have the Lucas book on jails. In it he says that for > >>>>> vnet > >>>>> you > >>>>> need to pick a dedicated interface, remove all networking IP > >>>>> configuration > >>>>> and only bring it up. Afterwards you set up jib and whatnot. > >>>>> > >>>>> All works well if I use a dedicated secondary interface (let's call > >>>>> it > >>>>> em1). If I use em0 however I cannot ping the jail. > >>>>> > >>>>> I would like to have a host with that has a single network interface > >>>>> which > >>>>> is used for both normal networking stuff as well as having the vnet > >>>>> jail > >>>>> run on it. > >>>>> > >>>>> Maybe I could create some sort of virtual interface and run vnet on > >>>>> it? > >>>>> > >>>>> Any ideas here? Thanks in advance! > >>>>> > >>>> Look at epair interfaces. > >>>> > >>>> You can put em0 and epair0a in a bridge together and add epair0b to > >>>> the > >>>> vnet jail. > >>>> That gets the vnet jail connected to your LAN. > >>>> > >>>> Or you can skip the bridge, assign an IP to epair0a and route between > >>>> the jail and your LAN. > >>>> > >>>> Regards, > >>>> Kristof > >>>> > >>> Hi Kristof, > >>> > >>> Thanks for your reply! > >>> > >>> considering your first idea. I did this, the jail gets created > >>> seemingly > >>> fine. However I cannot ping the ip of epair0b (this works when using a > >>> dedicated interface). > >>> Also I cannot reach my gateway from within the jail. This too works > >>> when > >>> using a dedicated interface. > >>> Btw I have "sysctl security.jail.allow_raw_sockets=1". > >>> snip: > > >>> > >> This is odd. Are you assigning a new MAC address to the epair interfaces > >> somewhere? Both ends of the epair seem to have a new MAC address, and > >> the same one at that. > >> > >> Regards, > >> Kristof > >> > > > > Not explicitly, no, I let the jib script do the epair creation. > > > To Arsenij Solovjev > > For the record sure would like to see your jail.conf file where you > setup this non-dedicated vnet jail system. > > I believe a non-dedicated vnet jail is for local access only. Is that > correct? > > The bridge setup is for public internet access? Is that correct? > > > To Kristof Provost > > In your reply you said. > "Or you can skip the bridge, assign an IP to epair0a and route between > the jail and your LAN." > Please explain this statement. Route how? > > > > > > > > > ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
Arsenij Solovjev wrote: On Wed, 14 Oct 2020 at 15:41, Kristof Provost wrote: On 14 Oct 2020, at 15:36, Arsenij Solovjev wrote: On Wed, 14 Oct 2020 at 14:42, Kristof Provost wrote: On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote: Hi all! Does anybody know if it's possible to run a vnet jail on a non-dedicated interface? I have the Lucas book on jails. In it he says that for vnet you need to pick a dedicated interface, remove all networking IP configuration and only bring it up. Afterwards you set up jib and whatnot. All works well if I use a dedicated secondary interface (let's call it em1). If I use em0 however I cannot ping the jail. I would like to have a host with that has a single network interface which is used for both normal networking stuff as well as having the vnet jail run on it. Maybe I could create some sort of virtual interface and run vnet on it? Any ideas here? Thanks in advance! Look at epair interfaces. You can put em0 and epair0a in a bridge together and add epair0b to the vnet jail. That gets the vnet jail connected to your LAN. Or you can skip the bridge, assign an IP to epair0a and route between the jail and your LAN. Regards, Kristof Hi Kristof, Thanks for your reply! considering your first idea. I did this, the jail gets created seemingly fine. However I cannot ping the ip of epair0b (this works when using a dedicated interface). Also I cannot reach my gateway from within the jail. This too works when using a dedicated interface. Btw I have "sysctl security.jail.allow_raw_sockets=1". snip: This is odd. Are you assigning a new MAC address to the epair interfaces somewhere? Both ends of the epair seem to have a new MAC address, and the same one at that. Regards, Kristof Not explicitly, no, I let the jib script do the epair creation. To Arsenij Solovjev For the record sure would like to see your jail.conf file where you setup this non-dedicated vnet jail system. I believe a non-dedicated vnet jail is for local access only. Is that correct? The bridge setup is for public internet access? Is that correct? To Kristof Provost In your reply you said. "Or you can skip the bridge, assign an IP to epair0a and route between the jail and your LAN." Please explain this statement. Route how? ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
Btw, an important detail I left out, I'm running a Samba DC with Bind9 inside the jail. Maybe this interferes somehow? On Wed, 14 Oct 2020 at 15:41, Kristof Provost wrote: > On 14 Oct 2020, at 15:36, Arsenij Solovjev wrote: > > On Wed, 14 Oct 2020 at 14:42, Kristof Provost wrote: > > > >> On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote: > >>> Hi all! > >>> Does anybody know if it's possible to run a vnet jail on a > >>> non-dedicated > >>> interface? I have the Lucas book on jails. In it he says that for > >>> vnet > >>> you > >>> need to pick a dedicated interface, remove all networking IP > >>> configuration > >>> and only bring it up. Afterwards you set up jib and whatnot. > >>> > >>> All works well if I use a dedicated secondary interface (let's call > >>> it > >>> em1). If I use em0 however I cannot ping the jail. > >>> > >>> I would like to have a host with that has a single network interface > >>> which > >>> is used for both normal networking stuff as well as having the vnet > >>> jail > >>> run on it. > >>> > >>> Maybe I could create some sort of virtual interface and run vnet on > >>> it? > >>> > >>> Any ideas here? Thanks in advance! > >>> > >> Look at epair interfaces. > >> > >> You can put em0 and epair0a in a bridge together and add epair0b to > >> the > >> vnet jail. > >> That gets the vnet jail connected to your LAN. > >> > >> Or you can skip the bridge, assign an IP to epair0a and route between > >> the jail and your LAN. > >> > >> Regards, > >> Kristof > >> > > > > Hi Kristof, > > > > Thanks for your reply! > > > > considering your first idea. I did this, the jail gets created > > seemingly > > fine. However I cannot ping the ip of epair0b (this works when using a > > dedicated interface). > > Also I cannot reach my gateway from within the jail. This too works > > when > > using a dedicated interface. > > Btw I have "sysctl security.jail.allow_raw_sockets=1". > > Here is my host ifconfig when putting em0 and epair0a in a bridge: > > > > em0: flags=8943 metric > > 0 > >> mtu 1500 > >> > > > options=812099 > > > > ether 9a:4c:eb:b5:95:bf > > > > inet 172.18.20.145 netmask 0xff00 broadcast 172.18.20.255 > > > > media: Ethernet autoselect (1000baseT ) > > > > status: active > > > > nd6 options=29 > > > > jailether: flags=8843 metric 0 > > mtu > >> 1500 > >> > > > options=81209b > > > > ether 56:39:b7:c5:2e:ec > > > > media: Ethernet autoselect (1000baseT ) > > > > status: active > > > > nd6 options=29 > > > > lo0: flags=8049 metric 0 mtu 16384 > >> > > options=680003 > > > > inet6 ::1 prefixlen 128 > > > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > > > > inet 127.0.0.1 netmask 0xff00 > > > > inet 10.43.84.1 netmask 0xff00 > > > > groups: lo > > > > nd6 options=21 > > > > em0bridge: flags=8843 metric 0 > > mtu > >> 1500 > >> > > ether 02:13:0b:48:53:00 > > > > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > > > > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > > > > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > > > > member: e0a_sambaad flags=143 > > > >ifmaxaddr 0 port 5 priority 128 path cost 2000 > > > > member: em0 flags=143 > > > >ifmaxaddr 0 port 1 priority 128 path cost 2 > > > > groups: bridge > > > > nd6 options=1 > > > > e0a_sambaad: > > flags=8943 > >> metric 0 mtu 1500 > >> > > options=8 > > > > ether 02:a4:c4:b5:95:bf > > > > hwaddr 02:78:fd:34:e8:0a > > > > groups: epair > > > > media: Ethernet 10Gbase-T (10Gbase-T ) > > > > status: active > > > > nd6 options=29 > > > > > > > > Here's the ifconfig from my within my jail: > > > > lo0: flags=8049 metric 0 mtu 16384 > >> > > options=680003 > > > > inet6 ::1 prefixlen 128 > > > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > > > > inet 127.0.0.1 netmask 0xff00 > > > > groups: lo > > > > nd6 options=21 > > > > e0b_sambaad: flags=8843 metric > > 0 > >> mtu 1500 > >> > > options=8 > > > > ether 0e:a4:c4:b5:95:bf > > hwaddr 02:78:fd:34:e8:0b > > > This is odd. Are you assigning a new MAC address to the epair interfaces > somewhere? Both ends of the epair seem to have a new MAC address, and > the same one at that. > > Regards, > Kristof > ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
On Wed, 14 Oct 2020 at 15:41, Kristof Provost wrote: > On 14 Oct 2020, at 15:36, Arsenij Solovjev wrote: > > On Wed, 14 Oct 2020 at 14:42, Kristof Provost wrote: > > > >> On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote: > >>> Hi all! > >>> Does anybody know if it's possible to run a vnet jail on a > >>> non-dedicated > >>> interface? I have the Lucas book on jails. In it he says that for > >>> vnet > >>> you > >>> need to pick a dedicated interface, remove all networking IP > >>> configuration > >>> and only bring it up. Afterwards you set up jib and whatnot. > >>> > >>> All works well if I use a dedicated secondary interface (let's call > >>> it > >>> em1). If I use em0 however I cannot ping the jail. > >>> > >>> I would like to have a host with that has a single network interface > >>> which > >>> is used for both normal networking stuff as well as having the vnet > >>> jail > >>> run on it. > >>> > >>> Maybe I could create some sort of virtual interface and run vnet on > >>> it? > >>> > >>> Any ideas here? Thanks in advance! > >>> > >> Look at epair interfaces. > >> > >> You can put em0 and epair0a in a bridge together and add epair0b to > >> the > >> vnet jail. > >> That gets the vnet jail connected to your LAN. > >> > >> Or you can skip the bridge, assign an IP to epair0a and route between > >> the jail and your LAN. > >> > >> Regards, > >> Kristof > >> > > > > Hi Kristof, > > > > Thanks for your reply! > > > > considering your first idea. I did this, the jail gets created > > seemingly > > fine. However I cannot ping the ip of epair0b (this works when using a > > dedicated interface). > > Also I cannot reach my gateway from within the jail. This too works > > when > > using a dedicated interface. > > Btw I have "sysctl security.jail.allow_raw_sockets=1". > > Here is my host ifconfig when putting em0 and epair0a in a bridge: > > > > em0: flags=8943 metric > > 0 > >> mtu 1500 > >> > > > options=812099 > > > > ether 9a:4c:eb:b5:95:bf > > > > inet 172.18.20.145 netmask 0xff00 broadcast 172.18.20.255 > > > > media: Ethernet autoselect (1000baseT ) > > > > status: active > > > > nd6 options=29 > > > > jailether: flags=8843 metric 0 > > mtu > >> 1500 > >> > > > options=81209b > > > > ether 56:39:b7:c5:2e:ec > > > > media: Ethernet autoselect (1000baseT ) > > > > status: active > > > > nd6 options=29 > > > > lo0: flags=8049 metric 0 mtu 16384 > >> > > options=680003 > > > > inet6 ::1 prefixlen 128 > > > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > > > > inet 127.0.0.1 netmask 0xff00 > > > > inet 10.43.84.1 netmask 0xff00 > > > > groups: lo > > > > nd6 options=21 > > > > em0bridge: flags=8843 metric 0 > > mtu > >> 1500 > >> > > ether 02:13:0b:48:53:00 > > > > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > > > > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > > > > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > > > > member: e0a_sambaad flags=143 > > > >ifmaxaddr 0 port 5 priority 128 path cost 2000 > > > > member: em0 flags=143 > > > >ifmaxaddr 0 port 1 priority 128 path cost 2 > > > > groups: bridge > > > > nd6 options=1 > > > > e0a_sambaad: > > flags=8943 > >> metric 0 mtu 1500 > >> > > options=8 > > > > ether 02:a4:c4:b5:95:bf > > > > hwaddr 02:78:fd:34:e8:0a > > > > groups: epair > > > > media: Ethernet 10Gbase-T (10Gbase-T ) > > > > status: active > > > > nd6 options=29 > > > > > > > > Here's the ifconfig from my within my jail: > > > > lo0: flags=8049 metric 0 mtu 16384 > >> > > options=680003 > > > > inet6 ::1 prefixlen 128 > > > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > > > > inet 127.0.0.1 netmask 0xff00 > > > > groups: lo > > > > nd6 options=21 > > > > e0b_sambaad: flags=8843 metric > > 0 > >> mtu 1500 > >> > > options=8 > > > > ether 0e:a4:c4:b5:95:bf > > hwaddr 02:78:fd:34:e8:0b > > > This is odd. Are you assigning a new MAC address to the epair interfaces > somewhere? Both ends of the epair seem to have a new MAC address, and > the same one at that. > > Regards, > Kristof > Not explicitly, no, I let the jib script do the epair creation. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
On 14 Oct 2020, at 15:36, Arsenij Solovjev wrote: On Wed, 14 Oct 2020 at 14:42, Kristof Provost wrote: On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote: Hi all! Does anybody know if it's possible to run a vnet jail on a non-dedicated interface? I have the Lucas book on jails. In it he says that for vnet you need to pick a dedicated interface, remove all networking IP configuration and only bring it up. Afterwards you set up jib and whatnot. All works well if I use a dedicated secondary interface (let's call it em1). If I use em0 however I cannot ping the jail. I would like to have a host with that has a single network interface which is used for both normal networking stuff as well as having the vnet jail run on it. Maybe I could create some sort of virtual interface and run vnet on it? Any ideas here? Thanks in advance! Look at epair interfaces. You can put em0 and epair0a in a bridge together and add epair0b to the vnet jail. That gets the vnet jail connected to your LAN. Or you can skip the bridge, assign an IP to epair0a and route between the jail and your LAN. Regards, Kristof Hi Kristof, Thanks for your reply! considering your first idea. I did this, the jail gets created seemingly fine. However I cannot ping the ip of epair0b (this works when using a dedicated interface). Also I cannot reach my gateway from within the jail. This too works when using a dedicated interface. Btw I have "sysctl security.jail.allow_raw_sockets=1". Here is my host ifconfig when putting em0 and epair0a in a bridge: em0: flags=8943 metric 0 mtu 1500 options=812099 ether 9a:4c:eb:b5:95:bf inet 172.18.20.145 netmask 0xff00 broadcast 172.18.20.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 jailether: flags=8843 metric 0 mtu 1500 options=81209b ether 56:39:b7:c5:2e:ec media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff00 inet 10.43.84.1 netmask 0xff00 groups: lo nd6 options=21 em0bridge: flags=8843 metric 0 mtu 1500 ether 02:13:0b:48:53:00 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: e0a_sambaad flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 2 groups: bridge nd6 options=1 e0a_sambaad: flags=8943 metric 0 mtu 1500 options=8 ether 02:a4:c4:b5:95:bf hwaddr 02:78:fd:34:e8:0a groups: epair media: Ethernet 10Gbase-T (10Gbase-T ) status: active nd6 options=29 Here's the ifconfig from my within my jail: lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff00 groups: lo nd6 options=21 e0b_sambaad: flags=8843 metric 0 mtu 1500 options=8 ether 0e:a4:c4:b5:95:bf hwaddr 02:78:fd:34:e8:0b This is odd. Are you assigning a new MAC address to the epair interfaces somewhere? Both ends of the epair seem to have a new MAC address, and the same one at that. Regards, Kristof ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
On Wed, 14 Oct 2020 at 14:42, Kristof Provost wrote:
> On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote:
> > Hi all!
> > Does anybody know if it's possible to run a vnet jail on a
> > non-dedicated
> > interface? I have the Lucas book on jails. In it he says that for vnet
> > you
> > need to pick a dedicated interface, remove all networking IP
> > configuration
> > and only bring it up. Afterwards you set up jib and whatnot.
> >
> > All works well if I use a dedicated secondary interface (let's call it
> > em1). If I use em0 however I cannot ping the jail.
> >
> > I would like to have a host with that has a single network interface
> > which
> > is used for both normal networking stuff as well as having the vnet
> > jail
> > run on it.
> >
> > Maybe I could create some sort of virtual interface and run vnet on
> > it?
> >
> > Any ideas here? Thanks in advance!
> >
> Look at epair interfaces.
>
> You can put em0 and epair0a in a bridge together and add epair0b to the
> vnet jail.
> That gets the vnet jail connected to your LAN.
>
> Or you can skip the bridge, assign an IP to epair0a and route between
> the jail and your LAN.
>
> Regards,
> Kristof
>
Hi Kristof,
Thanks for your reply!
considering your first idea. I did this, the jail gets created seemingly
fine. However I cannot ping the ip of epair0b (this works when using a
dedicated interface).
Also I cannot reach my gateway from within the jail. This too works when
using a dedicated interface.
Btw I have "sysctl security.jail.allow_raw_sockets=1".
Here is my host ifconfig when putting em0 and epair0a in a bridge:
em0: flags=8943 metric 0
> mtu 1500
>
options=812099
ether 9a:4c:eb:b5:95:bf
inet 172.18.20.145 netmask 0xff00 broadcast 172.18.20.255
media: Ethernet autoselect (1000baseT )
status: active
nd6 options=29
jailether: flags=8843 metric 0 mtu
> 1500
>
options=81209b
ether 56:39:b7:c5:2e:ec
media: Ethernet autoselect (1000baseT )
status: active
nd6 options=29
lo0: flags=8049 metric 0 mtu 16384
>
options=680003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff00
inet 10.43.84.1 netmask 0xff00
groups: lo
nd6 options=21
em0bridge: flags=8843 metric 0 mtu
> 1500
>
ether 02:13:0b:48:53:00
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: e0a_sambaad flags=143
ifmaxaddr 0 port 5 priority 128 path cost 2000
member: em0 flags=143
ifmaxaddr 0 port 1 priority 128 path cost 2
groups: bridge
nd6 options=1
e0a_sambaad: flags=8943
> metric 0 mtu 1500
>
options=8
ether 02:a4:c4:b5:95:bf
hwaddr 02:78:fd:34:e8:0a
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T )
status: active
nd6 options=29
Here's the ifconfig from my within my jail:
lo0: flags=8049 metric 0 mtu 16384
>
options=680003
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff00
groups: lo
nd6 options=21
e0b_sambaad: flags=8843 metric 0
> mtu 1500
>
options=8
ether 0e:a4:c4:b5:95:bf
hwaddr 02:78:fd:34:e8:0b
inet 172.18.20.197 netmask 0xff00 broadcast 172.18.20.255
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T )
status: active
nd6 options=29
The rc.conf in my jail contains the following:
ifconfig_e0b_sambaad="172.18.20.197/24"
>
defaultrouter="172.18.20.1"
And last but not least, here's my jail.conf:
Samba-AD_sambaad{
>
allow.mount.devfs="true";
vnet.interface="e0b_sambaad";
exec.prestart="/usr/local/scripts/jib addm sambaad em0";
exec.poststop="/usr/local/scripts/jib destroy sambaad";
sysvshm="new";
sysvmsg="new";
host.hostname="sambaad";
exec.start="'/etc/rc'";
allow.mount.fdescfs="true";
devfs_ruleset="6";
sysvsem="new";
allow.mount.procfs="true";
enforce_statfs="2";
exec.stop="'/etc/rc.shutdown'";
mount.devfs="true";
path="/cs/systemJheap/Samba-AD/j/sambaad/root";
vnet="new";
allow.raw_sockets="true";
allow.mount="true";
}
Do you have an idea why I cannot ping my jail from within my network when
using a non-dedicated interface?
BR
Arsenij
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: vnet Jail on a non-dedicated network interface
On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote: Hi all! Does anybody know if it's possible to run a vnet jail on a non-dedicated interface? I have the Lucas book on jails. In it he says that for vnet you need to pick a dedicated interface, remove all networking IP configuration and only bring it up. Afterwards you set up jib and whatnot. All works well if I use a dedicated secondary interface (let's call it em1). If I use em0 however I cannot ping the jail. I would like to have a host with that has a single network interface which is used for both normal networking stuff as well as having the vnet jail run on it. Maybe I could create some sort of virtual interface and run vnet on it? Any ideas here? Thanks in advance! Look at epair interfaces. You can put em0 and epair0a in a bridge together and add epair0b to the vnet jail. That gets the vnet jail connected to your LAN. Or you can skip the bridge, assign an IP to epair0a and route between the jail and your LAN. Regards, Kristof ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
vnet Jail on a non-dedicated network interface
Hi all! Does anybody know if it's possible to run a vnet jail on a non-dedicated interface? I have the Lucas book on jails. In it he says that for vnet you need to pick a dedicated interface, remove all networking IP configuration and only bring it up. Afterwards you set up jib and whatnot. All works well if I use a dedicated secondary interface (let's call it em1). If I use em0 however I cannot ping the jail. I would like to have a host with that has a single network interface which is used for both normal networking stuff as well as having the vnet jail run on it. Maybe I could create some sort of virtual interface and run vnet on it? Any ideas here? Thanks in advance! Arsenij ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326 rob2g2 changed: What|Removed |Added CC||spam...@bitbert.com --- Comment #18 from rob2g2 --- same problem here on FreeBSD 12.1 p10 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 249399] The last jail created using VNET has ARP problem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249399 Bjoern A. Zeeb changed: What|Removed |Added Assignee|j...@freebsd.org|n...@freebsd.org CC||b...@freebsd.org --- Comment #4 from Bjoern A. Zeeb --- Re-assign to net@ as the chances are much higher that network people can figure out ARP and things than jail people. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 249399] The last jail created using VNET has ARP problem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249399 fred.h...@yahoo.com changed: What|Removed |Added Status|Closed |Open Resolution|FIXED |--- --- Comment #3 from fred.h...@yahoo.com --- Re-opening this bug report as it has NOT been fixed in BETA3. I closed this ticket earlier as VNET work properly again on one of my test servers, unfortunately VNET still malfunctions on the second. OK, now we have a bug that will not always manifest itself, although the bugs behavior is consistent between reboots and always fails on the same server. The jail/vnet setup of both servers is similar but the number of running jails varies. I would like to work with someone to fix this problem, but do not know what debug information to provide. Symptoms: - (From the HOST) Initially when a list of jails are created all can be pinged. However, after the default ARP timeout, the LAST jail attached to the bridge fails to respond. (From another jail attached to the same VNET) All jails including the last one created are pingable. Everything appears normal. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 249399] The last jail created using VNET has ARP problem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249399 fred.h...@yahoo.com changed: What|Removed |Added Resolution|--- |FIXED Status|New |Closed --- Comment #2 from fred.h...@yahoo.com --- 12.2-BETA3 does not have the problem. Closing the bug report as FIXED. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 249399] The last jail created using VNET has ARP problem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=249399 Mark Linimon changed: What|Removed |Added Assignee|b...@freebsd.org|j...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326 --- Comment #17 from Markus Stoff --- (In reply to Mason Loring Bliss from comment #16) Yes, this will work. It still feels a bit hacky, though... ;-) -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-jail@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-jail To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
[Bug 238326] Kernel crash on jail stop (VIMAGE/VNET)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238326
--- Comment #16 from Mason Loring Bliss ---
Markus Stoff wrote:
> Running 'ifconfig ${epair}b -vnet ${jid}' before removing the jail avoids
> the kernel panic. However, I would prefer to shut my jails down in a
> clean way rather than just pulling the (network) plug.
While it's a little awkward-looking, you can do something like this to make
sure you've cleanly shut down and detached:
exec.prestop = "/usr/sbin/jexec ${name} /bin/sh /etc/rc.shutdown";
exec.prestop += "/sbin/ifconfig epair${ep}b -vnet ${name}";
exec.poststop = "ifconfig $bridge deletem epair${ep}a";
exec.poststop += "ifconfig epair${ep}a destroy";
The notable thing is that exec.prestop and exec.poststop run in system
context, not jail context, so you need the jexec to execute the clean
shutdown - but it works.
--
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
Re: how to make a non-vnet jail local only?
On 05/08/2020 15:17, Ernie Luzar wrote:
> Arthur Chance wrote:
>> On 05/08/2020 02:02, Ernie Luzar wrote:
>>> I have non-vnet jails working that can reach the public internet.
>>> But now I would like to make some local only non-vnet jails that can
>>> only access other local only non-vnet jails. BY local meaning have no
>>> access to the public internet.
>>>
>>> How do I make this happen?
>>>
>>> Thanks for any pointers.
>>
>> Create a second loopback interface (cloned_interfaces="lo1" in
>> /etc/rc.conf or ifconfig lo1 create for manual control) and put the
>> local jails on lo1 without access to any other interface.
>>
>
> I tested this already and it doesn't work.
>
> non-vnet jail with lo99 for the nic and ip address of 10.0.28.5 can
> still reach the public internet.
This surprises me. It's a while since I looked at the network handling
code, but I was under the impression any packet, whatever its address,
on an interface with the LOOPBACK flag set was not routed off the
machine. But see below.
> Also tested a non-vnet jail with re0 for the nic and ip address of
> 127.0.10.10 and it can NOT reach the public internet.
>
> Created a second non-vnet jail with re0 for the nic and ip address of
> 127.0.10.11 and it can NOT reach the public internet.
Also in the network code, any packet with a loopback address as either
source or destination is not routed off machine. This behaviour is
mandated by RFC 1122. It obviously works, no matter what the interface.
However, using a loopback address on a non-loopback interface is logged
as a bad address. You're not supposed to put loopback addresses on non
loopback interfaces.
> But these 2 jails can ping each other.
This does not involve off machine routing. Given that you wanted
non-vnet jails to talk to each other, this would seem to be exactly what
you need.
> So the nic loX has nothing to do with limiting the non-vnet jail to
> local host access only. Based on the above 2 tests it looks like the
> 127.0.0.2 through 127.255.255.254 ip address range is the local host
> controlling factor.
>
> Just to cover all the bases. The host firewall allows the lo0 interface
> to pass without any rules. The lo99 interface has no firewall rules at
> all or any NAT rules for 127.0.0.0/8. 10.0.0.0/8 is the only ip address
> range being NATed.
I suspect (well, guess) the NATing is what is allowing the lo99 packets
to be routed off machine. Try reading the relevant bits of the source
code (in /usr/src/sys/netinet, probably ip_{in,out}put.c and maybe
ip_fastfwd.c), plus the relevant firewall code if you're really interested.
> To see if 127.0.0.0/8 has some special internal limiting factor on it or
> if because the firewall does not NAT 127.0.0.0/8 is the cause of
> non-vnet jails not being able to reach the public internet.
As I said above, this is mandatory behaviour. Even if a packet with a
loopback address was emitted by your machine, your ISP should drop it as
part of their bogon filtering. (Sadly should isn't the same as will.)
> So I created a 3rd non-vnet jail with re0 for the nic and ip address of
> 192.168.10.10 and made no changes to the firewall or NAT. This jail can
> NOT reach the public internet, but can ping the other 2 local only jails
> 127.0.10.10 and 127.0.10.11.
>
> So the conclusion is that loX or 127.0.0.0/8 has nothing to do with
> being the controlling factor between local or public non-vnet jails. The
> real controlling factor is in the jails ip address being NATed or not.
>
> Can this conclusion be disputed?
I'm sure you can find someone on the net to dispute with you if you
really want. :-) Personally I can't be bothered.
The firewall rules also have an effect on routing (duh!), but as you've
not said which firewall you're using I can't address that. However, by
putting all jails you want not to access the outside world on their own
lo interface with their own subnet means you could simply firewall that
interface from talking to the outside which is nice and simple.
The approach I'm using these days is to use IPv6 for almost everything
that's purely in house, and avoid NAT like the horrible hack it is. If
you want a set of jails to only be able to speak to each other and the
host then using a cloned lo interface and pure link local addresses is
nice and simple and doesn't involve a firewall at all.
--
The number of people predicting the demise of Moore's Law doubles
every 18 months.
___
freebsd-jail@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-jail
To unsubscribe, send any mail to "freebsd-jail-unsubscr...@freebsd.org"
