Re: Cannot access internet with virtual switch
Thank you Koshibe-san for your reply. Here is the output of ping, after the steps: $ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes ping: sendmsg: Network is down ping: wrote 8.8.8.8 64 chars, ret=-1 ... So, it seems the ping fails, except, this time there is some output. > > Interestingly, while searching for addlocal, I encountered a > > presentation on switch [1]. On page 13 of that pdf, there is mention of > > the switch sharing the STP code with bridge. Would it be correct to > > assume that there would be no loops if there was STP in the switch? > > Even with a bridge, you'd need to enable STP and set priority values > on the ports for it to work, so you're correct - if there were any > loops, the bridge probably wouldn't have worked either. But you've > also seen that, for switch(4), the STP-related options aren't > available in ifconfig, and as far as I can tell switchd doesn't do > topology/loop detection (and probably won't want to rely on (R)STP to > do so). So, the code might be shared, but is likely not used. I do not know much about the network stack, but I went grepping in the source, and I encountered a bstp_create function call in the switch_clone_create function within sys/net/if_switch.c file. This bstp_create function seems to be defined in sys/net/bridgestp.c [2]. I may be wrong here, but the bridge seems to have some kind of STP, since there is the "rstp" in the ifconfig output. Whether the switch does indeed use STP, and if it does, does it work, is something that I unfortunately cannot determine. For tap0, I ran "tcpdump -nettti tap0" on a normal machine, without the above vether. I saw a large stream of messages. Then I ran it on em0 interface. The "tcpdump -nettti em0" closely matched the output of tap0 interface output. This led me to try to understand tap interfaces. I encountered an article [3] and an image [4]. My hunch is that tap0 is sort of the mirror equivalent of em0. In terms of that image, em0 is the green physical NIC and tap0 is the Virtual Uplink Port. And since they are connected to two ends of the same (imaginary) cable, they will have the same set of messages. This leads me to believe that I should pass all traffic on tap0. Again, this is based on my uneducated guess and searching, so I could be wrong. Passing all traffic on tap0 still does not lead to the ping reply. However, doing a tcpdump on em0 shows an echo request and echo reply whiile tcpdump on tap0 only shows an echo reply. This is irrespective of addlocal vether0. Also, this is irrespective of the pf configuration. Now, why does the echo request not show up on tap0 and what is exactly stopping the echo reply to reach the ping command - I cannot determine. I have done the usual block log, tcpdump -netti pflog0 routine. There is infrequent igmp2 output, but nothing related to icmp. In fact, I have also used a one line "pass" file for pf configuration, still no ping reply. Regards, ab [1] - http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/if_switch.c?rev=1.23&content-type=text/x-cvsweb-markup [2] - http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/bridgestp.c?rev=1.65&content-type=text/x-cvsweb-markup [3] - www.innervoice.in/blogs/2013/12/08/tap-interfaces-linux-bridge/ [4] - https://i2.wp.com/www.innervoice.in/blogs/wp-content/uploads/2013/12/VirtualNetwotk.png -|-|-|-|-|-|-|--
Re: Cannot access internet with virtual switch
On Mon, May 14, 2018 at 1:01 PM, Aham Brahmasmi wrote: > Thank you Koshibe-san for your reply. > >> I've actually held back on that diff since it's a bit insufficient by itself. > > Ok. > >> Actually, you said that you had just em0 on that switch. Can you try >> adding a local port (addlocal instead of add) alongside em0? It will >> be a vether(4) interface that needs to be given em0's current address, >> in its place. > > Should I be doing the following? And if yes, what address should em0 > have? > > $ cat /etc/hostname.vether0 > inet 1.2.3.4 255.255.255.0 > $ cat /etc/hostname.em0 > inet ?.?.?.? 255.255.255.0 > $ doas ifconfig switch0 create > $ doas ifconfig switch0 add em0 > $ doas ifconfig switch0 addlocal vether0 > $ doas ifconfig switch0 up > > Here, 1.2.3.4 is the external public IP address of the machine > originally assigned to em0. em0 shouldn't have an address, and you'll also want to explicitly enable vether0. Otherwise that looks fine. >> > There is a continuous stream of messages when running "switchd -dvv": >> > ... >> >> I can't say what they are without the full output, but you will tend >> to see broadcasts (periodic or otherwise) like your second example >> even on your bridge. From a second look at your earlier logs, it seems >> the 1->1 'loops' are generated by the switch seeing VLAN traffic in >> other parts of the network. > > Ok. Would sharing the full output of "switchd -dvv" help? I wouldn't worry much about it, unless adding the local port doesn't work for you. > Interestingly, while searching for addlocal, I encountered a > presentation on switch [1]. On page 13 of that pdf, there is mention of > the switch sharing the STP code with bridge. Would it be correct to > assume that there would be no loops if there was STP in the switch? Even with a bridge, you'd need to enable STP and set priority values on the ports for it to work, so you're correct - if there were any loops, the bridge probably wouldn't have worked either. But you've also seen that, for switch(4), the STP-related options aren't available in ifconfig, and as far as I can tell switchd doesn't do topology/loop detection (and probably won't want to rely on (R)STP to do so). So, the code might be shared, but is likely not used. > Regards, > ab > [1] - https://www.openbsd.org/papers/bsdcan2016-switchd.pdf > -|-|-|-|-|-|-|--
Re: Cannot access internet with virtual switch
Thank you Koshibe-san for your reply. > I've actually held back on that diff since it's a bit insufficient by itself. Ok. > Actually, you said that you had just em0 on that switch. Can you try > adding a local port (addlocal instead of add) alongside em0? It will > be a vether(4) interface that needs to be given em0's current address, > in its place. Should I be doing the following? And if yes, what address should em0 have? $ cat /etc/hostname.vether0 inet 1.2.3.4 255.255.255.0 $ cat /etc/hostname.em0 inet ?.?.?.? 255.255.255.0 $ doas ifconfig switch0 create $ doas ifconfig switch0 add em0 $ doas ifconfig switch0 addlocal vether0 $ doas ifconfig switch0 up Here, 1.2.3.4 is the external public IP address of the machine originally assigned to em0. > > There is a continuous stream of messages when running "switchd -dvv": > > ... > > I can't say what they are without the full output, but you will tend > to see broadcasts (periodic or otherwise) like your second example > even on your bridge. From a second look at your earlier logs, it seems > the 1->1 'loops' are generated by the switch seeing VLAN traffic in > other parts of the network. Ok. Would sharing the full output of "switchd -dvv" help? Interestingly, while searching for addlocal, I encountered a presentation on switch [1]. On page 13 of that pdf, there is mention of the switch sharing the STP code with bridge. Would it be correct to assume that there would be no loops if there was STP in the switch? Regards, ab [1] - https://www.openbsd.org/papers/bsdcan2016-switchd.pdf -|-|-|-|-|-|-|--
Re: Cannot access internet with virtual switch
> tap0 is a control interface created by switchd to communicate with > switches. It won't carry normal network traffic. (Last bit of noise, and I'm done) Actually, I think I'm wrong here. I'll need to dig a bit more to answer correctly.
Re: Cannot access internet with virtual switch
On Sat, May 12, 2018 at 8:54 PM, Ayaka Koshibe wrote: >> Currently, I am using a bridge. From what I understand from the patch >> and the cvsweb history, that patch is waiting for ok and commit. > > I've actually held back on that diff since it's a bit insufficient by itself. Sorry, I should elaborate. If you plan to try the local port, you may still want the patch in place to prevent the switch from disconnecting.
Re: Cannot access internet with virtual switch
> Currently, I am using a bridge. From what I understand from the patch > and the cvsweb history, that patch is waiting for ok and commit. I've actually held back on that diff since it's a bit insufficient by itself. > This time, the switch does not close. However, I am still unable to > ping 8.8.8.8 with the switch. As usual, I am able to ping 8.8.8.8 > using a bridge. Actually, you said that you had just em0 on that switch. Can you try adding a local port (addlocal instead of add) alongside em0? It will be a vether(4) interface that needs to be given em0's current address, in its place. > There is a continuous stream of messages when running "switchd -dvv": > ... I can't say what they are without the full output, but you will tend to see broadcasts (periodic or otherwise) like your second example even on your bridge. From a second look at your earlier logs, it seems the 1->1 'loops' are generated by the switch seeing VLAN traffic in other parts of the network. > Finally, I have a query - What does the tap0 interface do? I ask this > because currently I do not pass in/out tap0 traffic in pf. This is > because I do not know what tap0 is and what it does. tap0 is a control interface created by switchd to communicate with switches. It won't carry normal network traffic.
Re: Cannot access internet with virtual switch
> > $ cat /etc/hostname.switch0 > > add em0 > > up > > > > Here, em0 is the egress interface connected to the dedicated/bare-metal > > machine provider's network. This provider's network is beyond my > > control. As such, there might be a loop in the provider's network. > > (Sorry, was meaning to respond but got busy) > That's my suspicion, the network you're connected to, rather than your > switch. Say, if you are still trying to use switch(4), there's a > chance that this might help: > > https://marc.info/?l=openbsd-tech&m=152424475112610&w=2 Thank you Koshibe-san for your reply. Please, no sorry. I am grateful for your help. Currently, I am using a bridge. From what I understand from the patch and the cvsweb history, that patch is waiting for ok and commit. This is my first time running a patch, so I may have messed up the steps. Based on some inspiration from [1], I have used the following steps for running the patch on -current: $ doas user mod -G wsrc $ exit $ cd /usr $ cvs -qd anon...@anoncvs.ca.openbsd.org:/cvs checkout -P src $ cd /usr/src $ cvs -q up -Pd -A $ cd /usr/src/usr.sbin/switchd $ vi ofp13.c $ doas user mod -G wobj $ exit $ cd /usr/src/usr.sbin/switchd $ make obj $ make $ doas make install Then the usual switch0, switchd and switchctl steps, as mentioned in my previous mails. This time, the switch does not close. However, I am still unable to ping 8.8.8.8 with the switch. As usual, I am able to ping 8.8.8.8 using a bridge. There is a continuous stream of messages when running "switchd -dvv": ... switch_learn: updated mac 11:11:11:11:11:11 on switch 1 port 1 packet_input: 11:11:11:11:11:11 -> 22:22:22:22:22:22, port 1 -> 1 ofrelay_input_done: connection 1.1: 1528 bytes from switch 1 /dev/switch0 > any: version 1_3 type PACKET_IN length 1528 xid 11362 buffer NO_BUFFER length 1478 reason REASON_NO_MATCH table <0> cookie 0x0 000 match type OXM length 24 (padded to 26) ox match class OPENFLOW_BASIC type IN_PORT hasmask no length 4 1 ox match class OPENFLOW_BASIC type META hasmask no length 8 0 ... switch_learn: updated mac 22:22:22:22:22:22 on switch 1 port 1 packet_input: 22:22:22:22:22:22 -> ff:ff:ff:ff:ff:ff, port 1 -> 4294967295 any > /dev/switch0: version 1_3 type PACKET_OUT length 100 xid 10 buffer NO_BUFFER in_port <1> actions_len 16 action OUTPUT len 16 port FLOOD max_len NO_BUFFER ofrelay_input_done: connection 1.1: 110 bytes from switch 1 ... 1 messages with incrementing xids 11366...> 4294967295 messages with incrementing xids 11...> I have noticed in the ifconfig output that the bridge has "rstp" while there is no "rstp" or "stp" for the switch. It may be possible that the provider has redundant internet routes, and that the bridge with the spanning tree protocol might be able to handle it while the switch may not be. This is just my guess as a layman user and I may be wrong since I do not know whether spanning tree protocol is applicable for the switch. I base this on the absence of stp options in the man pages for switch/switchd/switchctl/switchd.conf. Furthermore, the output of "switchctl show summary" displays a large number of macs with ages within 10 seconds, although the switch has been up for upwards of 1000 seconds. These macs seem to never go above 10 seconds. Two macs in particular are always age 0s. There is only one switch - /dev/switch0. Finally, I have a query - What does the tap0 interface do? I ask this because currently I do not pass in/out tap0 traffic in pf. This is because I do not know what tap0 is and what it does. With a "block log" in pf and "tcpdump -nettti pflog0", I get the following output: tcpdump: WARNING: snaplen raised from 116 to 160 tcpdump: listening on pflog0, link-type PFLOG rule 1/(match) block in on tap0: > 239.255.255.250.1900: udp 94 (DF) [tos 0x38] [ttl 1] ... If I run a ping 8.8.8.8, the ping packets do not show up in the above output. Regards, ab [1] - https://ftp.openbsd.org/pub/OpenBSD/patches/6.3/common/005_httpd.patch.sig -|-|-|-|-|-|-|--
Re: Cannot access internet with virtual switch
> $ cat /etc/hostname.switch0 > add em0 > up > > Here, em0 is the egress interface connected to the dedicated/bare-metal > machine provider's network. This provider's network is beyond my > control. As such, there might be a loop in the provider's network. (Sorry, was meaning to respond but got busy) That's my suspicion, the network you're connected to, rather than your switch. Say, if you are still trying to use switch(4), there's a chance that this might help: https://marc.info/?l=openbsd-tech&m=152424475112610&w=2
Re: Cannot access internet with virtual switch
> Sent: Thursday, April 12, 2018 at 11:24 AM > From: "Ayaka Koshibe" > To: misc@openbsd.org > Subject: Re: Cannot access internet with virtual switch > > On Wed, Apr 11, 2018 at 6:25 AM, Aham Brahmasmi > wrote: > >> Sent: Wednesday, April 11, 2018 at 10:18 AM > >> From: "Ayaka Koshibe" > >> To: misc@openbsd.org > >> Subject: Re: Cannot access internet with virtual switch > >> > >> > This informs us that for a PACKET_OUT with action OUTPUT, it cannot > >> > have its port as ANY. Now, I do not know why for a PACKET_OUT message, > >> > an action OUTPUT cannot have port as ANY. More importantly, I do not > >> > know why the controller seems to be sending the PACKET_OUT with action > >> > OUTPUT and port ANY. > >> > >> A PACKET_OUT is usually a response to some message e.g. a PACKET_IN, > >> so it would probably help to see which message (if any) the switch > >> sent to the controller to receive that PACKET_OUT. > > > > Thank you Koshibe-san for your reply. > > > > From what I understand, the PACKET_IN for that PACKET_OUT seems to be > > the following: > > > > ofrelay_input_done: connection 1.1: 179 bytes from switch 1 > > /dev/switch0 > any: version 1_3 type PACKET_IN length 179 xid 81972 > > buffer NO_BUFFER length 129 reason REASON_NO_MATCH table <0> cookie > > 0x > > match type OXM length 24 (padded to 26) > > ox match class OPENFLOW_BASIC type IN_PORT hasmask no length 4 > > 1 > > ox match class OPENFLOW_BASIC type META hasmask no length 8 > > 0 > > switch_learn: updated mac ac:1f:6b:2e:22:ce on switch 1 port 1 > > packet_input: ac:1f:6b:2e:22:ce -> 00:c8:8b:e2:d6:87, port 1 -> 1 > > This seems to be the right message. It looks like switchd will set the > output port to ANY if it sees a loop (which port 1 -> 1 suggests), > intended for dropping the packet. Do you have loops? Thank you Koshibe-san for your reply. I looked up switch loops in order to understand your insight. Based on the configuration of hostname.switch0, I do not think there is a loop in the virtual switch. However, I may be wrong since I am not good at networks. $ cat /etc/hostname.switch0 add em0 up Here, em0 is the egress interface connected to the dedicated/bare-metal machine provider's network. This provider's network is beyond my control. As such, there might be a loop in the provider's network. Is there a way that I can verify if there is a loop in the network? If it helps in any way, a bridge works on the same network. $ cat /etc/hostname.bridge0 add em0 Regards, ab -|-|-|-|-|-|-|--
Re: Cannot access internet with virtual switch
On Wed, Apr 11, 2018 at 6:25 AM, Aham Brahmasmi wrote: >> Sent: Wednesday, April 11, 2018 at 10:18 AM >> From: "Ayaka Koshibe" >> To: misc@openbsd.org >> Subject: Re: Cannot access internet with virtual switch >> >> > This informs us that for a PACKET_OUT with action OUTPUT, it cannot >> > have its port as ANY. Now, I do not know why for a PACKET_OUT message, >> > an action OUTPUT cannot have port as ANY. More importantly, I do not >> > know why the controller seems to be sending the PACKET_OUT with action >> > OUTPUT and port ANY. >> >> A PACKET_OUT is usually a response to some message e.g. a PACKET_IN, >> so it would probably help to see which message (if any) the switch >> sent to the controller to receive that PACKET_OUT. > > Thank you Koshibe-san for your reply. > > From what I understand, the PACKET_IN for that PACKET_OUT seems to be > the following: > > ofrelay_input_done: connection 1.1: 179 bytes from switch 1 > /dev/switch0 > any: version 1_3 type PACKET_IN length 179 xid 81972 > buffer NO_BUFFER length 129 reason REASON_NO_MATCH table <0> cookie > 0x > match type OXM length 24 (padded to 26) > ox match class OPENFLOW_BASIC type IN_PORT hasmask no length 4 > 1 > ox match class OPENFLOW_BASIC type META hasmask no length 8 > 0 > switch_learn: updated mac ac:1f:6b:2e:22:ce on switch 1 port 1 > packet_input: ac:1f:6b:2e:22:ce -> 00:c8:8b:e2:d6:87, port 1 -> 1 This seems to be the right message. It looks like switchd will set the output port to ANY if it sees a loop (which port 1 -> 1 suggests), intended for dropping the packet. Do you have loops? > I have also attached the complete output of "doas switchd -d". This > is because I do not know whether the above message is the correct > PACKET_OUT message corresponding to the PACKET_IN message. > > Regards, > ab > -|-|-|-|-|-|-|--
Re: Cannot access internet with virtual switch
> Sent: Wednesday, April 11, 2018 at 10:18 AM > From: "Ayaka Koshibe" > To: misc@openbsd.org > Subject: Re: Cannot access internet with virtual switch > > > This informs us that for a PACKET_OUT with action OUTPUT, it cannot > > have its port as ANY. Now, I do not know why for a PACKET_OUT message, > > an action OUTPUT cannot have port as ANY. More importantly, I do not > > know why the controller seems to be sending the PACKET_OUT with action > > OUTPUT and port ANY. > > A PACKET_OUT is usually a response to some message e.g. a PACKET_IN, > so it would probably help to see which message (if any) the switch > sent to the controller to receive that PACKET_OUT. Thank you Koshibe-san for your reply. >From what I understand, the PACKET_IN for that PACKET_OUT seems to be the following: ofrelay_input_done: connection 1.1: 179 bytes from switch 1 /dev/switch0 > any: version 1_3 type PACKET_IN length 179 xid 81972 buffer NO_BUFFER length 129 reason REASON_NO_MATCH table <0> cookie 0x match type OXM length 24 (padded to 26) ox match class OPENFLOW_BASIC type IN_PORT hasmask no length 4 1 ox match class OPENFLOW_BASIC type META hasmask no length 8 0 switch_learn: updated mac ac:1f:6b:2e:22:ce on switch 1 port 1 packet_input: ac:1f:6b:2e:22:ce -> 00:c8:8b:e2:d6:87, port 1 -> 1 I have also attached the complete output of "doas switchd -d". This is because I do not know whether the above message is the correct PACKET_OUT message corresponding to the PACKET_IN message. Regards, ab -|-|-|-|-|-|-|-- listen on 0.0.0.0:6633 ofrelay_attach: new connection 1.1 ofp_open: new connection 1.1 from switch 0 any > /dev/switch0: version 1_3 type HELLO length 16 xid 0 version bitmap: version 1_0 version 1_3 0408 00014025 ofrelay_input_done: connection 1.1: 8 bytes from switch 0 0408 00014025 /dev/switch0 > any: version 1_3 type HELLO length 8 xid 81957 any > /dev/switch0: version 1_3 type FEATURES_REQUEST length 8 xid 1 04060020 0001 55330647 2d30f2ae fe00 000f ofrelay_input_done: connection 1.1: 32 bytes from switch 1 04060020 0001 55330647 2d30f2ae fe00 000f /dev/switch0 > any: version 1_3 type FEATURES_REPLY length 32 xid 1 datapath_id 0x553306472d30f2ae nbuffers 0 ntables 254 aux_id 0 capabilities 0x0f any > /dev/switch0: version 1_3 type MULTIPART_REQUEST length 16 xid 2 type TABLE_FEATURES flags empty table properties request any > /dev/switch0: version 1_3 type SET_CONFIG length 12 xid 3 flags miss_send_len NO_BUFFER 04130418 0002 000c 0408 2710 0018 00010004 00020004 00030004 00040004 00050004 00010018 00010004 00020004 00030004 00040004 00050004 00060018 00120004 00110004 00190004 00160004 0004 00070018 00120004 00110004 00190004 00160004 0004 00040018 00120004 00110004 00190004 00160004 0004 00050018 00120004 00110004 00190004 00160004 0004 0008008c 8004 8508 8706 8906 8a02 8d02 8e01 80001001 80001201 80001401 80001704 80001904 80001a02 80001c02 80001e02 80002002 80002202 80002402 80002601 80002801 80002a02 80002d04 80002f04 80003106 80003306 80003510 80003710 80003904 80003a01 80003c01 80003e10 80004006 80004206 80004c08 000a0088 8408 8606 8806 8a02 8c02 8e01 80001001 80001201 80001401 80001604 80001804 80001a02 80001c02 80001e02 80002002 80002202 80002402 80002601 80002801 80002a02 80002c04 80002e04 80003006 80003206 80003410 80003610 80003804 80003a01 80003c01 80003e10 80004006 80004206 80004c08 000c0084 8606 8806 8a02 8c02 8e01 80001001 80001201 80001401 80001604 80001804 80001a02 80001c02 80001e02 80002002 80002202 80002402 80002601 80002801 80002a02 80002c04 80002e04 80003006 80003206 80003410 80003610 80003804 80003a01 80003c01 80003e10 80004006 80004206 80004c08 000d0084 8606 8806 8a02 8c02 8e01 80001001 80001201 80001401 80001604 80001804 80001a02 80001c02 80001e02 80002002 80002202 80002402 80002601 80002801 80002a02 80002c04 80002e04 80003006 80003206 80003410 80003610 80003804 80003a01 80003c01 80003e10 80004006 80004206 80004c08 000e0084 8606 8806 8a02 8c02 8e01 80001001 80001201 80001401 80001604 80001804 80001a02 80001c02 80001e02 80002002 80002202 80002402 80002601 80002801 80002a02 80002c04 80002e04 80003006 80003206 80003410 80003610 80003804 80003a01 80003c01 80003e10 80004006 80004206 80004c08 000f0084 8606 8806 8a02 8c02 8e01 80001001 80001201 80001401 80001604 80001804 80001a02 80001c02 80001e02 80002002 80002202 800
Re: Cannot access internet with virtual switch
> This informs us that for a PACKET_OUT with action OUTPUT, it cannot > have its port as ANY. Now, I do not know why for a PACKET_OUT message, > an action OUTPUT cannot have port as ANY. More importantly, I do not > know why the controller seems to be sending the PACKET_OUT with action > OUTPUT and port ANY. A PACKET_OUT is usually a response to some message e.g. a PACKET_IN, so it would probably help to see which message (if any) the switch sent to the controller to receive that PACKET_OUT.
Re: Cannot access internet with virtual switch
> Sent: Monday, April 09, 2018 at 6:50 PM > From: "Aham Brahmasmi" > To: misc@openbsd.org > Subject: Re: Cannot access internet with virtual switch > > > Sent: Saturday, April 07, 2018 at 5:02 AM > > From: "Ayaka Koshibe" > > To: "Aham Brahmasmi" > > Cc: misc@openbsd.org > > Subject: Re: Cannot access internet with virtual switch > > > > On Fri, Apr 6, 2018 at 4:40 PM, Aham Brahmasmi > > wrote: > > > Hello misc, > > > > > > Problem > > > A physical server with a switch (add em0 up) cannot access the internet. > > > However, the same host with a bridge (add em0 up) can access the > > > internet. > > > > > > Steps > > > $ ifconfig > > > em0: flags=8843 mtu 1500 > > > lladdr 22:22:22:22:22:22 > > > index 1 priority 0 llprio 3 > > > groups: egress > > > media: Ethernet autoselect (1000baseT full-duplex,master) > > > status: active > > > inet 20.20.20.20 netmask 0xff00 broadcast 20.20.20.255 > > > ... > > > $ doas route -n show > > > Routing tables > > > > > > Internet: > > > Destination GatewayFlags Refs Use Mtu Prio Iface > > > default 20.20.20.1 UGS0 1XXX - 8 em0 > > > 224/4 127.0.0.1 URS00 32768 8 lo0 > > > 127/8 127.0.0.1 UGRS 00 32768 8 lo0 > > > 127.0.0.1 127.0.0.1 UHhl 1X 32768 1 lo0 > > > 20.20.20/24 20.20.20.20UCn1 9XX - 4 em0 > > > 20.20.20.1 33:33:33:33:33:33 UHLch 1 1XXX - 3 em0 > > > 20.20.20.20 44:44:44:44:44:44 UHLl 0X - 1 em0 > > > 20.20.20.25520.20.20.20UHb00 - 1 em0 > > > $ ping 8.8.8.8 > > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=61 time=x.xxx ms > > > ... > > > $ doas ifconfig switch0 create > > > $ doas ifconfig switch0 add em0 > > > $ doas ifconfig switch0 up > > > $ ping 8.8.8.8 > > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > > ^C > > > --- 8.8.8.8 ping statistics --- > > > 31 packets transmitted, 0 packets received, 100.0% packet loss > > > > Hi, > > > > Seems you haven't started switchd(8), or connected your switch to it > > -- it shouldn't forward traffic until you do so. > > > Hi Koshibe-san, > > Thank you for your reply. > > I have started switchd and connected to it. However, I still cannot > ping 8.8.8.8. Starting switchd in debug mode results in output which > broadly says error and closes the switch. > > Steps (after the above switch0 up) > $ cat /etc/switchd.conf > listen on 0.0.0.0 tls port 6633 > $ doas switchd -d > listen on 0.0.0.0 6633 > > (On another session) > $ switchctl connect /dev/switch0 > > (Back to main session) > ofrelay_input_done: ... > /dev/switch0 > any: ... > switch_learn: ... > packet_input: ... > any > /dev/switch0: ... > (above block repeated multiple times) > ... > ofrelay_input_done: connection 1.1: 76 bytes from switch 1 > 0401004c 0013 00020004 040d00a9 0013 0001 0010 > 0010 00c88be2 d687ac1f 6b2e22ce > 8100026f 08004500 006f42d2 > /dev/switch0 > any: version 1_3 type ERROR length 76 xid 19 > error type BAD_ACTION code 4 > ofp13_input: message not supported: ERROR > ofrelay_close: connection 1.1 closed > switch_remove: switch 1 removed. > > (Another session) > $ tail -10 /var/log/messages > Apr 9 XX:XX:XX MachineName /bsd: arp: attempt to add entry for GATEWAY_IP > on em0 by XX:XX:XX:XX:XX:XX on tap0 > (above message repeated infrequently) > > If it helps in any way, this machine is a dedicated/bare-metal machine > on a large dedicated/bare-metal machine provider's network. The em0 > interface is in the egress group, has a public IP and is connected to > the internet via the provider's network equipment. > > The end goal in using the switch is to enable multiple OpenBSD VM's with > with non-contiguous public IPs to be connected to the Internet as real > hosts. In https://www.openbsd.org/faq/faq6.html#VMMnet, this is the > Option 4, except using a switch instead of a bridge and public IPs > on the host network. > > Regards, > ab > -|-|-|-|-|---
Re: Cannot access internet with virtual switch
> Sent: Saturday, April 07, 2018 at 5:02 AM > From: "Ayaka Koshibe" > To: "Aham Brahmasmi" > Cc: misc@openbsd.org > Subject: Re: Cannot access internet with virtual switch > > On Fri, Apr 6, 2018 at 4:40 PM, Aham Brahmasmi wrote: > > Hello misc, > > > > Problem > > A physical server with a switch (add em0 up) cannot access the internet. > > However, the same host with a bridge (add em0 up) can access the > > internet. > > > > Steps > > $ ifconfig > > em0: flags=8843 mtu 1500 > > lladdr 22:22:22:22:22:22 > > index 1 priority 0 llprio 3 > > groups: egress > > media: Ethernet autoselect (1000baseT full-duplex,master) > > status: active > > inet 20.20.20.20 netmask 0xff00 broadcast 20.20.20.255 > > ... > > $ doas route -n show > > Routing tables > > > > Internet: > > Destination GatewayFlags Refs Use Mtu Prio Iface > > default 20.20.20.1 UGS0 1XXX - 8 em0 > > 224/4 127.0.0.1 URS00 32768 8 lo0 > > 127/8 127.0.0.1 UGRS 00 32768 8 lo0 > > 127.0.0.1 127.0.0.1 UHhl 1X 32768 1 lo0 > > 20.20.20/24 20.20.20.20UCn1 9XX - 4 em0 > > 20.20.20.1 33:33:33:33:33:33 UHLch 1 1XXX - 3 em0 > > 20.20.20.20 44:44:44:44:44:44 UHLl 0X - 1 em0 > > 20.20.20.25520.20.20.20UHb00 - 1 em0 > > $ ping 8.8.8.8 > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=61 time=x.xxx ms > > ... > > $ doas ifconfig switch0 create > > $ doas ifconfig switch0 add em0 > > $ doas ifconfig switch0 up > > $ ping 8.8.8.8 > > PING 8.8.8.8 (8.8.8.8): 56 data bytes > > ^C > > --- 8.8.8.8 ping statistics --- > > 31 packets transmitted, 0 packets received, 100.0% packet loss > > Hi, > > Seems you haven't started switchd(8), or connected your switch to it > -- it shouldn't forward traffic until you do so. Hi Koshibe-san, Thank you for your reply. I have started switchd and connected to it. However, I still cannot ping 8.8.8.8. Starting switchd in debug mode results in output which broadly says error and closes the switch. Steps (after the above switch0 up) $ cat /etc/switchd.conf listen on 0.0.0.0 tls port 6633 $ doas switchd -d listen on 0.0.0.0 6633 (On another session) $ switchctl connect /dev/switch0 (Back to main session) ofrelay_input_done: ... /dev/switch0 > any: ... switch_learn: ... packet_input: ... any > /dev/switch0: ... (above block repeated multiple times) ... ofrelay_input_done: connection 1.1: 76 bytes from switch 1 0401004c 0013 00020004 040d00a9 0013 0001 0010 0010 00c88be2 d687ac1f 6b2e22ce 8100026f 08004500 006f42d2 /dev/switch0 > any: version 1_3 type ERROR length 76 xid 19 error type BAD_ACTION code 4 ofp13_input: message not supported: ERROR ofrelay_close: connection 1.1 closed switch_remove: switch 1 removed. (Another session) $ tail -10 /var/log/messages Apr 9 XX:XX:XX MachineName /bsd: arp: attempt to add entry for GATEWAY_IP on em0 by XX:XX:XX:XX:XX:XX on tap0 (above message repeated infrequently) If it helps in any way, this machine is a dedicated/bare-metal machine on a large dedicated/bare-metal machine provider's network. The em0 interface is in the egress group, has a public IP and is connected to the internet via the provider's network equipment. The end goal in using the switch is to enable multiple OpenBSD VM's with with non-contiguous public IPs to be connected to the Internet as real hosts. In https://www.openbsd.org/faq/faq6.html#VMMnet, this is the Option 4, except using a switch instead of a bridge and public IPs on the host network. Regards, ab -|-|-|-|-|-|-|-- > > > $ ifconfig > > em0: flags=8b43 > > mtu 1500 > > lladdr 22:22:22:22:22:22 > > index 1 priority 0 llprio 3 > > groups: egress > > media: Ethernet autoselect (1000baseT full-duplex,master) > > status: active > > inet 20.20.20.20 netmask 0xff00 broadcast 20.20.20.255 > > switch0: flags=41 > > index 6 llprio 3 > > groups: switch > > datapath xx maxflow 1 maxgroup 1000 > > em0 flags=0<> > > port 1 ifpriority 0 ifcost 0 > > ... > > $ doas route -n show > > Routing tables > &
Re: Cannot access internet with virtual switch
On Fri, Apr 6, 2018 at 4:40 PM, Aham Brahmasmi wrote: > Hello misc, > > Problem > A physical server with a switch (add em0 up) cannot access the internet. > However, the same host with a bridge (add em0 up) can access the > internet. > > Steps > $ ifconfig > em0: flags=8843 mtu 1500 > lladdr 22:22:22:22:22:22 > index 1 priority 0 llprio 3 > groups: egress > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 20.20.20.20 netmask 0xff00 broadcast 20.20.20.255 > ... > $ doas route -n show > Routing tables > > Internet: > Destination GatewayFlags Refs Use Mtu Prio Iface > default 20.20.20.1 UGS0 1XXX - 8 em0 > 224/4 127.0.0.1 URS00 32768 8 lo0 > 127/8 127.0.0.1 UGRS 00 32768 8 lo0 > 127.0.0.1 127.0.0.1 UHhl 1X 32768 1 lo0 > 20.20.20/24 20.20.20.20UCn1 9XX - 4 em0 > 20.20.20.1 33:33:33:33:33:33 UHLch 1 1XXX - 3 em0 > 20.20.20.20 44:44:44:44:44:44 UHLl 0X - 1 em0 > 20.20.20.25520.20.20.20UHb00 - 1 em0 > $ ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=61 time=x.xxx ms > ... > $ doas ifconfig switch0 create > $ doas ifconfig switch0 add em0 > $ doas ifconfig switch0 up > $ ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > ^C > --- 8.8.8.8 ping statistics --- > 31 packets transmitted, 0 packets received, 100.0% packet loss Hi, Seems you haven't started switchd(8), or connected your switch to it -- it shouldn't forward traffic until you do so. > $ ifconfig > em0: flags=8b43 mtu > 1500 > lladdr 22:22:22:22:22:22 > index 1 priority 0 llprio 3 > groups: egress > media: Ethernet autoselect (1000baseT full-duplex,master) > status: active > inet 20.20.20.20 netmask 0xff00 broadcast 20.20.20.255 > switch0: flags=41 > index 6 llprio 3 > groups: switch > datapath xx maxflow 1 maxgroup 1000 > em0 flags=0<> > port 1 ifpriority 0 ifcost 0 > ... > $ doas route -n show > Routing tables > > Internet: > Destination GatewayFlags Refs Use Mtu Prio Iface > default 20.20.20.1 UGS0 1XXX - 8 em0 > 224/4 127.0.0.1 URS00 32768 8 lo0 > 127/8 127.0.0.1 UGRS 00 32768 8 lo0 > 127.0.0.1 127.0.0.1 UHhl 1X 32768 1 lo0 > 20.20.20/24 20.20.20.20UCn1 9XX - 4 em0 > 20.20.20.1 33:33:33:33:33:33 UHLch 1 1XXX - 3 em0 > 20.20.20.20 44:44:44:44:44:44 UHLl 0X - 1 em0 > 20.20.20.25520.20.20.20UHb00 - 1 em0 > $ doas ifconfig switch0 destroy > $ ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=61 time=x.xxx ms > > Repeating the above steps with bridge0 does let the ping pass through > after the bridge is brought up. The only delta between the switch and > bridge output is in the ifconfig. > $ ifconfig > bridge0: flags=41 > index 8 llprio 3 > groups: bridge > priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rtsp > em0 flags=3 > port 1 ifpriority 0 ifcost 0 > ... > > I have run "doas route -n monitor" in a separate session while doing > this. However, I cannot comprehend the output. pf is not involved - > running tcpdump -nettti pflog0 with the catchall "block log" produces > the normal output of blocked packets with the bridge. However, it stops > producing the normal output of blocked packets with the switch. Once the > switch is destroyed, it is back to normal blocked packets output. > > What am I doing wrong/missing? The only thing that stands out to me is > the em0 flags=0<> line in the ifconfig for the switch. And I do not know > what to make of it. > > Regards, > ab > -|-|-|-|-|-|-|-- >