Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi Andrew, On Tue, Feb 11, 2020 at 11:47:04AM +0100, goldsi...@gmx.de wrote: > Am 11.02.2020 um 10:20 schrieb Andrew Pullin: > > [..] > > > > The part that I am stuck on presently is that Host2 has no knowledge of > > how to route packets from PPP out to the WAN, say, 8.8.8.8. > > Between ip4_forward, IP_FORWARD, ip4_route_src, LWIP_HOOK_IP4_ROUTE, > > LWIP_HOOK_IP4_ROUTE_SRC, and the ESP32 port modifications, I am not > > quite sure where to make the right intervention. It is ... a lot to take in. > > > > As far as I understand the standard usage of the stack for a single > > netif, outgoing packets elide the entire forwarding/routing system > > because they are specified to be transmitted on a particular netif? Is > > that right? > > > > Fundamentally, I only have eth0 and pp1 (to use the lwip netif naming > > system) and anything not specifically with IP dest = eth0 or pp1 should > > be sent on towards Host2's gateway (after NAT source address IP rewriting). > > Is that logically complete, and if so, where should it be implemented? > > I would think you'd just set the ppp netif as default netif (so it is > just used for all outgoing trafic where no better route is found) and > keep eth0 without a gateway (which should ensure only matching subnet > packets are transmitted on this netif). Yes, but the default netif (default route) must be set on eth0, not pp1 :) Sylvain signature.asc Description: Digital signature ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Am 11.02.2020 um 10:20 schrieb Andrew Pullin: > [..] > > The part that I am stuck on presently is that Host2 has no knowledge of > how to route packets from PPP out to the WAN, say, 8.8.8.8. > Between ip4_forward, IP_FORWARD, ip4_route_src, LWIP_HOOK_IP4_ROUTE, > LWIP_HOOK_IP4_ROUTE_SRC, and the ESP32 port modifications, I am not > quite sure where to make the right intervention. It is ... a lot to take in. > > As far as I understand the standard usage of the stack for a single > netif, outgoing packets elide the entire forwarding/routing system > because they are specified to be transmitted on a particular netif? Is > that right? > > Fundamentally, I only have eth0 and pp1 (to use the lwip netif naming > system) and anything not specifically with IP dest = eth0 or pp1 should > be sent on towards Host2's gateway (after NAT source address IP rewriting). > Is that logically complete, and if so, where should it be implemented? I would think you'd just set the ppp netif as default netif (so it is just used for all outgoing trafic where no better route is found) and keep eth0 without a gateway (which should ensure only matching subnet packets are transmitted on this netif). Regards, Simon signature.asc Description: OpenPGP digital signature ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
On 1/31/2020 4:48 PM, Sylvain Rochet wrote: Or how host3/C could be configured for routing a subnet (now 11.0.0.0/32) that is essentially "private" between host2/B and host1/A. That is the "192.168.0.0/24 via 172.16.1.1" route in your example, or "11.0.0.0/32 via 10.0.0.016" in my case. It seems like what I actually do want is NAT capability on host2, so that the host1<->host2 connection would be abstracted away. But it looks like there are only a few few-years-old implementations of NAT for lwip. Uh oh. I would really like to avoid trying to integrate that in from an old branch of lwip. Any idea if I can get this to work without needing to try to bring up NAT? It it were me and if I don't need Host2 to reach "outside" and Host1 software can't be changed, I would just write a big hack to forward relevant packets (e.g. no DHCP, no ARP, IP) to the PPP link, pushing the DHCP assigned address to the PPP peer Host1 so NAT is not necessary at all and nothing have to be done for Host1 to send packets to "outside". And if it were me and if I need Host2 to reach "outside" and Host1 software can't be changed, I would write a quick'n'dirty ARP Proxy support, probably at the Ethernet low level driver. Note, in this case, it is normal for the PPP link to be within the Host2/Host3 subnet, it is actually mandatory. But doing so won't work if you need DHCP support because you need DHCP server to offer two addresses (one for Host2, one for Host1), so you need two MAC addresses and two DHCP clients running. It's way more difficult than just hacking the forwarding and sharing the same address. But, above all, if it were me, I would just write a TCP or UDP proxy on Host2. Host1 would then only establish TCP or UDP sessions to Host2 with a home made proxy protocol (basically packets with two destination IP fields), and Host2 would create the real session to the outside world. Host1 - Host2 Host3 -- Internet 192.168.0.1 192.168.0.2 10.0.200.113 192.0.2.1 <---TCP1> <---TCP2---> TCP1: src: 192.168.0.1 dst: 192.168.0.2 real-dst: 192.0.2.1 (in TCP payload, e.g. at the beginning of the TCP session) TCP2 (created on the fly using payload hidden real-dst): src: 10.0.200.113 dst: 192.0.2.1 Wow, thanks for all the good ideas. I can mostly change the code on both Host1 and Host2 here, but I am really trying to do this with minimal changes to the upper layers, especially on Host1. As described: end up with a transparently "working" internet connection there. The ARP proxy case is fascinating. Since Host1 used to be a fully internet connected device, it has everything it would need, i.e. a DHCP client, DNS client, etc. If I could make Host2 appear to represent two devices with two valid (IEEE registered) MAC addresses and the network operator was OK with two DHCP clients existing in one physical box, that seems like a good solution. But: modifying the low-level ethernet driver seem daunting. The TCP proxy case also seems quite interesting, since it should be all app layer code, and only need a TCP listener and TCP client. Alas, we also need UDP for DNS. For TCP, it would be easy enough to start the stream with the target IP:port and a verifier, but for UDP, the datragram would have to be dissected, deal with the IP pseudo-header, etc. But here's the good news: I found a patch to add NAPT functionality to lwip, I am sure it has been linked to somewhere in this mailing list before: https://gist.github.com/NeoCat/da3c141813980edaa256ad351cab3a2c Seems pretty interesting, and it seems like it is almost working, too. This would be a great solution, since then I can run a private static IP subnet between Host1 <-> Host2, and to the upstream router, it should just look like a single device Host2 (other than NAT detection via TTL or other methods). The part that I am stuck on presently is that Host2 has no knowledge of how to route packets from PPP out to the WAN, say, 8.8.8.8. Between ip4_forward, IP_FORWARD, ip4_route_src, LWIP_HOOK_IP4_ROUTE, LWIP_HOOK_IP4_ROUTE_SRC, and the ESP32 port modifications, I am not quite sure where to make the right intervention. It is ... a lot to take in. As far as I understand the standard usage of the stack for a single netif, outgoing packets elide the entire forwarding/routing system because they are specified to be transmitted on a particular netif? Is that right? Fundamentally, I only have eth0 and pp1 (to use the lwip netif naming system) and anything not specifically with IP dest = eth0 or pp1 should be sent on towards Host2's gateway (after NAT source address IP rewriting). Is that logically complete, and if so, where should it be implemented? Thanks, Andrew Pullin ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi Simon, On Thu, Jan 30, 2020 at 10:11:23PM +0100, goldsi...@gmx.de wrote: > > Well, yes, my bad. The question wasn't phrased well. The thing I don't > understand (because I'm not using it) probably all boils down to using > lwIP as a PPP 'proxy' to remote networks without doing NAT at the same > time (which we don't support in mainline). That's not standard routing... Or the contrary, that's standard IP routing, it's just totally useless with residential ISPs because most don't allow customers to add route entries on their gateway so it's just a flat subnet with no routing capability. Business ISPs usually offer the option, it's very common for business customers to have a delegated subnet that's properly routable. Sylvain signature.asc Description: Digital signature ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi Andrew, On Fri, Jan 31, 2020 at 12:11:56AM -0800, Andrew Pullin wrote: > On 1/30/20 11:42 AM, Sylvain Rochet wrote: > > > > 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that > > shouldn't be a problem here, this is usually a bad practice and it > > should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't > > know there is a hole in the subnet, since there are only 3 hosts here, > > which is an expection rather than the rule, it is safe by luck. > > This seems relatively easy to fix: I can just change the PPP-negotiated IP's > to 11.0.0.0/32, so the host1 and host2 devices are on a different subnet on > the PPP side. 11.0.0.0/32 is outside RFC1918 private subnets, using it sounds like a bad idea. > As for the /32, I am unsure if I need to change that. Do I? The netmask is only meaningful for layer 2 with a broadcast domain (e.g. Ethernet), PPP is point to point, there is only one peer, netmask as no meaning. (To be fair that's not entirely true, it use to be used for PPP to provide more than one IPv4 addresses to customers, but I don't think it's still used in the wild, and it's still not a broadcast domain, it's just a pool of addresses.) > Darn. I saw some code for "UNUSED - PROXY ARP", but it was all removed with > #if 0 blocks. You are lost in the wrong place ;-), ARP Proxy support have to live in the Ethernet ARP driver, and that support does not exist. What you saw is just PPP support to add a Proxy ARP rule into the Ethernet driver (basically no more than one function call :-), it is not the ARP Proxy support. > Here is where I am starting to get worried about how I am possibly going > make this solution work. > I am trying to, in general, get my host1 device connected to the outside > world via host2's connection to host3. > > The motivation for doing this via IP stacks is because my host1 (PPP client) > already has a ton of code implemented using TCP/IP; host1 is a piece of > hardware that used to have its own MAC/PHY, but is now being repurposed. > Being able to re-use all of that code without having to remap every function > at a high-level to some other mechanism (say, AT commands, also needing > host2 implementations) will be a huge boon. > > My knowledge here is clearly limited, but is there a general or dynamic way > to set up the routes I would need here? > > Since I want to be able to attach host2 to any existing network and have > this system work, I think this is what I would need. > > I do not know how host2/B could pass the information needed to host1/A to > configure the next hop route, or "172.16.1.0/24 via 192.168.0.2" in your > example. This seems like it would require a mechanism for host2/B to > disclose to host1/A what IP address it gets (say, via DHCP) on it's eth0 > side. It exists… it's called OSPF… but… erm… don't expect that to be available behind a set-top box or whatever "gateway/router" used by ISP. I'm running OSPF at home but I'm not the regular ISP customer :-) > Or how host3/C could be configured for routing a subnet (now > 11.0.0.0/32) that is essentially "private" between host2/B and > host1/A. That is the "192.168.0.0/24 via 172.16.1.1" route in your > example, or "11.0.0.0/32 via 10.0.0.016" in my case. > > It seems like what I actually do want is NAT capability on host2, so > that the host1<->host2 connection would be abstracted away. But it > looks like there are only a few few-years-old implementations of NAT > for lwip. Uh oh. I would really like to avoid trying to integrate that > in from an old branch of lwip. > > Any idea if I can get this to work without needing to try to bring up > NAT? It it were me and if I don't need Host2 to reach "outside" and Host1 software can't be changed, I would just write a big hack to forward relevant packets (e.g. no DHCP, no ARP, IP) to the PPP link, pushing the DHCP assigned address to the PPP peer Host1 so NAT is not necessary at all and nothing have to be done for Host1 to send packets to "outside". And if it were me and if I need Host2 to reach "outside" and Host1 software can't be changed, I would write a quick'n'dirty ARP Proxy support, probably at the Ethernet low level driver. Note, in this case, it is normal for the PPP link to be within the Host2/Host3 subnet, it is actually mandatory. But doing so won't work if you need DHCP support because you need DHCP server to offer two addresses (one for Host2, one for Host1), so you need two MAC addresses and two DHCP clients running. It's way more difficult than just hacking the forwarding and sharing the same address. But, above all, if it were me, I would just write a TCP or UDP proxy on Host2. Host1 would then only establish TCP or UDP sessions to Host2 with a home made proxy protocol (basically packets with two destination IP fields), and Host2 would create the real session to the outside world. Host1 - Host2 Host3 -- Internet 192.168.0.1
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
On 1/30/20 11:42 AM, Sylvain Rochet wrote: There are no ARP in PPP, but whatever, this configuration is correct. Roger that. At the very least, ARP is enabled on the NetX equivalent of a netif on the PPP-only host. But it must be inactive and not accomplishing anything. 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that shouldn't be a problem here, this is usually a bad practice and it should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't know there is a hole in the subnet, since there are only 3 hosts here, which is an expection rather than the rule, it is safe by luck. This seems relatively easy to fix: I can just change the PPP-negotiated IP's to 11.0.0.0/32, so the host1 and host2 devices are on a different subnet on the PPP side. As for the /32, I am unsure if I need to change that. Do I? WiFi AP's: - AP only (no routing) - Connected to Host3 by Ethernet - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control association) Host3: - Router (pfSense) - DHCP server runs here - Routes to WAN, does NAT, eth0 <-> eth1 - default route: set by DHCP, toward eth1 - no other route (as far as I know) - Intranet WiFi AP's are connected via switch to eth0 - Intranet Ethernet devices are connected via switch to eth0 - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95 - eth1 : IP from ISP, facing public IPv4 internet, MAC 00:e0:67:18:54:94 Here is the real problem, you need a route on Host3 telling that 10.0.0.2/32 is behind 10.0.200.136 (or 10.0.200.113). You can also add a route for 10.0.0.1/32 but it's just a bonus. Without this route, Host3 don't know how to route the packet and will try to find an host on the 10.0.0.0/16 subnet using ARP. Then it's just luck, and depends on what Host3 and Host2 are doing: - If the packet is sent in a broadcast Ethernet frame by Host3 it might works if Host2 catches the packet and forwards it to Host1. - If the ARP request gets answered by Host2 when Host3 send their ARP request to find who is 10.0.0.2 on the 10.0.0.0/16 subnet, then it also works. This is known as ARP Proxy, lwIP does not support that. It's just luck, using lazy features of IP stack to make it works even if the configuration is broken should be avoided :-) Darn. I saw some code for "UNUSED - PROXY ARP", but it was all removed with #if 0 blocks. Layout: Host1 <-> Host2 <-> Host3 <-> public internet PPPoS Ethernet or Host1 <-> Host2 <> WiFi AP <--> Host3 <---> public internet PPPoS WiFi Ethernet For the case of using the WiFi connection, there are separate WiFi AP's in the way. Other than that, the two cases are host2 connecting to the local subnet via wlan0 or connecting via eth0. I guess the only reason it works with WiFi and not with Ethernet is because WiFi is intrinsically broadcast-based while Ethernet is switched (well, I hope so, unless you live in '90s :p). This is the "works because Host2 catches the broadcast packet" case I've said before. Wow. What a situation ... good to know there is a rational explanation as to why it worked with seemingly no network setup difference. I am not manually adding any static routes in either Host1 or Host2. I was worried that Host2 would not be able to accomplish the "connection sharing" until I found the IP_FORWARDING option. That initially worked with Host2 using WiFi without any manually added static routes. This is not "connection sharing", this is IP routing, not NAT !, "connection sharing" is usually used for hosts that do SNAT on their Interface facing Internet. IP routing needs a complete routing table on all hosts telling how to reach (i.e. which next-hop to use) every host of the whole network. Example with 2 subnets and 3 Hosts: A-eth0 - eth0-B-eth1 - eth0-C 192.168.0.1/24192.168.0.2/24 172.16.1.1/24 172.16.1.2/24 Host A routing table: 192.168.0.0/24 dev eth0 172.16.1.0/24 via 192.168.0.2 Host B routing table: 192.168.0.0/24 dev eth0 172.16.1.0/24 dev eth1 Host C routing table: 192.168.0.0/24 via 172.16.1.1 172.16.1.0/24 dev eth0 dev = directly connected via = next-hop, gateway if you prefer What you could see is that each host have a similar routing table, just the next-hop changes. Default route is just a way to reduce the routing table size by removing all entries that have an identical next-hop. Here is where I am starting to get worried about how I am possibly going make this solution work. I am trying to, in general, get my host1 device connected to the outside world via host2's connection to host3. The motivation for doing this via IP stacks is because my host1 (PPP client) already has a ton of code implemented using TCP/IP; host1 is a piece of hardware that
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Am 30.01.2020 um 22:02 schrieb Sylvain Rochet: Hi Simon, On Thu, Jan 30, 2020 at 09:31:11PM +0100, goldsi...@gmx.de wrote: Am 30.01.2020 um 20:42 schrieb Sylvain Rochet: On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote: Host2: - Micontroller device, ESP32 - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used mutually exclusively - Running lwIP - WiFi and Ethernet are independently verified to work to reach WAN from this host - PPPoS server - ppp0 : 10.0.0.1/32 - IP for server and client are hard-coded in - I believe it is /32, as the netmask is reported as 255.255.255.255 - When running w/ WiFi: - wlan0 : ip 10.0.200.136/16 (IP and /16 from DHCP server) - When running w/ Ethernet: - eth0 : ip 10.0.200.113/16 (IP and /16 from DHCP server) One odd thing I find here is that lwIP does not support 2 interfaces in the same subnet. So you would have to take care that only one netif at a time has its link set up (or its address set or its administrative status up, or whatever). In any case, having both wlan0 and eth0 up and running with an address in the same subnet will result in packets being accepted on both netifs but transmitted on whichever was registered last (or was it first?). I assumed that both were not active at the same time, good catch, this might be an issue as well. This is not "connection sharing", this is IP routing, not NAT !, "connection sharing" is usually used for hosts that do SNAT on their Interface facing Internet. IP routing needs a complete routing table on all hosts telling how to reach (i.e. which next-hop to use) every host of the whole network. Example with 2 subnets and 3 Hosts: A-eth0 - eth0-B-eth1 - eth0-C 192.168.0.1/24192.168.0.2/24 172.16.1.1/24 172.16.1.2/24 Host A routing table: 192.168.0.0/24 dev eth0 172.16.1.0/24 via 192.168.0.2 Host B routing table: 192.168.0.0/24 dev eth0 172.16.1.0/24 dev eth1 Host C routing table: 192.168.0.0/24 via 172.16.1.1 172.16.1.0/24 dev eth0 dev = directly connected via = next-hop, gateway if you prefer What you could see is that each host have a similar routing table, just the next-hop changes. Default route is just a way to reduce the routing table size by removing all entries that have an identical next-hop. Which brings me to a question: how do we define such routing rules in lwIP? Hmm... I'm surprised by the question :-) In lwIP, "dev" rules don't need to be added, the ip4_route function "create" them dynamically using the interface address/netmask. Well, yes, my bad. The question wasn't phrased well. The thing I don't understand (because I'm not using it) probably all boils down to using lwIP as a PPP 'proxy' to remote networks without doing NAT at the same time (which we don't support in mainline). That's not standard routing... But thanks you for asking that because there is a huge issue about that with the aforementioned case. Since "dev" rules are not ordered because we just foreach on interfaces list until a match, we don't respect that smaller subnets should be checked and matched first. So if an IP subnet is within the IP subnet of another interface, although perfectly legal configuration but *bad* design, it does not work in lwIP. In lwIP it actually depends on the order interfaces were added :-) Which results in me citing my mantra "lwIP does *not* support multiple netifs in the same subnet". (And for those who ask why: most of the time, this is just bad network desing. For the rest where it is intended: we could make it work, but the burdon on the rest of us not needing it has been too much, up to now.) For "via" routes, we have LWIP_HOOK_IP4_ROUTE. Yeah, well, this is probably not used much, being a hook that people need to implement on their own? A default implementation for a routing algorithm woule be nice... Regards, Simon ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi Simon, On Thu, Jan 30, 2020 at 09:31:11PM +0100, goldsi...@gmx.de wrote: > Am 30.01.2020 um 20:42 schrieb Sylvain Rochet: > > On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote: > > > > > > Host2: > > > - Micontroller device, ESP32 > > > - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), > > > used > > >mutually exclusively > > > - Running lwIP > > > - WiFi and Ethernet are independently verified to work to reach WAN > > > from > > >this host > > > - PPPoS server > > > - ppp0 : 10.0.0.1/32 > > > - IP for server and client are hard-coded in > > > - I believe it is /32, as the netmask is reported as > > > 255.255.255.255 > > > - When running w/ WiFi: > > > - wlan0 : ip 10.0.200.136/16 (IP and /16 from DHCP server) > > > - When running w/ Ethernet: > > > - eth0 : ip 10.0.200.113/16 (IP and /16 from DHCP server) > > One odd thing I find here is that lwIP does not support 2 interfaces in > the same subnet. So you would have to take care that only one netif at a > time has its link set up (or its address set or its administrative > status up, or whatever). In any case, having both wlan0 and eth0 up and > running with an address in the same subnet will result in packets being > accepted on both netifs but transmitted on whichever was registered last > (or was it first?). I assumed that both were not active at the same time, good catch, this might be an issue as well. > > This is not "connection sharing", this is IP routing, not NAT !, > > "connection sharing" is usually used for hosts that do SNAT on their > > Interface facing Internet. > > > > IP routing needs a complete routing table on all hosts telling how to > > reach (i.e. which next-hop to use) every host of the whole network. > > > > Example with 2 subnets and 3 Hosts: > > > > A-eth0 - eth0-B-eth1 - eth0-C > >192.168.0.1/24192.168.0.2/24 172.16.1.1/24 172.16.1.2/24 > > > > Host A routing table: > > 192.168.0.0/24 dev eth0 > > 172.16.1.0/24 via 192.168.0.2 > > > > Host B routing table: > > 192.168.0.0/24 dev eth0 > > 172.16.1.0/24 dev eth1 > > > > Host C routing table: > > 192.168.0.0/24 via 172.16.1.1 > > 172.16.1.0/24 dev eth0 > > > > dev = directly connected > > via = next-hop, gateway if you prefer > > > > What you could see is that each host have a similar routing table, just > > the next-hop changes. Default route is just a way to reduce the routing > > table size by removing all entries that have an identical next-hop. > > Which brings me to a question: how do we define such routing rules in > lwIP? Hmm... I'm surprised by the question :-) In lwIP, "dev" rules don't need to be added, the ip4_route function "create" them dynamically using the interface address/netmask. But thanks you for asking that because there is a huge issue about that with the aforementioned case. Since "dev" rules are not ordered because we just foreach on interfaces list until a match, we don't respect that smaller subnets should be checked and matched first. So if an IP subnet is within the IP subnet of another interface, although perfectly legal configuration but *bad* design, it does not work in lwIP. In lwIP it actually depends on the order interfaces were added :-) For "via" routes, we have LWIP_HOOK_IP4_ROUTE. Sylvain signature.asc Description: Digital signature ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Am 30.01.2020 um 20:42 schrieb Sylvain Rochet: Hi Andrew, On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote: Uh-oh, I think I did a poor job of explaining it. Although you seem to have surmised most of it. The full rundown: Host1: - Micontroller device with no MAC or PHY - running NetX IP stack - Runs a PPP client, specifically PPPoS using TTL UART - default route: ppp0 - ppp0 is the only network interface available - ARP is supported and enabled - IP 10.0.0.2 is assigned during PPP negotiation There are no ARP in PPP, but whatever, this configuration is correct. Host2: - Micontroller device, ESP32 - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used mutually exclusively - Running lwIP - WiFi and Ethernet are independently verified to work to reach WAN from this host - PPPoS server - ppp0 : 10.0.0.1/32 - IP for server and client are hard-coded in - I believe it is /32, as the netmask is reported as 255.255.255.255 - When running w/ WiFi: - wlan0 : ip 10.0.200.136/16 (IP and /16 from DHCP server) - When running w/ Ethernet: - eth0 : ip 10.0.200.113/16 (IP and /16 from DHCP server) One odd thing I find here is that lwIP does not support 2 interfaces in the same subnet. So you would have to take care that only one netif at a time has its link set up (or its address set or its administrative status up, or whatever). In any case, having both wlan0 and eth0 up and running with an address in the same subnet will result in packets being accepted on both netifs but transmitted on whichever was registered last (or was it first?). - Routing is unknown. lwIP built with IP_FORWARD enabled. - default route: wlan0 or eth0 (assumed) - ARP is enabled. 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that shouldn't be a problem here, this is usually a bad practice and it should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't know there is a hole in the subnet, since there are only 3 hosts here, which is an expection rather than the rule, it is safe by luck. WiFi AP's: - AP only (no routing) - Connected to Host3 by Ethernet - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control association) Host3: - Router (pfSense) - DHCP server runs here - Routes to WAN, does NAT, eth0 <-> eth1 - default route: set by DHCP, toward eth1 - no other route (as far as I know) - Intranet WiFi AP's are connected via switch to eth0 - Intranet Ethernet devices are connected via switch to eth0 - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95 - eth1 : IP from ISP, facing public IPv4 internet, MAC 00:e0:67:18:54:94 Here is the real problem, you need a route on Host3 telling that 10.0.0.2/32 is behind 10.0.200.136 (or 10.0.200.113). You can also add a route for 10.0.0.1/32 but it's just a bonus. Without this route, Host3 don't know how to route the packet and will try to find an host on the 10.0.0.0/16 subnet using ARP. Then it's just luck, and depends on what Host3 and Host2 are doing: - If the packet is sent in a broadcast Ethernet frame by Host3 it might works if Host2 catches the packet and forwards it to Host1. - If the ARP request gets answered by Host2 when Host3 send their ARP request to find who is 10.0.0.2 on the 10.0.0.0/16 subnet, then it also works. This is known as ARP Proxy, lwIP does not support that. It's just luck, using lazy features of IP stack to make it works even if the configuration is broken should be avoided :-) Layout: Host1 <-> Host2 <-> Host3 <-> public internet PPPoS Ethernet or Host1 <-> Host2 <> WiFi AP <--> Host3 <---> public internet PPPoS WiFi Ethernet For the case of using the WiFi connection, there are separate WiFi AP's in the way. Other than that, the two cases are host2 connecting to the local subnet via wlan0 or connecting via eth0. I guess the only reason it works with WiFi and not with Ethernet is because WiFi is intrinsically broadcast-based while Ethernet is switched (well, I hope so, unless you live in '90s :p). This is the "works because Host2 catches the broadcast packet" case I've said before. I am not manually adding any static routes in either Host1 or Host2. I was worried that Host2 would not be able to accomplish the "connection sharing" until I found the IP_FORWARDING option. That initially worked with Host2 using WiFi without any manually added static routes. This is not "connection sharing", this is IP routing, not NAT !, "connection sharing" is usually used for hosts that do SNAT on their Interface facing Internet. IP routing needs a complete routing table on all hosts telling how to reach (i.e. which
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi Andrew, On Wed, Jan 29, 2020 at 08:15:11PM -0800, Andrew Pullin wrote: > > Uh-oh, I think I did a poor job of explaining it. Although you seem to have > surmised most of it. > > The full rundown: > > Host1: > - Micontroller device with no MAC or PHY > - running NetX IP stack > - Runs a PPP client, specifically PPPoS using TTL UART > - default route: ppp0 > - ppp0 is the only network interface available > - ARP is supported and enabled > - IP 10.0.0.2 is assigned during PPP negotiation There are no ARP in PPP, but whatever, this configuration is correct. > Host2: > - Micontroller device, ESP32 > - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used > mutually exclusively > - Running lwIP > - WiFi and Ethernet are independently verified to work to reach WAN from > this host > - PPPoS server > - ppp0 : 10.0.0.1/32 > - IP for server and client are hard-coded in > - I believe it is /32, as the netmask is reported as 255.255.255.255 > - When running w/ WiFi: > - wlan0 : ip 10.0.200.136/16 (IP and /16 from DHCP server) > - When running w/ Ethernet: > - eth0 : ip 10.0.200.113/16 (IP and /16 from DHCP server) > - Routing is unknown. lwIP built with IP_FORWARD enabled. > - default route: wlan0 or eth0 (assumed) > - ARP is enabled. 10.0.0.1/32 and 10.0.0.2/32 are within 10.0.0.0/16 subnet, that shouldn't be a problem here, this is usually a bad practice and it should be avoided. This is a problem for hosts on 10.0.0.0/16 that don't know there is a hole in the subnet, since there are only 3 hosts here, which is an expection rather than the rule, it is safe by luck. > WiFi AP's: > - AP only (no routing) > - Connected to Host3 by Ethernet > - MACs 0c:8d:db:6e:f0:03 or 0c:8d:db:6e:f0:88 (can't control > association) > > Host3: > - Router (pfSense) > - DHCP server runs here > - Routes to WAN, does NAT, eth0 <-> eth1 > - default route: set by DHCP, toward eth1 > - no other route (as far as I know) > - Intranet WiFi AP's are connected via switch to eth0 > - Intranet Ethernet devices are connected via switch to eth0 > - eth0 : 10.0.1.1/16 , facing intranet, MAC 00:e0:67:18:54:95 > - eth1 : IP from ISP, facing public IPv4 internet, MAC 00:e0:67:18:54:94 Here is the real problem, you need a route on Host3 telling that 10.0.0.2/32 is behind 10.0.200.136 (or 10.0.200.113). You can also add a route for 10.0.0.1/32 but it's just a bonus. Without this route, Host3 don't know how to route the packet and will try to find an host on the 10.0.0.0/16 subnet using ARP. Then it's just luck, and depends on what Host3 and Host2 are doing: - If the packet is sent in a broadcast Ethernet frame by Host3 it might works if Host2 catches the packet and forwards it to Host1. - If the ARP request gets answered by Host2 when Host3 send their ARP request to find who is 10.0.0.2 on the 10.0.0.0/16 subnet, then it also works. This is known as ARP Proxy, lwIP does not support that. It's just luck, using lazy features of IP stack to make it works even if the configuration is broken should be avoided :-) > Layout: > > Host1 <-> Host2 <-> Host3 > <-> public internet > PPPoS Ethernet > > or > > Host1 <-> Host2 <> WiFi AP > <--> Host3 <---> public internet > PPPoS WiFi Ethernet > > > For the case of using the WiFi connection, there are separate WiFi AP's in > the way. > Other than that, the two cases are host2 connecting to the local subnet via > wlan0 or connecting via eth0. I guess the only reason it works with WiFi and not with Ethernet is because WiFi is intrinsically broadcast-based while Ethernet is switched (well, I hope so, unless you live in '90s :p). This is the "works because Host2 catches the broadcast packet" case I've said before. > I am not manually adding any static routes in either Host1 or Host2. > I was worried that Host2 would not be able to accomplish the "connection > sharing" until I found the IP_FORWARDING option. That initially worked with > Host2 using WiFi without any manually added static routes. This is not "connection sharing", this is IP routing, not NAT !, "connection sharing" is usually used for hosts that do SNAT on their Interface facing Internet. IP routing needs a complete routing table on all hosts telling how to reach (i.e. which next-hop to use) every host of the whole network. Example with 2 subnets and 3 Hosts: A-eth0 - eth0-B-eth1 - eth0-C 192.168.0.1/24192.168.0.2/24 172.16.1.1/24 172.16.1.2/24 Host A routing table: 192.168.0.0/24 dev eth0 172.16.1.0/24 via 192.168.0.2 Host B routing table: 192.168.0.0/24 dev eth0 172.16.1.0/24 dev eth1 Host C routing
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi, On 2020-01-30 05:15, Andrew Pullin wrote: On 1/29/20 5:42 AM, Sylvain Rochet wrote: Hi Andrew, On Tue, Jan 28, 2020 at 04:57:19PM -0800, Andrew Pullin wrote: Hi folks, I am stuck on an issue here where I am trying to use lwip's IP_FORWARDING feature. I am trying to forward packets between a PPP server netif and a WAN interface. It is working on case, where the WAN-connected device is using WiFi, but then it fails to work in another case where the WAN device is configured to use an Ethernet interface. This is to ultimately bring Ethernet connectivity to a device without Ethernet, but which can act as a PPP client (via UART). Lwip is running on a device with Ethernet and using the lwip PPP server implementation. I have read the other threads involving PPP and forwarding, but I don't believe I have seen quite this same problem arise. Here is how the devices are arranged for each case: Working: 10.0.0.2 --[UART]--> 10.0.0.1 --> 10.0.200.121 --> 10.0.1.1 ---> WAN (PPP client) (PPP server, ESP32, lwip) (WiFi STA netif) ( router ) In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1) and the netmask as 255.255.0.0 Not working: 10.0.0.2 --[UART]--> 10.0.0.1 --> 10.0.200.113 --> 10.0.1.1 ---> WAN (PPP client) (PPP server, ESP32, lwip) (Ethernet netif) ( router ) In this case, the Ethernet netif reports the gateway IP correctly (10.0.1.1) and the netmask as 255.255.0.0 , which appears to be the same as for the working WiFi netif case. The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually configured. They are hardcoded into the PPP server setup, and the client appears to be able to take it's 10.0.0.2 address via IPCP over PPP. To test that the WAN connection "works" for the PPP client, I am issuing pings from PPP client device. In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway), and 8.8.8.8. All work. In the "not working" case on ethernet, I can ping 10.0.0.1 (just over the PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable. Of course, I am building with IP_FORWARD defined to 1. Debug logs do reflect that forwarding is happening. There are several shims in the Espressif SDK over the actual lwip netif sets/calls. As far as I can see, the same setup is done for the ethernet netif as is for the wifi netif, but this is an obviously suspicious point. In my case, I am using an ESP32 chip, and all of the lwip init and setup is done for me in the SDK. As far as I can see, the init for each adapter/netif type looks the same. I have logs for each case, too: Working (wifi netif) - https://pastebin.com/RgqHa5CR Not working (ethernet netif) - https://pastebin.com/L80raiRF One note here is: the PPP client is not running lwip. For hardware & legacy reasons, it is running another network stack with a totally different PPP client implementation. But, given that the working/not working is a function of the PPP server netif's, I did not suspect a mis-configuration issue in the PPP client. Any insight here would be greatly appreciated. I am not too familiar with the deep under-the-hood details of network stacks. I tried to understand but I failed. Before anything else, could you share the real network configuration of all network interfaces, the routing table on all hosts, and any speciafic features enabled on each interface (NAT, ARP-Proxy, …), for all configurations you tried ? Example: [Host1]ppp0 --- ppp0[Host2]eth0 --- eth0[Host3]eth1 --- Internet Host1: running a PPP client, that's all we know ppp0: 10.0.0.2/32 default route: ppp0 no other route Host2: running lwIP ppp0: 10.0.0.1/32 eth0: 10.0.200.121/16 default route: 10.0.1.1 no other route Host3: gateway ? to what ? eth0: 10.0.1.1/16 eth1: public IPv4, facing Internet, set by DHCP, SNAT default route: set by DHCP, toward eth1 no other route Hint: the configuration above can't work, but I'm not sure this is what you currently have... Sylvain Uh-oh, I think I did a poor job of explaining it. Although you seem to have surmised most of it. The full rundown: Host1: - Micontroller device with no MAC or PHY - running NetX IP stack - Runs a PPP client, specifically PPPoS using TTL UART - default route: ppp0 - ppp0 is the only network interface available - ARP is supported and enabled - IP 10.0.0.2 is assigned during PPP negotiation Host2: - Micontroller device, ESP32 - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used mutually exclusively - Running lwIP - WiFi and Ethernet are independently verified to work to reach WAN from this host - PPPoS server - ppp0 : 10.0.0.1/32 - IP for server and client are hard-coded in - I believe it is /32, as the netmask is reported as
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
On 1/29/20 5:42 AM, Sylvain Rochet wrote: Hi Andrew, On Tue, Jan 28, 2020 at 04:57:19PM -0800, Andrew Pullin wrote: Hi folks, I am stuck on an issue here where I am trying to use lwip's IP_FORWARDING feature. I am trying to forward packets between a PPP server netif and a WAN interface. It is working on case, where the WAN-connected device is using WiFi, but then it fails to work in another case where the WAN device is configured to use an Ethernet interface. This is to ultimately bring Ethernet connectivity to a device without Ethernet, but which can act as a PPP client (via UART). Lwip is running on a device with Ethernet and using the lwip PPP server implementation. I have read the other threads involving PPP and forwarding, but I don't believe I have seen quite this same problem arise. Here is how the devices are arranged for each case: Working: 10.0.0.2 --[UART]--> 10.0.0.1 --> 10.0.200.121 --> 10.0.1.1 ---> WAN (PPP client) (PPP server, ESP32, lwip) (WiFi STA netif) ( router ) In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1) and the netmask as 255.255.0.0 Not working: 10.0.0.2 --[UART]--> 10.0.0.1 --> 10.0.200.113 --> 10.0.1.1 ---> WAN (PPP client) (PPP server, ESP32, lwip) (Ethernet netif) ( router ) In this case, the Ethernet netif reports the gateway IP correctly (10.0.1.1) and the netmask as 255.255.0.0 , which appears to be the same as for the working WiFi netif case. The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually configured. They are hardcoded into the PPP server setup, and the client appears to be able to take it's 10.0.0.2 address via IPCP over PPP. To test that the WAN connection "works" for the PPP client, I am issuing pings from PPP client device. In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway), and 8.8.8.8. All work. In the "not working" case on ethernet, I can ping 10.0.0.1 (just over the PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable. Of course, I am building with IP_FORWARD defined to 1. Debug logs do reflect that forwarding is happening. There are several shims in the Espressif SDK over the actual lwip netif sets/calls. As far as I can see, the same setup is done for the ethernet netif as is for the wifi netif, but this is an obviously suspicious point. In my case, I am using an ESP32 chip, and all of the lwip init and setup is done for me in the SDK. As far as I can see, the init for each adapter/netif type looks the same. I have logs for each case, too: Working (wifi netif) - https://pastebin.com/RgqHa5CR Not working (ethernet netif) - https://pastebin.com/L80raiRF One note here is: the PPP client is not running lwip. For hardware & legacy reasons, it is running another network stack with a totally different PPP client implementation. But, given that the working/not working is a function of the PPP server netif's, I did not suspect a mis-configuration issue in the PPP client. Any insight here would be greatly appreciated. I am not too familiar with the deep under-the-hood details of network stacks. I tried to understand but I failed. Before anything else, could you share the real network configuration of all network interfaces, the routing table on all hosts, and any speciafic features enabled on each interface (NAT, ARP-Proxy, …), for all configurations you tried ? Example: [Host1]ppp0 --- ppp0[Host2]eth0 --- eth0[Host3]eth1 --- Internet Host1: running a PPP client, that's all we know ppp0: 10.0.0.2/32 default route: ppp0 no other route Host2: running lwIP ppp0: 10.0.0.1/32 eth0: 10.0.200.121/16 default route: 10.0.1.1 no other route Host3: gateway ? to what ? eth0: 10.0.1.1/16 eth1: public IPv4, facing Internet, set by DHCP, SNAT default route: set by DHCP, toward eth1 no other route Hint: the configuration above can't work, but I'm not sure this is what you currently have... Sylvain Uh-oh, I think I did a poor job of explaining it. Although you seem to have surmised most of it. The full rundown: Host1: - Micontroller device with no MAC or PHY - running NetX IP stack - Runs a PPP client, specifically PPPoS using TTL UART - default route: ppp0 - ppp0 is the only network interface available - ARP is supported and enabled - IP 10.0.0.2 is assigned during PPP negotiation Host2: - Micontroller device, ESP32 - has both WiFi and Ethernet PHY hardware (IP101GRI Ethernet PHY), used mutually exclusively - Running lwIP - WiFi and Ethernet are independently verified to work to reach WAN from this host - PPPoS server - ppp0 : 10.0.0.1/32 - IP for server and client are hard-coded in - I believe it is /32, as the netmask is reported as 255.255.255.255 - When running w/ WiFi: - wlan0 : ip 10.0.200.136/16 (IP and
Re: [lwip-users] IP forwarding to/from PPP working for one netif, but not another
Hi Andrew, On Tue, Jan 28, 2020 at 04:57:19PM -0800, Andrew Pullin wrote: > Hi folks, > > I am stuck on an issue here where I am trying to use lwip's IP_FORWARDING > feature. > I am trying to forward packets between a PPP server netif and a WAN > interface. It is working on case, where the WAN-connected device is using > WiFi, but then it fails to work in another case where the WAN device is > configured to use an Ethernet interface. > > This is to ultimately bring Ethernet connectivity to a device without > Ethernet, but which can act as a PPP client (via UART). > Lwip is running on a device with Ethernet and using the lwip PPP server > implementation. > > I have read the other threads involving PPP and forwarding, but I don't > believe I have seen quite this same problem arise. > > Here is how the devices are arranged for each case: > > Working: > 10.0.0.2 --[UART]--> 10.0.0.1 --> > 10.0.200.121 --> 10.0.1.1 ---> WAN > (PPP client) (PPP server, ESP32, lwip) (WiFi STA netif) > ( router ) > > In this case, the WiFi netif reports the gateway IP correctly (10.0.1.1) and > the netmask as 255.255.0.0 > > Not working: > 10.0.0.2 --[UART]--> 10.0.0.1 --> > 10.0.200.113 --> 10.0.1.1 ---> WAN > (PPP client) (PPP server, ESP32, lwip) > (Ethernet netif) ( router ) > > In this case, the Ethernet netif reports the gateway IP correctly (10.0.1.1) > and the netmask as 255.255.0.0 , which appears to be the same as for the > working WiFi netif case. > > The addresses 10.0.0.1 and 10.0.0.2 on each side of the PPP are manually > configured. They are hardcoded into the PPP server setup, and the client > appears to be able to take it's 10.0.0.2 address via IPCP over PPP. > > > To test that the WAN connection "works" for the PPP client, I am issuing > pings from PPP client device. > In the "working" case, I can ping 10.0.0.1, 10.0.1.1 (network gateway), and > 8.8.8.8. All work. > In the "not working" case on ethernet, I can ping 10.0.0.1 (just over the > PPP link), but 10.0.1.1 and 8.8.8.8 appear unreachable. > > Of course, I am building with IP_FORWARD defined to 1. Debug logs do reflect > that forwarding is happening. > There are several shims in the Espressif SDK over the actual lwip netif > sets/calls. > As far as I can see, the same setup is done for the ethernet netif as is for > the wifi netif, but this is an obviously suspicious point. > > In my case, I am using an ESP32 chip, and all of the lwip init and setup is > done for me in the SDK. > As far as I can see, the init for each adapter/netif type looks the same. > > I have logs for each case, too: > Working (wifi netif) - https://pastebin.com/RgqHa5CR > Not working (ethernet netif) - https://pastebin.com/L80raiRF > > One note here is: the PPP client is not running lwip. For hardware & legacy > reasons, it is running another network stack with a totally different PPP > client implementation. > But, given that the working/not working is a function of the PPP server > netif's, I did not suspect a mis-configuration issue in the PPP client. > > Any insight here would be greatly appreciated. I am not too familiar with > the deep under-the-hood details of network stacks. I tried to understand but I failed. Before anything else, could you share the real network configuration of all network interfaces, the routing table on all hosts, and any speciafic features enabled on each interface (NAT, ARP-Proxy, …), for all configurations you tried ? Example: [Host1]ppp0 --- ppp0[Host2]eth0 --- eth0[Host3]eth1 --- Internet Host1: running a PPP client, that's all we know ppp0: 10.0.0.2/32 default route: ppp0 no other route Host2: running lwIP ppp0: 10.0.0.1/32 eth0: 10.0.200.121/16 default route: 10.0.1.1 no other route Host3: gateway ? to what ? eth0: 10.0.1.1/16 eth1: public IPv4, facing Internet, set by DHCP, SNAT default route: set by DHCP, toward eth1 no other route Hint: the configuration above can't work, but I'm not sure this is what you currently have... Sylvain signature.asc Description: Digital signature ___ lwip-users mailing list lwip-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/lwip-users
