Re: How to retrieve original source address with FTP/NAT/TPROXY
Hello,
I have done some more digging into the kernel code to try to understand
what is happening. I noticed that in net/netfilter/nf_nat_ftp.c, the
expectfn on the FTP data connection expectation is set to
nf_nat_follow_master().
I was wondering about the purpose of this function? Could it be the
reason I do not get the source address I expect from accept()?
My understanding is that source NAT is usually done in the POSTROUTING
chain and that accept() should always return the source address
untouched. Is this correct?
Thank you for your help,
Gregory
On 02/12/2018 05:39 PM, Gregory Vander Schueren wrote:
Hi Pablo,
Thank you for getting back to me. Also thanks for pointing me to
libnetfilter_conntrack, I will definitely have a look.
I am using kernel 4.1.39 and the issue can be reproduced with the
following ruleset on the Proxy host:
iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
iptables -t mangle -A PREROUTING -p tcp ! --dport 21 -i eth0 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port
Then I simply retrieve an FTP file (in passive mode) from the client and
I use a toy C program (see below) on the Proxy to retrieve the source IP.
I am still wondering if the IP returned by accept is a normal behavior.
What do you think? Don't we expect accept() to return the client's IP?
Regards,
Gregory
#include
#include
int main()
{
struct sockaddr_in addr;
socklen_t len = sizeof(addr);
int sock;
int one = 1;
addr.sin_family = AF_INET;
addr.sin_port = htons();
addr.sin_addr.s_addr = INADDR_ANY;
sock = socket(PF_INET, SOCK_STREAM, 0);
setsockopt(sock, SOL_IP, IP_TRANSPARENT, , sizeof(one));
bind(sock, (struct sockaddr*), sizeof(addr));
listen(sock, 1);
accept(sock,(struct sockaddr *), );
char *ip = inet_ntoa(addr.sin_addr);
printf("%s\n", ip);
}
On 02/08/2018 05:55 PM, Pablo Neira Ayuso wrote:
Hi Gregory,
On Tue, Feb 06, 2018 at 03:40:20PM +0100, Gregory Vander Schueren wrote:
Hello,
I have the following IPv4 network:
FTPClient <-> Proxy <--> FTPServer.
10.0.0.2 10.0.0.1 1.1.1.1 1.1.1.2
FTPClient connects to FTPServer in PASSIVE mode, meaning the FTPClient
initiates the data connection towards FTPServer. Proxy performs NAT
in the
POSTROUTING chain using the iptables MASQUERADE target. On Proxy, I
use the
iptables TPROXY target to redirect the FTP data connection towards a
local
socket.
Upon accept() on this socket, the address returned by accept() is
1.1.1.1,
not the IP of the Client (10.0.0.2) as I expected. Using
getpeername() also
returns 1.1.1.1. For other TCP connections than FTP accept() or
getpeername() returns 10.0.0.2.
I noticed this only occurs when using the NF_CONNTRACK_FTP and
NF_NAT_FTP
kernel modules.
Note that I was able to retrieve the FTPClient IP on Proxy from
/proc/net/ip_conntrack. I also made a quick patch to add a
SO_ORIGINAL_SRC socket option
(similar to SO_ORIGINAL_DST) which allows to retrieve the FTPClient
IP. Since this option does not exist yet, I am wondering if this is
relevant to add such an option?
You can use libnetfilter_conntrack to do this these days, via ctnetlink.
Also, this does not occur in IPv6.
Is this behavior normal?
Probably it is related to your ruleset? You could post an example to
reproduce the issue and your kernel version.
Thanks.
--
--
DISCLAIMER.
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended recipient
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] inet: don't call skb_orphan if tproxy happens in layer 2
Hi Florian & Pablo, Thank your very much for your quick feedback. On 02/16/2018 12:28 PM, Pablo Neira Ayuso wrote: On Fri, Feb 16, 2018 at 12:07:06PM +0100, Florian Westphal wrote: Gregory Vander Schueren <gregory.vanderschue...@tessares.net> wrote: [ cc netdev ] If sysctl bridge-nf-call-iptables is enabled, iptables chains are already traversed from the bridging code. In such case, tproxy already happened when reaching ip_rcv. Thus no need to call skb_orphan as this would actually undo tproxy. I don't like this because it adds yet another test in fastpath, and for a use case that has apparently never worked before. Agreed. I also thought this was not ideal but I did find another way to easily fix this. We noticed issues when using tproxy with net.bridge.bridge-nf-call-iptables enabled. In such case, ip_rcv() basically undo tproxy's job. The following patch proposes a fix. I question wheter its a good idea to mix tproxy with bridges. Tproxy relies on policy routing, but a bridge doesn't route :-) I guess you use bridge snat mac mangling to force local delivery of packets that are otherwise bridged? Indeed, we use DNAT MAC mangling. If yes, can you use ebtables brouting instead? This would bypass the bridge (so no iptables invocation from bridge prerouting anymore). We were actually pondering over the usage of MAC DNAT vs brouting. I'll thus follow your suggestion and use brouting instead then. We will try to get rid of nf-call-iptables eventually. Good to know! There might be (more complicated) ways to avoid this problem without adding code in normal network path, but lets check other options first. Agreed. If there's a fix for this, that should be away from the fast path, not in ip_rcv(). -- -- DISCLAIMER. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] inet: don't call skb_orphan if tproxy happens in layer 2
If sysctl bridge-nf-call-iptables is enabled, iptables chains are already
traversed from the bridging code. In such case, tproxy already happened when
reaching ip_rcv. Thus no need to call skb_orphan as this would actually undo
tproxy.
Fixes: 71f9dacd2e4d (inet: Call skb_orphan before tproxy activates)
Signed-off-by: Gregory Vander Schueren <gregory.vanderschue...@tessares.net>
Signed-off-by: Matthieu Baerts <matthieu.bae...@tessares.net>
---
Hi,
We noticed issues when using tproxy with net.bridge.bridge-nf-call-iptables
enabled. In such case, ip_rcv() basically undo tproxy's job. The following
patch proposes a fix.
Feedback would be most welcome,
Gregory
include/linux/netfilter_bridge.h | 12
net/ipv4/ip_input.c | 9 +++--
net/ipv6/ip6_input.c | 9 +++--
3 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/include/linux/netfilter_bridge.h b/include/linux/netfilter_bridge.h
index b671fdf..b1c48f5 100644
--- a/include/linux/netfilter_bridge.h
+++ b/include/linux/netfilter_bridge.h
@@ -66,12 +66,24 @@ static inline bool nf_bridge_in_prerouting(const struct
sk_buff *skb)
{
return skb->nf_bridge && skb->nf_bridge->in_prerouting;
}
+
+static inline bool
+nf_bridge_has_called_iptables(const struct sk_buff *skb)
+{
+ return skb->nf_bridge != NULL;
+}
#else
#define br_drop_fake_rtable(skb) do { } while (0)
static inline bool nf_bridge_in_prerouting(const struct sk_buff *skb)
{
return false;
}
+
+static inline bool
+nf_bridge_has_called_iptables(const struct sk_buff *skb)
+{
+ return false;
+}
#endif /* CONFIG_BRIDGE_NETFILTER */
#endif
diff --git a/net/ipv4/ip_input.c b/net/ipv4/ip_input.c
index 57fc13c..2450205 100644
--- a/net/ipv4/ip_input.c
+++ b/net/ipv4/ip_input.c
@@ -143,6 +143,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -487,8 +488,12 @@ int ip_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt,
memset(IPCB(skb), 0, sizeof(struct inet_skb_parm));
IPCB(skb)->iif = skb->skb_iif;
- /* Must drop socket now because of tproxy. */
- skb_orphan(skb);
+ /* If nf_bridge calls iptables then tproxy already happened.
+* No need to call skb_orphan as this would undo tproxy.
+*/
+ if (!nf_bridge_has_called_iptables(skb))
+ /* Must drop socket now because of tproxy. */
+ skb_orphan(skb);
return NF_HOOK(NFPROTO_IPV4, NF_INET_PRE_ROUTING,
net, NULL, skb, dev, NULL,
diff --git a/net/ipv6/ip6_input.c b/net/ipv6/ip6_input.c
index 9ee208a..60ce4735 100644
--- a/net/ipv6/ip6_input.c
+++ b/net/ipv6/ip6_input.c
@@ -32,6 +32,7 @@
#include
#include
+#include
#include
#include
@@ -202,8 +203,12 @@ int ipv6_rcv(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt
rcu_read_unlock();
- /* Must drop socket now because of tproxy. */
- skb_orphan(skb);
+ /* If nf_bridge calls iptables then tproxy already happened.
+* No need to call skb_orphan as this would undo tproxy.
+*/
+ if (!nf_bridge_has_called_iptables(skb))
+ /* Must drop socket now because of tproxy. */
+ skb_orphan(skb);
return NF_HOOK(NFPROTO_IPV6, NF_INET_PRE_ROUTING,
net, NULL, skb, dev, NULL,
--
2.7.4
--
--
DISCLAIMER.
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended recipient
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to retrieve original source address with FTP/NAT/TPROXY
Hi Pablo,
Thank you for getting back to me. Also thanks for pointing me to
libnetfilter_conntrack, I will definitely have a look.
I am using kernel 4.1.39 and the issue can be reproduced with the
following ruleset on the Proxy host:
iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
iptables -t mangle -A PREROUTING -p tcp ! --dport 21 -i eth0 -j TPROXY
--tproxy-mark 0x1/0x1 --on-port
Then I simply retrieve an FTP file (in passive mode) from the client and
I use a toy C program (see below) on the Proxy to retrieve the source IP.
I am still wondering if the IP returned by accept is a normal behavior.
What do you think? Don't we expect accept() to return the client's IP?
Regards,
Gregory
#include
#include
int main()
{
struct sockaddr_in addr;
socklen_t len = sizeof(addr);
int sock;
int one = 1;
addr.sin_family = AF_INET;
addr.sin_port = htons();
addr.sin_addr.s_addr = INADDR_ANY;
sock = socket(PF_INET, SOCK_STREAM, 0);
setsockopt(sock, SOL_IP, IP_TRANSPARENT, , sizeof(one));
bind(sock, (struct sockaddr*), sizeof(addr));
listen(sock, 1);
accept(sock,(struct sockaddr *), );
char *ip = inet_ntoa(addr.sin_addr);
printf("%s\n", ip);
}
On 02/08/2018 05:55 PM, Pablo Neira Ayuso wrote:
Hi Gregory,
On Tue, Feb 06, 2018 at 03:40:20PM +0100, Gregory Vander Schueren wrote:
Hello,
I have the following IPv4 network:
FTPClient <-> Proxy <--> FTPServer.
10.0.0.2 10.0.0.1 1.1.1.11.1.1.2
FTPClient connects to FTPServer in PASSIVE mode, meaning the FTPClient
initiates the data connection towards FTPServer. Proxy performs NAT in the
POSTROUTING chain using the iptables MASQUERADE target. On Proxy, I use the
iptables TPROXY target to redirect the FTP data connection towards a local
socket.
Upon accept() on this socket, the address returned by accept() is 1.1.1.1,
not the IP of the Client (10.0.0.2) as I expected. Using getpeername() also
returns 1.1.1.1. For other TCP connections than FTP accept() or
getpeername() returns 10.0.0.2.
I noticed this only occurs when using the NF_CONNTRACK_FTP and NF_NAT_FTP
kernel modules.
Note that I was able to retrieve the FTPClient IP on Proxy from
/proc/net/ip_conntrack. I also made a quick patch to add a SO_ORIGINAL_SRC
socket option
(similar to SO_ORIGINAL_DST) which allows to retrieve the FTPClient
IP. Since this option does not exist yet, I am wondering if this is
relevant to add such an option?
You can use libnetfilter_conntrack to do this these days, via ctnetlink.
Also, this does not occur in IPv6.
Is this behavior normal?
Probably it is related to your ruleset? You could post an example to
reproduce the issue and your kernel version.
Thanks.
--
--
DISCLAIMER.
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended recipient
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
How to retrieve original source address with FTP/NAT/TPROXY
Hello, I have the following IPv4 network: FTPClient <-> Proxy <--> FTPServer. 10.0.0.2 10.0.0.1 1.1.1.11.1.1.2 FTPClient connects to FTPServer in PASSIVE mode, meaning the FTPClient initiates the data connection towards FTPServer. Proxy performs NAT in the POSTROUTING chain using the iptables MASQUERADE target. On Proxy, I use the iptables TPROXY target to redirect the FTP data connection towards a local socket. Upon accept() on this socket, the address returned by accept() is 1.1.1.1, not the IP of the Client (10.0.0.2) as I expected. Using getpeername() also returns 1.1.1.1. For other TCP connections than FTP accept() or getpeername() returns 10.0.0.2. I noticed this only occurs when using the NF_CONNTRACK_FTP and NF_NAT_FTP kernel modules. Note that I was able to retrieve the FTPClient IP on Proxy from /proc/net/ip_conntrack. I also made a quick patch to add a SO_ORIGINAL_SRC socket option (similar to SO_ORIGINAL_DST) which allows to retrieve the FTPClient IP. Since this option does not exist yet, I am wondering if this is relevant to add such an option? Also, this does not occur in IPv6. Is this behavior normal? What is the rationale behind this? Any help in understanding what happens would be much appreciated, Gregory -- -- DISCLAIMER. This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
