Re: [PATCH RFC 0/4] net: add bpfilter

On Mon, Feb 19, 2018 at 01:52:18PM +0100, Harald Welte wrote: > Hi Daniel, > > On Mon, Feb 19, 2018 at 01:03:17PM +0100, Daniel Borkmann wrote: > > Hi Harald, > > > > On 02/17/2018 01:11 PM, Harald Welte wrote: > > [...] [...] > It would be an interesting test to see if e.g. docker would run on

Re: nft/bpf interpreters and spectre2. Was: [PATCH RFC 0/4] net: add bpfilter

[resend as plaintext, apparently mobile gmail will send HTML mails] On Thu, Feb 22, 2018 at 3:20 AM, Alexei Starovoitov wrote: > On Wed, Feb 21, 2018 at 01:13:03PM +0100, Florian Westphal wrote: >> >> Obvious candidates are: meta, numgen, limit, objref, quota,

Re: nft/bpf interpreters and spectre2. Was: [PATCH RFC 0/4] net: add bpfilter

On Thu, Feb 22, 2018 at 12:39:15PM +0100, Pablo Neira Ayuso wrote: > Hi Alexei, > > On Wed, Feb 21, 2018 at 06:20:37PM -0800, Alexei Starovoitov wrote: > > On Wed, Feb 21, 2018 at 01:13:03PM +0100, Florian Westphal wrote: > > > > > > Obvious candidates are: meta, numgen, limit, objref, quota,

Re: nft/bpf interpreters and spectre2. Was: [PATCH RFC 0/4] net: add bpfilter

Hi Alexei, On Wed, Feb 21, 2018 at 06:20:37PM -0800, Alexei Starovoitov wrote: > On Wed, Feb 21, 2018 at 01:13:03PM +0100, Florian Westphal wrote: > > > > Obvious candidates are: meta, numgen, limit, objref, quota, reject. > > > > We should probably also consider removing > >

nft/bpf interpreters and spectre2. Was: [PATCH RFC 0/4] net: add bpfilter

On Wed, Feb 21, 2018 at 01:13:03PM +0100, Florian Westphal wrote: > > Obvious candidates are: meta, numgen, limit, objref, quota, reject. > > We should probably also consider removing > CONFIG_NFT_SET_RBTREE and CONFIG_NFT_SET_HASH and just always > build both too (at least rbtree since that

Re: [PATCH RFC 0/4] net: add bpfilter

Pablo Neira Ayuso wrote: > On Tue, Feb 20, 2018 at 05:52:54PM -0800, Alexei Starovoitov wrote: > > On Tue, Feb 20, 2018 at 11:44:31AM +0100, Pablo Neira Ayuso wrote: > > > > > > Don't get me wrong, no software is safe from security issues, but if you > > > don't abstract

Re: [PATCH RFC 0/4] net: add bpfilter

On Tue, Feb 20, 2018 at 05:52:54PM -0800, Alexei Starovoitov wrote: > On Tue, Feb 20, 2018 at 11:44:31AM +0100, Pablo Neira Ayuso wrote: > > > > Don't get me wrong, no software is safe from security issues, but if you > > don't abstract your resources in the right way, you have more chance to

Re: [PATCH RFC 0/4] net: add bpfilter

On Tue, Feb 20, 2018 at 11:44:31AM +0100, Pablo Neira Ayuso wrote: > > Don't get me wrong, no software is safe from security issues, but if you > don't abstract your resources in the right way, you have more chance to > have experimence more problems. interesting point. The key part of

Re: [PATCH RFC 0/4] net: add bpfilter

Hi Michal, On Tue, Feb 20, 2018 at 10:35:41AM +0100, Michal Kubecek wrote: > On Mon, Feb 19, 2018 at 06:09:39PM +0100, Phil Sutter wrote: > > What puzzles me about your argumentation is that you seem to propose for > > the kernel to cover up flaws in userspace. Spinning this concept further > >

Re: [PATCH RFC 0/4] net: add bpfilter

From: Pablo Neira Ayuso Date: Tue, 20 Feb 2018 11:44:31 +0100 > * Lack of sufficient abstraction: bpf is not only exposing its own > software bugs through its interface, but it will also bite the dust > with CPU bugs due to lack of glue code to hide details behind the >

Re: [PATCH RFC 0/4] net: add bpfilter

On 02/20/2018 11:44 AM, Pablo Neira Ayuso wrote: > Hi David! > > On Mon, Feb 19, 2018 at 12:22:26PM -0500, David Miller wrote: > [...] >> Netfilter's chronic performance differential is why a lot of mindshare >> was lost to userspace networking technologies. > > Claiming that Netfilter is the

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 12:15:37PM -0500, David Miller wrote: > From: Phil Sutter > Date: Mon, 19 Feb 2018 18:09:39 +0100 > > > What puzzles me about your argumentation is that you seem to propose for > > the kernel to cover up flaws in userspace. Spinning this concept

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David! On Mon, Feb 19, 2018 at 12:22:26PM -0500, David Miller wrote: [...] > Netfilter's chronic performance differential is why a lot of mindshare > was lost to userspace networking technologies. Claiming that Netfilter is the reason for the massive adoption of userspace networking isn't a

Re: [PATCH RFC 0/4] net: add bpfilter

On Mon, Feb 19, 2018 at 06:09:39PM +0100, Phil Sutter wrote: > What puzzles me about your argumentation is that you seem to propose for > the kernel to cover up flaws in userspace. Spinning this concept further > would mean that if there would be an old bug in iproute2 we should think > of adding

Re: [PATCH RFC 0/4] net: add bpfilter

> I see several possible areas of contention: > > 1) If you aim for a non-feature-complete support of iptables rules, it >will create confusion to the users. Right, you need full feature parity to be avoid ending up having to maintain two implementations. It seems uncontroversial that BPF

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, 19 Feb 2018, Florian Westphal wrote: > David Miller wrote: > > > > Florian, first of all, the whole "change the iptables binary" idea is > > a non-starter. For the many reasons I have described in the various > > postings I have made today. > > > > It

Re: [PATCH RFC 0/4] net: add bpfilter

David Miller wrote: > From: Phil Sutter > Date: Mon, 19 Feb 2018 18:14:11 +0100 > > > OK, so reading between the lines you're saying that nftables project > > has failed to provide an adequate successor to iptables? > > Whilst it is great that the atomic table

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 01:41:29PM -0500, David Miller wrote: > From: Phil Sutter > Date: Mon, 19 Feb 2018 19:05:51 +0100 > > > On Mon, Feb 19, 2018 at 12:22:26PM -0500, David Miller wrote: > >> From: Phil Sutter > >> Date: Mon, 19 Feb 2018 18:14:11 +0100 >

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 19:37:30 +0100 > I was speaking of actual *users* as in indiiduals running their own > systems, companies running their own servers/datacenter. The fact that > some ISP (or its supplier) decisdes that one of my IP packets is routed

Re: [PATCH RFC 0/4] net: add bpfilter

From: Arturo Borrero Gonzalez Date: Mon, 19 Feb 2018 19:06:12 +0100 > Yes, probably major datacenters (google? facebook?, amazon?) they > don't even care about what Debian is doing, since they are crafting > their own distro anyway. But there are *a lot* of other people

Re: [PATCH RFC 0/4] net: add bpfilter

From: Phil Sutter Date: Mon, 19 Feb 2018 19:05:51 +0100 > Hi David, > > On Mon, Feb 19, 2018 at 12:22:26PM -0500, David Miller wrote: >> From: Phil Sutter >> Date: Mon, 19 Feb 2018 18:14:11 +0100 >> >> > OK, so reading between the lines you're saying that nftables

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 12:29:08PM -0500, David Miller wrote: > People with an Android phone in their pocket is using iptables, and > the overhead and performance of those rules really does matter. It > determines how long your battery life is, etc. I am not the android expert.

Re: [PATCH RFC 0/4] net: add bpfilter

On 19 February 2018 at 16:36, David Miller wrote: > > In my opinion, any resistence to integration with eBPF and XDP will > lead to even less adoption of netfilter as a technology. > > Therefore my plan is to move everything to be integrated around these > important core

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 12:22:26PM -0500, David Miller wrote: > From: Phil Sutter > Date: Mon, 19 Feb 2018 18:14:11 +0100 > > > OK, so reading between the lines you're saying that nftables project > > has failed to provide an adequate successor to iptables? > > Whilst it

Re: [PATCH RFC 0/4] net: add bpfilter

On 19 February 2018 at 16:27, David Miller wrote: > From: Florian Westphal > Date: Mon, 19 Feb 2018 16:15:55 +0100 > >> Would you be willing to merge nftables into kernel tools directory >> then? > > Did you miss the part where I explained that people

Re: [PATCH RFC 0/4] net: add bpfilter

On 19 February 2018 at 16:36, David Miller wrote: > > I think netfilter is at a real crossroads right now. > I don't think so. The Netfilter Project and the Netfilter Community already "agreed" on nftables and we are working on it. But this isn't a secret, right? We have

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 10:31:39AM -0500, David Miller wrote: > > Why is it practical to replace your kernel but not practical to replace > > a small userspace tool running on top of it? > > The container is just userspace components. Those are really baked in > and are never

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 18:20:40 +0100 > It's like with any migration. People were using ipchains for a long > time even after iptables existed. Many people simply don't care > about packet filter performance. It's only a small fraction of their >

Re: [PATCH RFC 0/4] net: add bpfilter

From: Phil Sutter Date: Mon, 19 Feb 2018 18:14:11 +0100 > OK, so reading between the lines you're saying that nftables project > has failed to provide an adequate successor to iptables? Whilst it is great that the atomic table update problem was solved, I think the emphasis on

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 10:36:51AM -0500, David Miller wrote: > nftables has been proported as "better" for years, yet large > institutions did not migrate to it. In fact, they explicitly > disabled NFTABLES in their kernel config. It's like with any migration. People were using

Re: [PATCH RFC 0/4] net: add bpfilter

From: Phil Sutter Date: Mon, 19 Feb 2018 18:09:39 +0100 > What puzzles me about your argumentation is that you seem to propose for > the kernel to cover up flaws in userspace. Spinning this concept further > would mean that if there would be an old bug in iproute2 we should think >

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 10:44:59AM -0500, David Miller wrote: > From: Harald Welte > Date: Mon, 19 Feb 2018 16:38:08 +0100 > > > On Mon, Feb 19, 2018 at 10:27:27AM -0500, David Miller wrote: > >> > Would you be willing to merge nftables into kernel tools

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 10:31:39AM -0500, David Miller wrote: > From: Harald Welte > Date: Mon, 19 Feb 2018 16:27:46 +0100 > > > On Mon, Feb 19, 2018 at 10:13:35AM -0500, David Miller wrote: > > > >> Florian, first of all, the whole "change the iptables binary"

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 16:38:08 +0100 > On Mon, Feb 19, 2018 at 10:27:27AM -0500, David Miller wrote: >> > Would you be willing to merge nftables into kernel tools directory >> > then? >> >> Did you miss the part where I explained that people explicitly

Re: [PATCH RFC 0/4] net: add bpfilter

From: Jan Engelhardt Date: Mon, 19 Feb 2018 16:37:57 +0100 (CET) > On Monday 2018-02-19 16:32, David Miller wrote: > >>From: Harald Welte >>Date: Mon, 19 Feb 2018 16:23:21 +0100 >> >>> Also, as long as legacy ip_tables/x_tables is still in the kernel, you

Re: [PATCH RFC 0/4] net: add bpfilter

Dear David, On Mon, Feb 19, 2018 at 10:27:27AM -0500, David Miller wrote: > > Would you be willing to merge nftables into kernel tools directory > > then? > > Did you miss the part where I explained that people explicitly disable > NFTABLES in their kernel configs in most if not all large

Re: [PATCH RFC 0/4] net: add bpfilter

On Monday 2018-02-19 16:32, David Miller wrote: >From: Harald Welte >Date: Mon, 19 Feb 2018 16:23:21 +0100 > >> Also, as long as legacy ip_tables/x_tables is still in the kernel, you >> can still run your old userspace against that old implementation in the >> kernel. >

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 16:23:21 +0100 >> Like it or not iptables ABI based filtering is going to be in the data >> path for many years if not a decade or more to come. > > I beg to differ. For some people, yes. but then, as Florian points > out, they

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 16:23:21 +0100 > Also, as long as legacy ip_tables/x_tables is still in the kernel, you > can still run your old userspace against that old implementation in the > kernel. But without offloading, and the various other benefits

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 16:27:46 +0100 > On Mon, Feb 19, 2018 at 10:13:35AM -0500, David Miller wrote: > >> Florian, first of all, the whole "change the iptables binary" idea is >> a non-starter. For the many reasons I have described in the various >>

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 09:44:51AM -0500, David Miller wrote: > I see talk about "just replacing the iptables binary". > > A long time ago in a galaxy far far away, that would be a reasonable > scheme. But that kind of approach won't work in the realities of > today. > > You aren't

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Mon, Feb 19, 2018 at 10:13:35AM -0500, David Miller wrote: > Florian, first of all, the whole "change the iptables binary" idea is > a non-starter. For the many reasons I have described in the various > postings I have made today. > > It is entirely impractical. Why is it

Re: [PATCH RFC 0/4] net: add bpfilter

From: Florian Westphal Date: Mon, 19 Feb 2018 16:20:23 +0100 > See my other mail, where I explained, in great detail, the problems > of the xtables UAPI. As the person who wrote the bpfilter UAPI parser for this, you don't need to explain this to me. But it's not going

Re: [PATCH RFC 0/4] net: add bpfilter

From: Florian Westphal Date: Mon, 19 Feb 2018 16:15:55 +0100 > Would you be willing to merge nftables into kernel tools directory > then? Did you miss the part where I explained that people explicitly disable NFTABLES in their kernel configs in most if not all large datacenters?

Re: [PATCH RFC 0/4] net: add bpfilter

David Miller wrote: > From: Florian Westphal > Date: Mon, 19 Feb 2018 15:53:14 +0100 > > > Sure, but looking at all the things that were added to iptables > > to alleviate some of the issues (ipset for instance) show that we need a > > meaningful re-design

Re: [PATCH RFC 0/4] net: add bpfilter

David Miller wrote: > From: Florian Westphal > Date: Mon, 19 Feb 2018 15:59:35 +0100 > > > David Miller wrote: > >> It also means that the scope of developers who can contribute and work > >> on the translater is much larger. > > > >

Re: [PATCH RFC 0/4] net: add bpfilter

From: Florian Westphal Date: Mon, 19 Feb 2018 15:59:35 +0100 > David Miller wrote: >> It also means that the scope of developers who can contribute and work >> on the translater is much larger. > > How so? Translator is in userspace in nftables case too?

Re: [PATCH RFC 0/4] net: add bpfilter

From: Florian Westphal Date: Mon, 19 Feb 2018 15:53:14 +0100 > Sure, but looking at all the things that were added to iptables > to alleviate some of the issues (ipset for instance) show that we need a > meaningful re-design of how things work conceptually. As you said iptables

Re: [PATCH RFC 0/4] net: add bpfilter

David Miller wrote: > From: Daniel Borkmann > Date: Mon, 19 Feb 2018 13:03:17 +0100 > > > Thought was that it would be more suitable to push all the complexity of > > such translation into user space which brings couple of additional > > advantages >

Re: [PATCH RFC 0/4] net: add bpfilter

From: Daniel Borkmann Date: Mon, 19 Feb 2018 13:03:17 +0100 > Thought was that it would be more suitable to push all the complexity of > such translation into user space which brings couple of additional advantages > as well: the translation can become very complex and thus

Re: [PATCH RFC 0/4] net: add bpfilter

David Miller wrote: > > How many of those wide-spread applications are you aware of? The two > > projects you have pointed out (docker and kubernetes) don't. As the > > assumption that many such tools would need to be supported drives a lot > > of the design decisions, I

Re: [PATCH RFC 0/4] net: add bpfilter

From: Harald Welte Date: Mon, 19 Feb 2018 13:52:18 +0100 >> Right, having a custom iptables, libiptc or LD_PRELOAD approach would work >> as well of course, but it still wouldn't address applications that have >> their own custom libs programmed against iptables uapi

Re: [PATCH RFC 0/4] net: add bpfilter

Hi Daniel, On Mon, Feb 19, 2018 at 01:03:17PM +0100, Daniel Borkmann wrote: > Hi Harald, > > On 02/17/2018 01:11 PM, Harald Welte wrote: > [...] > >> As rule translation can potentially become very complex, this is performed > >> entirely in user space. In order to ease deployment,

Re: [PATCH RFC 0/4] net: add bpfilter

Daniel Borkmann wrote: > As rule translation can potentially become very complex, this is performed > entirely in user space. In order to ease deployment, request_module() code > is extended to allow user mode helpers to be invoked. Idea is that user mode > helpers are built

Re: [PATCH RFC 0/4] net: add bpfilter

Harald Welte wrote: > I believe _if_ one wants to use the approach of "hiding" eBPF behind > iptables, then either [..] > b) you must introduce new 'tables', like an 'xdp' table which then has >the notion of processing very early in processing, way before the >normal

Re: [PATCH RFC 0/4] net: add bpfilter

Florian Westphal wrote: > David Miller wrote: > > From: Florian Westphal > > Date: Fri, 16 Feb 2018 17:14:08 +0100 > > > > > Any particular reason why translating iptables rather than nftables > > > (it should be possible to monitor the

Re: [PATCH RFC 0/4] net: add bpfilter

David Miller wrote: > From: Florian Westphal > Date: Fri, 16 Feb 2018 17:14:08 +0100 > > > Any particular reason why translating iptables rather than nftables > > (it should be possible to monitor the nftables changes that are > > announced by kernel and

Re: [PATCH RFC 0/4] net: add bpfilter

Daniel Borkmann wrote: > Hi Florian, > > On 02/16/2018 05:14 PM, Florian Westphal wrote: > > Florian Westphal wrote: > >> Daniel Borkmann wrote: > >> Several questions spinning at the moment, I will probably come up with > >> more: >

Re: [PATCH RFC 0/4] net: add bpfilter

Hi Daniel, On Fri, Feb 16, 2018 at 02:40:19PM +0100, Daniel Borkmann wrote: > This is a very rough and early proof of concept that implements bpfilter. > The basic idea of bpfilter is that it can process iptables queries and > translate them in user space into BPF programs which can then get

Re: [PATCH RFC 0/4] net: add bpfilter

Hi Daniel, On Fri, Feb 16, 2018 at 09:44:01PM +0100, Daniel Borkmann wrote: > We started out with the > iptables part in the demo as the majority of bigger infrastructure projects > all still rely heavily on it (e.g. docker, k8s to just name two big ones). docker is exec'ing the iptables command

Re: [PATCH RFC 0/4] net: add bpfilter

Hi David, On Fri, Feb 16, 2018 at 05:33:54PM -0500, David Miller wrote: > From: Florian Westphal > > > Any particular reason why translating iptables rather than nftables > > (it should be possible to monitor the nftables changes that are > > announced by kernel and act on

Re: [PATCH RFC 0/4] net: add bpfilter

From: Florian Westphal Date: Fri, 16 Feb 2018 17:14:08 +0100 > Any particular reason why translating iptables rather than nftables > (it should be possible to monitor the nftables changes that are > announced by kernel and act on those)? As Daniel said, iptables is by far the

Re: [PATCH RFC 0/4] net: add bpfilter

From: Florian Westphal Date: Fri, 16 Feb 2018 15:57:27 +0100 > 4. Do you plan to reimplement connection tracking in userspace? > If no, how will the bpf program interact with it? The natural way to handle this, as with anything BPF related, is with appropriate BPF helpers which

Re: [PATCH RFC 0/4] net: add bpfilter

Hi Florian, On 02/16/2018 05:14 PM, Florian Westphal wrote: > Florian Westphal wrote: >> Daniel Borkmann wrote: >> Several questions spinning at the moment, I will probably come up with >> more: > > ... and here there are some more ... > > One of the many

Re: [PATCH RFC 0/4] net: add bpfilter

Hi Florian, thanks for your feedback! More inline: On 02/16/2018 03:57 PM, Florian Westphal wrote: > Daniel Borkmann wrote: >> This is a very rough and early proof of concept that implements bpfilter. > > [..] > >> Also, as a benefit from such design, we get BPF JIT

Re: [PATCH RFC 0/4] net: add bpfilter

Florian Westphal wrote: > Daniel Borkmann wrote: > Several questions spinning at the moment, I will probably come up with > more: ... and here there are some more ... One of the many pain points of xtables design is the assumption of 'used only by

Re: [PATCH RFC 0/4] net: add bpfilter

Daniel Borkmann wrote: > This is a very rough and early proof of concept that implements bpfilter. [..] > Also, as a benefit from such design, we get BPF JIT compilation on x86_64, > arm64, ppc64, sparc64, mips64, s390x and arm32, but also rule offloading > into HW for

[PATCH RFC 0/4] net: add bpfilter

This is a very rough and early proof of concept that implements bpfilter. The basic idea of bpfilter is that it can process iptables queries and translate them in user space into BPF programs which can then get attached at various locations. For simplicity, in this RFC we demo attaching them to