Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Marcelo Ricardo Leitner 
On Wed, Apr 04, 2018 at 09:38:02PM +0200, Michael Welzl wrote:
> 
> 
> Sent from my iPhone
> 
> > On 4 Apr 2018, at 21:23, Michael Richardson  wrote:
> > 
> > 
> > Dave Taht  wrote:
> >> How dead is posix these days? Ietf does not generally do apis well.
> > 
> > I disagree.
> > 
> > IETF has lore says that it doesn't do APIs well, and so it's a
> > self-fullfiling prophecy.  Everyone knows it's true without any actual
> > evidence, so nobody tries.
> > 
> > Who does do APIs well today?  Microsoft's API story is a disaster,
> > so despite decades of market dominance, their networking APIs are not
> > defacto standard.  While it looks like Linux does APIs well, actually,
> > it just does a really good job at getting the public APIs implemented.
> > 
> > The IPv6 BSD sockets API is a roaring success.
> > It's just not enough given MIF, DNSSD, IPsec, LISP, etc.  The problem
> > space has expanded.
> > 
> > The problem is getting API work funded across Google, Apple, MS, *BSD,
> > and Linux.
> 
> fwiw, apple is on board of taps and actively involved, and the neat
> project  ( www.neat-project.org ) made an open source library that
> runs on linux and bsd systems

Neat has a very interesting concept. If they can pull that out, it
will be very nice.

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Marcelo Ricardo Leitner 
On Wed, Apr 04, 2018 at 01:06:22PM +0200, Luca Muscariello wrote:
> I'm aware of this one. The last time I checked Linux patches seemed to be
> abandoned.
> Hit ratio could be v v low if you remove UDP encap. Look at IPSEC.

And SCTP.

> 
> 
> On Wed, Apr 4, 2018 at 12:52 PM, Mikael Abrahamsson 
> wrote:
> 
> > On Wed, 4 Apr 2018, Luca Muscariello wrote:
> >
> > And yes, flow queueing, absolutely. Flow isolation, becomes fundamental is
> >> such a zoo, or jungle.
> >>
> >
> > There was talk in IETF about a transport protocol that was proposed to do
> > a lot of things TCP doesn't do, but still retain some things that has been
> > useful with TCP.
> >
> > I think it was this one:
> >
> > https://datatracker.ietf.org/doc/draft-ietf-nvo3-gue/
> >
> > I'd like to see it not over UDP, but rather as a native IP protocol. The
> > talk was about having the network being able to look into the state machine
> > of the protocol (MSS size, equivalent of SYN, etc) but not into payload
> > (which would be end-to-end encrypted). It would also be able to do muxed
> > streams/message based to avoid head-of-line-blocking because of single
> > packet loss.
> >
> > So any of this that comes up then the whole FQ machinery might benefit
> > frmo being able to identify flows in any new protocol, but I imagine this
> > is not a hard thing to do. I still have hopes for the flow label in IPv6 to
> > do this job, even though it hasn't seen wide adoption so far.
> >
> >
> > --
> > Mikael Abrahamssonemail: swm...@swm.pp.se
> >

> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Michael Welzl 


Sent from my iPhone

> On 4 Apr 2018, at 21:23, Michael Richardson  wrote:
> 
> 
> Dave Taht  wrote:
>> How dead is posix these days? Ietf does not generally do apis well.
> 
> I disagree.
> 
> IETF has lore says that it doesn't do APIs well, and so it's a
> self-fullfiling prophecy.  Everyone knows it's true without any actual
> evidence, so nobody tries.
> 
> Who does do APIs well today?  Microsoft's API story is a disaster,
> so despite decades of market dominance, their networking APIs are not
> defacto standard.  While it looks like Linux does APIs well, actually,
> it just does a really good job at getting the public APIs implemented.
> 
> The IPv6 BSD sockets API is a roaring success.
> It's just not enough given MIF, DNSSD, IPsec, LISP, etc.  The problem
> space has expanded.
> 
> The problem is getting API work funded across Google, Apple, MS, *BSD,
> and Linux.

fwiw, apple is on board of taps and actively involved, and the neat project  ( 
www.neat-project.org ) made an open source library that runs on linux and bsd 
systems


> 
> --
> ]   Never tell me the odds! | ipv6 mesh networks [
> ]   Michael Richardson, Sandelman Software Works| network architect  [
> ] m...@sandelman.ca  http://www.sandelman.ca/|   ruby on rails
> [
> 

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Michael Richardson 

Dave Taht  wrote:
> How dead is posix these days? Ietf does not generally do apis well.

I disagree.

IETF has lore says that it doesn't do APIs well, and so it's a
self-fullfiling prophecy.  Everyone knows it's true without any actual
evidence, so nobody tries.

Who does do APIs well today?  Microsoft's API story is a disaster,
so despite decades of market dominance, their networking APIs are not
defacto standard.  While it looks like Linux does APIs well, actually,
it just does a really good job at getting the public APIs implemented.

The IPv6 BSD sockets API is a roaring success.
It's just not enough given MIF, DNSSD, IPsec, LISP, etc.  The problem
space has expanded.

The problem is getting API work funded across Google, Apple, MS, *BSD,
and Linux.

--
]   Never tell me the odds! | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works| network architect  [
] m...@sandelman.ca  http://www.sandelman.ca/|   ruby on rails[



signature.asc
Description: PGP signature
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread David Collier-Brown 
The phenomenon is called "lava flow", and is a classic anti-pattern 
illustrated at http://antipatterns.com/lavaflow.htm
Their approach to fixing is ancient, though: there are 
correctness-preserving refactorings for some of the problem space.


Alas, I don't know if middleboxes are correctable... maybe if they are 
ones which only care about the IP layer?


--dave

On 04/04/18 08:45 AM, Jesper Louis Andersen wrote:
On Tue, Apr 3, 2018 at 5:04 PM Jim Gettys > wrote:


​To get to really good RTT's (with low jitter), you still need
​fq_codel (or similar).  You just can't get there by hacking TCP
no matter how hard you try...


I agree with you on all points here. However, any change which patches 
an existing bad system is far more likely to win in the long run, also 
if it is bad in some way. Momentum is a killer of good solutions. I 
wish I had a ramification for this observation, but I currently don't.


My hunch is that every new generation of young programmers wants to 
put their mark on the system. As a result, they take what worked well 
on level N-1 and proceed to build N on top of it. But the beanstalk 
never withers, so each level is present in said stack, still, after 
all these years.


(Aside: The codel approach also has worked really well for me 
internally in Erlang systems as a way to maintain queue load. Far 
better than many other flow control schemes).



___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat



--
David Collier-Brown, | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dav...@spamcop.net   |  -- Mark Twain

___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Jesper Louis Andersen 
On Tue, Apr 3, 2018 at 5:04 PM Jim Gettys  wrote:

> ​To get to really good RTT's (with low jitter), you still need ​fq_codel
> (or similar).  You just can't get there by hacking TCP no matter how hard
> you try...
>
>
I agree with you on all points here. However, any change which patches an
existing bad system is far more likely to win in the long run, also if it
is bad in some way. Momentum is a killer of good solutions. I wish I had a
ramification for this observation, but I currently don't.

My hunch is that every new generation of young programmers wants to put
their mark on the system. As a result, they take what worked well on level
N-1 and proceed to build N on top of it. But the beanstalk never withers,
so each level is present in said stack, still, after all these years.

(Aside: The codel approach also has worked really well for me internally in
Erlang systems as a way to maintain queue load. Far better than many other
flow control schemes).
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Luca Muscariello 
I'm aware of this one. The last time I checked Linux patches seemed to be
abandoned.
Hit ratio could be v v low if you remove UDP encap. Look at IPSEC.


On Wed, Apr 4, 2018 at 12:52 PM, Mikael Abrahamsson 
wrote:

> On Wed, 4 Apr 2018, Luca Muscariello wrote:
>
> And yes, flow queueing, absolutely. Flow isolation, becomes fundamental is
>> such a zoo, or jungle.
>>
>
> There was talk in IETF about a transport protocol that was proposed to do
> a lot of things TCP doesn't do, but still retain some things that has been
> useful with TCP.
>
> I think it was this one:
>
> https://datatracker.ietf.org/doc/draft-ietf-nvo3-gue/
>
> I'd like to see it not over UDP, but rather as a native IP protocol. The
> talk was about having the network being able to look into the state machine
> of the protocol (MSS size, equivalent of SYN, etc) but not into payload
> (which would be end-to-end encrypted). It would also be able to do muxed
> streams/message based to avoid head-of-line-blocking because of single
> packet loss.
>
> So any of this that comes up then the whole FQ machinery might benefit
> frmo being able to identify flows in any new protocol, but I imagine this
> is not a hard thing to do. I still have hopes for the flow label in IPv6 to
> do this job, even though it hasn't seen wide adoption so far.
>
>
> --
> Mikael Abrahamssonemail: swm...@swm.pp.se
>
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Mikael Abrahamsson 

On Wed, 4 Apr 2018, Luca Muscariello wrote:

And yes, flow queueing, absolutely. Flow isolation, becomes fundamental 
is such a zoo, or jungle.


There was talk in IETF about a transport protocol that was proposed to do 
a lot of things TCP doesn't do, but still retain some things that has been 
useful with TCP.


I think it was this one:

https://datatracker.ietf.org/doc/draft-ietf-nvo3-gue/

I'd like to see it not over UDP, but rather as a native IP protocol. The 
talk was about having the network being able to look into the state 
machine of the protocol (MSS size, equivalent of SYN, etc) but not into 
payload (which would be end-to-end encrypted). It would also be able to do 
muxed streams/message based to avoid head-of-line-blocking because of 
single packet loss.


So any of this that comes up then the whole FQ machinery might benefit 
frmo being able to identify flows in any new protocol, but I imagine this 
is not a hard thing to do. I still have hopes for the flow label in IPv6 
to do this job, even though it hasn't seen wide adoption so far.


--
Mikael Abrahamssonemail: swm...@swm.pp.se
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Luca Muscariello 
I'm looking at TAPS too as I'm looking for a general transport API other
than TCP/UDP.

The kind of transport services we have developed in here
https://git.fd.io/cicn/ do not fit in the current API.
Full user land implementation, which seems to be accepted nowadays, but not
at all a few years back.
The API however is a hack of the INET API, which is rather obsolete, but
this is what current apps can use.
So, hope TAPS gets traction.

And yes, IPv6, absolutely. Going through IPv4 middle boxes is a nightmare.
No hope there.

And yes, flow queueing, absolutely. Flow isolation, becomes fundamental is
such a zoo, or jungle.











On Wed, Apr 4, 2018 at 10:52 AM, Mikael Abrahamsson 
wrote:

> On Wed, 4 Apr 2018, Dave Taht wrote:
>
> How dead is posix these days? Ietf does not generally do apis well.
>>
>
> POSIX nowadays is
>
> http://pubs.opengroup.org/onlinepubs/9699919799/
>
> My take on it is that the IETF should not be scared to do APIs, even
> though there is a lot of resistance still.
>
> However, the IETF should not do POSIX APIs, but instead something of their
> own.
>
> --
> Mikael Abrahamssonemail: swm...@swm.pp.se
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Mikael Abrahamsson 

On Wed, 4 Apr 2018, Michael Welzl wrote:

well - they have been refusing too long to do them at all. i guess 
that’s part of the problem


It's not about refusing to do so, it's because other SDOs have told the 
IETF not to. If IETF tries to touch POSIX, the SDO that does POSIX doesn't 
appreciate this.


--
Mikael Abrahamssonemail: swm...@swm.pp.se___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Mikael Abrahamsson 

On Wed, 4 Apr 2018, Dave Taht wrote:


How dead is posix these days? Ietf does not generally do apis well.


POSIX nowadays is

http://pubs.opengroup.org/onlinepubs/9699919799/

My take on it is that the IETF should not be scared to do APIs, even 
though there is a lot of resistance still.


However, the IETF should not do POSIX APIs, but instead something of their 
own.


--
Mikael Abrahamssonemail: swm...@swm.pp.se
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Michael Welzl 
well - they have been refusing too long to do them at all. i guess that’s part 
of the problem

Sent from my iPhone

> On 4 Apr 2018, at 09:42, Dave Taht  wrote:
> 
> How dead is posix these days? Ietf does not generally do apis well.
> 
>> On Tue, Apr 3, 2018, 9:14 AM Michael Welzl  wrote:
>> 
>>> On Apr 3, 2018, at 4:48 PM, Jesper Louis Andersen 
>>>  wrote:
>>> 
 On Tue, Apr 3, 2018 at 4:27 PM Michael Welzl  wrote:
 please, please, people, take a look at the ietf taps (“transport 
 services”) working group  :-)
 
>>> 
>>> I tried looking it up. It seems the TAPS WG is about building a consistent 
>>> interface to different protocols in order to get a new interface rather 
>>> than, say, the bsd socket interface.
>>> 
>>> But my search turned up several drafts from the WG. Did you have one in 
>>> particular in mind?
>> 
>> Thanks for taking a look!
>> Indeed, it’s about a consistent interface - I was provoked to send this 
>> message by the reference to ossification, and talk of messages (lacking in 
>> TCP).
>> Sure, when you’re in control of both ends of a connection, you can build 
>> whatever you want on top of UDP - but there’s a lot of wheel re-inventing 
>> there. Really, the transport layer can’t change as long as applications (or 
>> their libraries) are exposed to only the services of TCP and UDP, and 
>> thereby statically bound to these transport protocols.
>> 
>> I think I’d recommend this draft as a starting point:  
>> https://taps-api.github.io/drafts/draft-trammell-taps-interface.html
>> 
>> Cheers,
>> Michael
>> 
>> ___
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Dave Taht 
How dead is posix these days? Ietf does not generally do apis well.

On Tue, Apr 3, 2018, 9:14 AM Michael Welzl  wrote:

>
> On Apr 3, 2018, at 4:48 PM, Jesper Louis Andersen <
> jesper.louis.ander...@gmail.com> wrote:
>
> On Tue, Apr 3, 2018 at 4:27 PM Michael Welzl  wrote:
>
>> please, please, people, take a look at the ietf taps (“transport
>> services”) working group  :-)
>>
>>
> I tried looking it up. It seems the TAPS WG is about building a consistent
> interface to different protocols in order to get a new interface rather
> than, say, the bsd socket interface.
>
> But my search turned up several drafts from the WG. Did you have one in
> particular in mind?
>
>
> Thanks for taking a look!
> Indeed, it’s about a consistent interface - I was provoked to send this
> message by the reference to ossification, and talk of messages (lacking in
> TCP).
> Sure, when you’re in control of both ends of a connection, you can build
> whatever you want on top of UDP - but there’s a lot of wheel re-inventing
> there. Really, the transport layer can’t change as long as applications (or
> their libraries) are exposed to only the services of TCP and UDP, and
> thereby statically bound to these transport protocols.
>
> I think I’d recommend this draft as a starting point:
> https://taps-api.github.io/drafts/draft-trammell-taps-interface.html
>
> Cheers,
> Michael
>
> ___
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Seen in passing: mention of Valve's networking scheme and RFC 5348

2018-04-04 Thread Mikael Abrahamsson 

On Tue, 3 Apr 2018, Michael Welzl wrote:

Sure, when you’re in control of both ends of a connection, you can build 
whatever you want on top of UDP - but there’s a lot of wheel 
re-inventing there. Really, the transport layer can’t change as long as 
applications (or their libraries) are exposed to only the services of 
TCP and UDP, and thereby statically bound to these transport protocols.


I'm aware of TAPS and I have been trying to gather support for this kind 
of effort for years now, and I'm happy to see there is movement. I have 
also heard encouraging talk from several entities interested in actually 
doing serious work in this area, including some opensourcing part of their 
now non-FOSS code-base as part of that work.


So we need applications to be able to get more access to what's going on 
the wire, including access to non-TCP/UDP, but also to be able to create 
"pluggable TCP-stacks" so that a host can have several different ones, and 
the user can install new ones even on older operating systems.


With more and more IPv6 around, I hope we'll be able to deploy new 
protocols that are not TCP/UDP (A+P), and that this will bring back some 
innovation in that area.


--
Mikael Abrahamssonemail: swm...@swm.pp.se___
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat