> On Nov 11, 2017, at 2:09 PM, Michael Welzl <[email protected]> wrote:
>
> Hi,
>
> In line:
>
>> On Nov 11, 2017, at 11:58 AM, Tommy Pauly <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>>
>>
>>> On Nov 11, 2017, at 10:36 AM, Michael Welzl <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>>
>>>> On Nov 11, 2017, at 10:06 AM, Tommy Pauly <[email protected]
>>>> <mailto:[email protected]>> wrote:
>>>>
>>>> Hi Michael,
>>>>
>>>> Just a couple initial notes that may help:
>>>>
>>>> - The version diff you should look at is between -01 and -03. -02 is the
>>>> same as -03, but had a typo.
>>>>
>>>> - You've mentioned previously that you thought that Post requires both
>>>> peers to use that API. That is absolutely not the case in any way. Having
>>>> implemented Post myself, I am only communicating to "legacy" servers that
>>>> know nothing about Post. I think this fundamental understanding needs to
>>>> be cleared up before coming to any conclusions. We can discuss during the
>>>> WG.
>>>
>>> Oh?! That would be fantastic!! But I stumbled over a number of things
>>> that require a system on the other side, or they just couldn’t happen (like
>>> the “carrier forking” notification I quoted below - but there were more).
>>> If you say you can talk to a “legacy” server that knows nothing about Post,
>>> WHAT does that server speak? Legacy which protocol? Maybe it’s just a
>>> matter of me thinking TCP, and you thinking TLS… dunno! Tell me :)
>>
>> So, since Post (and TAPS in general) isn't defining any new protocol
>> features, we need to of course be compatible with all existing protocols.
>> Certainly, some functionality is a bit degenerate in some cases.
>>
>> Whatever the server speaks, the client needs to conform to. So, we use our
>> Post implementation for raw TCP, raw UDP, TLS over TCP, HTTP/2 Streams over
>> TLS over TCP, QUIC streams over UDP, etc. These are all standard server
>> configurations, but Post offers a client application a single set of APIs to
>> talk over any of these protocols.
>
> Oh, great! But then you can’t guarantee that a message arrives as a message
> in case of raw TCP, or you assume application-level framing. This is what
> I’ve been proposing, but the post-sockets draft reads entirely different from
> that.
Messages do indeed "always work", but they may be degenerate. There's no new
protocols here, just ways of looking at them:
- UDP: One message per datagram
- TCP: Entire stream is one big message that only ever ends when that half of
the stream closes
- TLS: Generally viewed like TLS, but can view one record as one message
- HTTP/QUIC: One message per request/response (so, framing)
- Length-value framing on streams, as used by some protocols (like IKEv2 over
TCP): one message per L-V frame
Etc, etc.
That is what the draft does say in Section 2.2. Note how a TCP stream is the
message in some cases.
A Message is the unit of communication between applications.
Messages can represent relatively small structures, such as requests
in a request/response protocol such as HTTP; relatively large
structures, such as files of arbitrary size in a filesystem; and
structures of indeterminate length, such as a stream of bytes in a
protocol like TCP.
>
>
>> In the case of "Carrier Forking", that's essentially what you say to the
>> protocol stack when you want to open a new stream to the same place. So,
>> when I call "fork Carrier", it turns into:
>> - For raw UDP/TCP or TLS (or anything that is mono-streamed), open a new
>> five-tuple to the same remote endpoint
>> - For HTTP/2 and QUIC, open a new stream on the same connection
>
> Sure, that’s what I thought it would. Now, with SCTP, you wouldn’t get a
> “fork request” equivalent on the other side: the client would just start
> using a new stream, and that’s it. (“request” also indicates the possibility
> to say no, which also doesn’t exist in SCTP AFAIK, at least not in the API -
> this would be a “stream reset”). Now, in QUIC, I don’t know exactly how that
> is, but I would have thought it’s the same - you just go ahead and use a new
> stream. No “fork request” on the other side - just a new stream (in post,
> carrier) being used.
The term "fork-request" is misleading, yes. It really should just say: "forking
a carrier creates a new flow to the same Remote, which may be a new stream on a
multiplexing protocol. The protocol will handle any required signaling to the
remote to open a new stream, based on the protocol".
>
>
>>>> - The point of the API is to give the shape of the application
>>>> interaction, which is why you find it general. I believe that many of the
>>>> protocol-specific options like you mention (disabling Nagle,
>>>> retransmissions, etc) don't belong to this main abstract API draft, but to
>>>> a different document that goes into how to configure options that can be
>>>> used for setups like TCP/UDP. Again, in my implementation of Post, all of
>>>> these options exist as part of the Configuration object mentioned in the
>>>> draft. However, as I'll discuss in the WG, I believe that the general
>>>> shape of the API needs to be defined to be more-forward looking than just
>>>> what we've specified with the minset for TCP/UDP/SCTP. Essentially, these
>>>> become a set of Configuration options that can be used, but as transport
>>>> protocols continue to evolve (and when we need more protocol-specific
>>>> options), we need to be able to expand. Certain aspects of the minset,
>>>> like the connection state management, are of course general and common
>>>> enough to be part of the highest level of API description.
>>>
>>> It’s clear to me that we want a higher abstraction level than what the list
>>> from minset has - e.g., rather than a DSCP value, it would be better to
>>> specify general requirements (low latency or such) for a carrier. Rather
>>> than saying “disable Nagle”, we could also say “ low latency, even if it
>>> comes at the cost of some overhead” - we do such things in the NEAT API
>>> too. You’re focused on the interaction with the application (e.g.
>>> callback-based instead of traditional socket-style) - which is fine, but I
>>> think doesn’t have much to do with the actual protocol choice.
>>>
>>> As for being more forward-looking, I wonder what the new transport features
>>> are that we’re missing out on (things that apps really see). I follow QUIC
>>> from a slightly too large distance (I simply run out of cycles there :( ),
>>> but so far I’m not sure there’s anything we’d be missing (but that’s maybe
>>> also because I’m not an HTTP/2 expert either). Except security of course,
>>> but there the argument is perhaps that it’s enough to consider falling back
>>> to TLS?
>>
>> QUIC is using TLS 1.3, so it can give equivalent security properties to
>> HTTP/2. Doing fallback between these is definitely an important feature. I
>> think this flexibility is why the API needs to allow per-protocol configs.
>
> Sure, but I think (and that was my point) this doesn’t change anything about
> the stuff above (security is in your separate document) - so what exactly are
> the “more forward looking” requirements except for these security needs?
I'm trying to distinguish certain aspects of the minset that are:
a) fundamental to all transports, that we have reason to believe will be the
case for all future transports in some fashion
CONNECT, SEND, RECEIVE, CLOSE, etc
b) derived from the protocols we happened to survey or based on current
semantics (sockets), based on what exists
Things I see in the NEAT draft like SET_MIN_CHECKSUM_COVERAGE, SET_TTL,
SET_LOW_WATERMARK. Important to have, but often the values the application
needs to set relate to which protocol might be used, and may have different
requirements for future protocols.
So, there's a set of things in category (a) that should be stable as the main
API go forward for quite a while, while things in category (b) may belong as
configuration properties, in a list that may shrink or expand over time as new
developments are made.
>
>
>>> As for how you represent Nagle etc. in your code, that sounds good to me,
>>> but I see an issue in that so much transport functionality is simply not
>>> covered by the draft. E.g., the packet sizes / fragmentation thing is
>>> important too … yes that’s lower level stuff, but UDP-based applications
>>> will only get inefficient if they don’t get this information exposed, which
>>> risks having app programmers move to the old ways again IMO.
>>
>> A Post API definitely allows access to IP-packet level configuration and
>> metadata. However, you're right that the draft doesn't specify this in
>> detail now. I still think that may be a separate doc, but yeah.
>
> I wouldn’t mind that (though it’s not up to me to decide), but —
>
>
>>> So, the general design approach of post-sockets is IMO fine, but what I’m
>>> missing is a long section with the nitty-gritty details on how such a
>>> system would really be implemented (not the happy eyeballing bit, this is
>>> fine to be elsewhere - but “how would this work with current protocols”).
>>> If I was to implement it from the current draft, I’d have a ton of question
>>> marks - e.g. the draft convinced me that there must be a system on both
>>> sides (this issue is also not discussed in the draft).
>>
>> Getting feedback on how to clarify this for the implementer will be really
>> important, so thanks for this feedback! I'd also like to discuss with the WG
>> the scope of various documents that we want to produce. If you have one
>> document that does the abstract connection-management and I/O API and all
>> the protocol options and the racing/fallback strategy, that would be a very
>> long and complex doc.
>
> Great - I’m happy that this is constructive! Regarding the requests I’m
> making about post sockets, a reason I think more of these implementation
> level considerations need to be in the draft is that this gave me so many
> hiccups of the sort “but then you need it on both sides”, “but then you need
> to add a lot more protocol functionality”, “but do you think you could do
> that?” … etc.
> I guess no matter if there are multiple docs or one, any document for charter
> item 3 should leave its reader with the warm feeling of understanding how
> this could be implemented.
>
> To figure out these details, I’d like to mention that the minset is here to
> help :-) (that’s what it’s meant for). Yes it’s too low-level to be a
> good API proposal in its own right, but that’s also not the point - it tells
> you what can and can’t be done. Every decision in it can be traced back to
> specific reasoning based on the original RFCs; even if you end up disagreeing
> with reasoning X about a mechanism Y from a RFC Z, at least you know
> precisely what you’re doing in relation to the TCP, MPTCP, SCTP, UDP,
> UDP-Lite and LEDBAT specs).
>
> Cheers,
> Michael
_______________________________________________
Taps mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/taps
