On Jul 19, 2007, at 6:15 AM, David Barrett wrote:
Hi Cullen, I think you bring up lot of really great and detailed
questions.
However, speaking for myself at least, answering those questions
doesn't
really address my concern. All I want to know is:
Does ICE work, and how well?
What "works" means is entirely open to debate.
It may have been in the past, but the debate I think is closed,
because the requirements captured the definition of "works". If you
want to re-open the requirements discussion, please be clear about that.
I've tossed out "high peer
connectivity" and "low connection setup time" as the most important
criteria, and I've put hard data behind those numbers as a stake in
the
ground for comparison, but I'm open to other metrics as well.
See the requirements. One not often mentioned but clearly a
requirement that ICE works very hard to meet is "use the most direct
path available".
But regardless of the metrics, I want somebody who uses it and
loves it and
bases their business upon its success to stand up and explain why it's
great.
And I want this person to provide real-world data, sampled on a wide
scale, proving that ICE "works" -- by any definition.
This is in fact an extremely important exercise, and precisely the
activity which according to RFC2026 is what should be happening after
approval as proposed standard, and before advancement to draft
standard.
Basically, I want ICE firmly grounded in wide-scale testing and
real-world
data, and answering the questions below doesn't bring us any closer
to that.
Whether it's level 10 or level 12 doesn't produce new data. How its
algorithms differ from competing ones doesn't show that they're any
better
or worse. Detailed proposals for change don't shed any light on
whether
those changes improve or worsen it, because its current state is
completely
unmeasured.
Showing how the algorithms work better, or work in more cases that
are covered by the requirements does in fact show that it's better,
with respect to the requirements. Again, you're arguing indirectly
that we should re-open the requirements discussion, which I do not
think would be terribly productive, but would support if there is in
fact consensus to do so.
The most powerful thing the IETF can do to standardize the world
upon ICE is
prove that it works better than any alternative. Because until that's
proven, we just need to take it as a matter of faith.
I do not think that is desirable. The perfect is the enemy of the
good. ICE does not have to be better than anything else. It just has
to be "good enough" and see wide adoption.
That's my 2ยข (as an individual, not as an IAB member, in case anybody
cares).
DaveO
-david
-----Original Message-----
From: Cullen Jennings [mailto:[EMAIL PROTECTED]
Sent: Wednesday, July 18, 2007 7:56 PM
To: SIP; Behave WG; [EMAIL PROTECTED]
Subject: Re: [Sip] Re: [BEHAVE] Re: ICE deployment data before LC
for RFC
First I apologize for the cross post - this is a mmusic draft, I
would appreciate replies on the mmusic list only.
Few things I am looking for here ....
1) specific reasons why this work is less appropriate for standards
track than other work that IETF puts on the standards track. Keep in
mind the first step of standards track is Proposed Standards which
requires basically no interoperability, then it could move to Draft
Standard where interoperability is shown and finally Full Standard if
it turns out to be a good idea. If you are going to post a reply to
this point, make sure you have read RFC 2026.
2) I am interested in the specifics of problems that ICE fails to
solve that the WG has not previous considered. There are lots of
people that have implemented versions of it and think it is a good
idea - if people are having some problems with it, I'd love to hear
about details of things that are wrong. That it has a lot of messages
in some use cases has been known for a long time. It used to have far
more messages - engineering was done to consider what was a
reasonable amount for call set up times and a new design was done to
bring the message to a level most people invovled with the design
felt was reasonable. If there are specific other than these that are
wrong and not considered in the design, now would be a good time to
get them on the table.
3) If people think that Skype for example has a simpler solution that
works better than ICE. Feel free to describe exactly what it is.
Several people invovled with the ICE design have looked in extreme
detail at Skype and others and so far none of them have come back
with what we should do different than ICE.
4) ICE can be used in a mode where it only provided the reflexive
candidate. People were pretty clear that this was far inferior to the
modes where there are other candidates. If someone would like to
provide exactly how hole punching works better than this very limited
ICE technique, I'd be glad to hear the details. This has certainly
been an topic that any could have brought up over the last several
years. If someone has specifics, I suggest they get out a full
proposal really soon. I am very doubtful that hole punching solution
is fundamentally different than the type of hole punching ICE does.
5) Some people have suggested that if all NATs were behave complaint,
then we would not need ICE. I strongly disagree. I think the only
part of ICE we would not need is the TURN relays which are more or
less an optional part. Again, specifics on what can be removed are
welcome.
ICE is far more complicated that anyone wants - the problem is, no
one has come up with a proposal that looks like it comes anywhere
close to meeting the requirements and if simpler. The design space
has been extensively explored by many people. The raw data to
understand what will work on what NATs have been painfully tested and
documented in publications such as:
http://nutss.gforge.cis.cornell.edu/pub/imc05-tcpnat.pdf
http://tools.ietf.org/id/draft-jennings-behave-test-results-04.txt
https://www.guha.cc/saikat/stunt-results.php?
(Ford used to have a page too but it seems to be gone)
I have a very open mind to considering better ways to do ICE - but I
need specific suggestions or it is impossible for me to really think
about if the suggestions represent a better path or not.
I would love to see more open source ICE code (there is lots already)
- I feel that this is an area where ICE is likely just table stakes
and owning an proprietary ICE stack will not turn out to be a
competitive advantage. Having open source implementations where the
development cost was shared and everyone could get working products
to market faster would be an advantage to companies. Clearly I expect
the bulk of ICE implementations to not be open source but I still
would be keen on doing what I can to contribute to an open source
version as personally I feel that is in the interest of end users, my
employer, and the IETF as a SDO.
Cullen <with my individual hat on>
On Jul 18, 2007, at 9:12 AM, Dean Willis wrote:
On Jul 18, 2007, at 3:10 AM, Magnus Westerlund wrote:
Henry Sinnreich skrev:
Following some discussions and soul searching, the best
approach for
ICE-17 for LC would be to _go ahead and make it an experimental
RFC_.
This would support the development of interoperable
implementations, the
collection of deployment data and also hopefully open source code.
Though I have full faith in ICE, following the practices that
made the
IETF successful take precedent here, I believe.
This is a change from the attached that I wrote on 7/16/2007.
Note: We can only hope the proliferation of SBC in VoIP service
provider
networks will not make these concerns and ICE for SIP a moot
issue, but
this is another topic.
Henry,
I don't quite understand why you are requesting a status change of
ICE? To me it appears that ICE is almost finished spec. It has
been implemented, used and feedback has been influencing the
protocol. It is definitly one of the better examples of current
IETF work that actually has running code. Sure, the
implementations may not be fully updated yet to the latest draft
version. But I think it is safe to say that ICE works in the
intended core cases. If there are some corner cases where it
fails, that may be. But that is not information we will have until
really large scale deployement or someone makes very extensive
tests.
Henry has, I believe, been very consistent in urging the IETF to
have more "running code" experience on every specification (not
just ICE) than our process requires. He has, IIRC, also advocated
changing the process to require running code before publishing an
RFC. He's even suggested at times that one should have running code
before submitting a protocol draft.
Another point to consider -- I've known Henry to bark at a tree
with no squirrel in it occasionally, but most of the time he's
pretty reliable, more so than most technology predictors. If he's
worried about ICE, it's probably because he's been talking to
people who are trying to implement it and they're telling him it is
difficult or that they're having performance or reliability issues.
I imagine that there are plenty of internal development teams in
various companies that are working on this and not necessarily
bringing their results back to the IETF. After all, if everybody
sees the emperor as fully clothed, they'd be embarrassed to
proclaim otherwise. Of course, I wish all of these developers were
tied into the IETF, but then I'm sure I don't want to see every ice-
implementor's question on the SIP list. That is why we have a sip-
implementors mailing list, right?
Personally, I think ICE is the best solution we have. We KNOW of
cases where hole-punching doesn't work, and I think ICE will
produce usable results in most of those cases. But that's just my
personal opinion from reading the specs. I haven't implemented ICE,
and I haven't used anybody else's implementation. Sure, it's
complex, but that's because the underlying problem is complex --
there are just too many different behaviors out there in "NAT and
firewall land". Of course, I could be wrong, too. It's happened
before, once or twice ;-). Just how sure are we that we're right
about this?
Are we designing a "spruce goose" or a practical system? Only
experience will tell us. But some good flight reports from the
prototypes could go a long way towards reassuring us of the
design's utility.
So far we've had a couple of people report hearsay evidence -- that
somebody else has an ICE implementation and it works. Any of you
folks have personal experience you can share? Have there been
prototypes exercised at the interop events?
--
Dean
_______________________________________________
Behave mailing list
[EMAIL PROTECTED]
https://www1.ietf.org/mailman/listinfo/behave
_______________________________________________
Behave mailing list
[EMAIL PROTECTED]
https://www1.ietf.org/mailman/listinfo/behave
_______________________________________________
Sip mailing list https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [EMAIL PROTECTED] for questions on current sip
Use [EMAIL PROTECTED] for new developments on the application of sip
_______________________________________________
Sip mailing list https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [EMAIL PROTECTED] for questions on current sip
Use [EMAIL PROTECTED] for new developments on the application of sip
