Actually, I'm not trying to reopen the requirements debate. I just want ICE measured against the requirements that were agreed upon.
And I completely agree that perfect is the enemy of the good. The only ally of the good is reality, and I want to enlist her on our side. -david > -----Original Message----- > From: David R Oran [mailto:[EMAIL PROTECTED] > Sent: Thursday, July 19, 2007 7:23 AM > To: David Barrett > Cc: 'Cullen Jennings'; 'SIP'; 'Behave WG'; [EMAIL PROTECTED] > Subject: Re: [Sip] Re: [BEHAVE] Re: ICE deployment data before LC for RFC > > > On Jul 19, 2007, at 6:15 AM, David Barrett wrote: > > > Hi Cullen, I think you bring up lot of really great and detailed > > questions. > > > > However, speaking for myself at least, answering those questions > > doesn't > > really address my concern. All I want to know is: > > > > Does ICE work, and how well? > > > > What "works" means is entirely open to debate. > It may have been in the past, but the debate I think is closed, > because the requirements captured the definition of "works". If you > want to re-open the requirements discussion, please be clear about that. > > > I've tossed out "high peer > > connectivity" and "low connection setup time" as the most important > > criteria, and I've put hard data behind those numbers as a stake in > > the > > ground for comparison, but I'm open to other metrics as well. > > > See the requirements. One not often mentioned but clearly a > requirement that ICE works very hard to meet is "use the most direct > path available". > > > But regardless of the metrics, I want somebody who uses it and > > loves it and > > bases their business upon its success to stand up and explain why it's > > great. > > And I want this person to provide real-world data, sampled on a wide > > scale, proving that ICE "works" -- by any definition. > > > This is in fact an extremely important exercise, and precisely the > activity which according to RFC2026 is what should be happening after > approval as proposed standard, and before advancement to draft > standard. > > > Basically, I want ICE firmly grounded in wide-scale testing and > > real-world > > data, and answering the questions below doesn't bring us any closer > > to that. > > Whether it's level 10 or level 12 doesn't produce new data. How its > > algorithms differ from competing ones doesn't show that they're any > > better > > or worse. Detailed proposals for change don't shed any light on > > whether > > those changes improve or worsen it, because its current state is > > completely > > unmeasured. > > > Showing how the algorithms work better, or work in more cases that > are covered by the requirements does in fact show that it's better, > with respect to the requirements. Again, you're arguing indirectly > that we should re-open the requirements discussion, which I do not > think would be terribly productive, but would support if there is in > fact consensus to do so. > > > The most powerful thing the IETF can do to standardize the world > > upon ICE is > > prove that it works better than any alternative. Because until that's > > proven, we just need to take it as a matter of faith. > > > I do not think that is desirable. The perfect is the enemy of the > good. ICE does not have to be better than anything else. It just has > to be "good enough" and see wide adoption. > > That's my 2ยข (as an individual, not as an IAB member, in case anybody > cares). > > DaveO > > -david > > > >> -----Original Message----- > >> From: Cullen Jennings [mailto:[EMAIL PROTECTED] > >> Sent: Wednesday, July 18, 2007 7:56 PM > >> To: SIP; Behave WG; [EMAIL PROTECTED] > >> Subject: Re: [Sip] Re: [BEHAVE] Re: ICE deployment data before LC > >> for RFC > >> > >> > >> First I apologize for the cross post - this is a mmusic draft, I > >> would appreciate replies on the mmusic list only. > >> > >> Few things I am looking for here .... > >> > >> 1) specific reasons why this work is less appropriate for standards > >> track than other work that IETF puts on the standards track. Keep in > >> mind the first step of standards track is Proposed Standards which > >> requires basically no interoperability, then it could move to Draft > >> Standard where interoperability is shown and finally Full Standard if > >> it turns out to be a good idea. If you are going to post a reply to > >> this point, make sure you have read RFC 2026. > >> > >> 2) I am interested in the specifics of problems that ICE fails to > >> solve that the WG has not previous considered. There are lots of > >> people that have implemented versions of it and think it is a good > >> idea - if people are having some problems with it, I'd love to hear > >> about details of things that are wrong. That it has a lot of messages > >> in some use cases has been known for a long time. It used to have far > >> more messages - engineering was done to consider what was a > >> reasonable amount for call set up times and a new design was done to > >> bring the message to a level most people invovled with the design > >> felt was reasonable. If there are specific other than these that are > >> wrong and not considered in the design, now would be a good time to > >> get them on the table. > >> > >> 3) If people think that Skype for example has a simpler solution that > >> works better than ICE. Feel free to describe exactly what it is. > >> Several people invovled with the ICE design have looked in extreme > >> detail at Skype and others and so far none of them have come back > >> with what we should do different than ICE. > >> > >> 4) ICE can be used in a mode where it only provided the reflexive > >> candidate. People were pretty clear that this was far inferior to the > >> modes where there are other candidates. If someone would like to > >> provide exactly how hole punching works better than this very limited > >> ICE technique, I'd be glad to hear the details. This has certainly > >> been an topic that any could have brought up over the last several > >> years. If someone has specifics, I suggest they get out a full > >> proposal really soon. I am very doubtful that hole punching solution > >> is fundamentally different than the type of hole punching ICE does. > >> > >> 5) Some people have suggested that if all NATs were behave complaint, > >> then we would not need ICE. I strongly disagree. I think the only > >> part of ICE we would not need is the TURN relays which are more or > >> less an optional part. Again, specifics on what can be removed are > >> welcome. > >> > >> ICE is far more complicated that anyone wants - the problem is, no > >> one has come up with a proposal that looks like it comes anywhere > >> close to meeting the requirements and if simpler. The design space > >> has been extensively explored by many people. The raw data to > >> understand what will work on what NATs have been painfully tested and > >> documented in publications such as: > >> > >> http://nutss.gforge.cis.cornell.edu/pub/imc05-tcpnat.pdf > >> > >> http://tools.ietf.org/id/draft-jennings-behave-test-results-04.txt > >> > >> https://www.guha.cc/saikat/stunt-results.php? > >> > >> (Ford used to have a page too but it seems to be gone) > >> > >> I have a very open mind to considering better ways to do ICE - but I > >> need specific suggestions or it is impossible for me to really think > >> about if the suggestions represent a better path or not. > >> > >> I would love to see more open source ICE code (there is lots already) > >> - I feel that this is an area where ICE is likely just table stakes > >> and owning an proprietary ICE stack will not turn out to be a > >> competitive advantage. Having open source implementations where the > >> development cost was shared and everyone could get working products > >> to market faster would be an advantage to companies. Clearly I expect > >> the bulk of ICE implementations to not be open source but I still > >> would be keen on doing what I can to contribute to an open source > >> version as personally I feel that is in the interest of end users, my > >> employer, and the IETF as a SDO. > >> > >> Cullen <with my individual hat on> > >> > >> > >> > >> On Jul 18, 2007, at 9:12 AM, Dean Willis wrote: > >> > >>> > >>> On Jul 18, 2007, at 3:10 AM, Magnus Westerlund wrote: > >>> > >>>> Henry Sinnreich skrev: > >>>>> Following some discussions and soul searching, the best > >>>>> approach for > >>>>> ICE-17 for LC would be to _go ahead and make it an experimental > >>>>> RFC_. > >>>>> This would support the development of interoperable > >>>>> implementations, the > >>>>> collection of deployment data and also hopefully open source code. > >>>>> Though I have full faith in ICE, following the practices that > >>>>> made the > >>>>> IETF successful take precedent here, I believe. > >>>>> This is a change from the attached that I wrote on 7/16/2007. > >>>>> Note: We can only hope the proliferation of SBC in VoIP service > >>>>> provider > >>>>> networks will not make these concerns and ICE for SIP a moot > >>>>> issue, but > >>>>> this is another topic. > >>>> > >>>> Henry, > >>>> > >>>> I don't quite understand why you are requesting a status change of > >>>> ICE? To me it appears that ICE is almost finished spec. It has > >>>> been implemented, used and feedback has been influencing the > >>>> protocol. It is definitly one of the better examples of current > >>>> IETF work that actually has running code. Sure, the > >>>> implementations may not be fully updated yet to the latest draft > >>>> version. But I think it is safe to say that ICE works in the > >>>> intended core cases. If there are some corner cases where it > >>>> fails, that may be. But that is not information we will have until > >>>> really large scale deployement or someone makes very extensive > >>>> tests. > >>> > >>> Henry has, I believe, been very consistent in urging the IETF to > >>> have more "running code" experience on every specification (not > >>> just ICE) than our process requires. He has, IIRC, also advocated > >>> changing the process to require running code before publishing an > >>> RFC. He's even suggested at times that one should have running code > >>> before submitting a protocol draft. > >>> > >>> Another point to consider -- I've known Henry to bark at a tree > >>> with no squirrel in it occasionally, but most of the time he's > >>> pretty reliable, more so than most technology predictors. If he's > >>> worried about ICE, it's probably because he's been talking to > >>> people who are trying to implement it and they're telling him it is > >>> difficult or that they're having performance or reliability issues. > >>> I imagine that there are plenty of internal development teams in > >>> various companies that are working on this and not necessarily > >>> bringing their results back to the IETF. After all, if everybody > >>> sees the emperor as fully clothed, they'd be embarrassed to > >>> proclaim otherwise. Of course, I wish all of these developers were > >>> tied into the IETF, but then I'm sure I don't want to see every ice- > >>> implementor's question on the SIP list. That is why we have a sip- > >>> implementors mailing list, right? > >>> > >>> Personally, I think ICE is the best solution we have. We KNOW of > >>> cases where hole-punching doesn't work, and I think ICE will > >>> produce usable results in most of those cases. But that's just my > >>> personal opinion from reading the specs. I haven't implemented ICE, > >>> and I haven't used anybody else's implementation. Sure, it's > >>> complex, but that's because the underlying problem is complex -- > >>> there are just too many different behaviors out there in "NAT and > >>> firewall land". Of course, I could be wrong, too. It's happened > >>> before, once or twice ;-). Just how sure are we that we're right > >>> about this? > >>> > >>> Are we designing a "spruce goose" or a practical system? Only > >>> experience will tell us. But some good flight reports from the > >>> prototypes could go a long way towards reassuring us of the > >>> design's utility. > >>> > >>> So far we've had a couple of people report hearsay evidence -- that > >>> somebody else has an ICE implementation and it works. Any of you > >>> folks have personal experience you can share? Have there been > >>> prototypes exercised at the interop events? > >>> > >>> -- > >>> Dean > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> Behave mailing list > >>> [EMAIL PROTECTED] > >>> https://www1.ietf.org/mailman/listinfo/behave > >> > >> > >> _______________________________________________ > >> Behave mailing list > >> [EMAIL PROTECTED] > >> https://www1.ietf.org/mailman/listinfo/behave > > > > > > > > _______________________________________________ > > Sip mailing list https://www1.ietf.org/mailman/listinfo/sip > > This list is for NEW development of the core SIP Protocol > > Use [EMAIL PROTECTED] for questions on current sip > > Use [EMAIL PROTECTED] for new developments on the application of sip _______________________________________________ Sip mailing list https://www1.ietf.org/mailman/listinfo/sip This list is for NEW development of the core SIP Protocol Use [EMAIL PROTECTED] for questions on current sip Use [EMAIL PROTECTED] for new developments on the application of sip
