Hi Qui, inline.
> -----Original Message----- > From: Qin Wu [mailto:[email protected]] > Sent: Monday, February 17, 2014 9:13 AM > To: Scharf, Michael (Michael) > Cc: IETF ALTO > Subject: RE: Review of draft-ietf-alto-deployments-08#appendix-A > > Hi,Michael: > Thanks for your clarification, please see my followup comments. > > Regards! > -Qin > -----Original Message----- > >>From: Scharf, Michael (Michael) [mailto:michael.scharf@alcatel- > lucent.com] > >>Sent: Sunday, February 16, 2014 3:55 AM > >>To: Qin Wu > >>Cc: IETF ALTO > >>Subject: RE: Review of draft-ietf-alto-deployments-08#appendix-A > > >>Hi Qin, > > >>Please find some comments inline <MSC/>. > > >>From: Qin Wu [mailto:[email protected]] > >>Sent: Saturday, February 15, 2014 4:57 AM > >>To: Scharf, Michael (Michael) > >>Cc: IETF ALTO > >>Subject: RE: Review of draft-ietf-alto-deployments-08#appendix-A > > >>Hi, Michael: > >>Thanks for re-work and updating to monitoring section. > >>I have a few comments to section 3.4 you updated. > >>1. The monitoring infrastructure can be third party,Or can be > integrated into ALTO server? If it is the third party based, I think > the trust > >>relationship between monitoring system and alto server needs to be > established, otherwise, security concern may be raised. > > <MSC> > >I think the most realistic case it that it is neither third party, nor > an integrated function of the ALTO server. Instead, a network service > provider > >would probably monitor the benefits of its ALTO service by a > monitoring infrastructure, which could be an extension of the general > network monitoring > >OAM. It could be possible that third parties (e.g., academic > researchers) want to monitor the benefits as well; in that case, there > is indeed a question > >whether they would have access to network service provider data beyond > what the ALTO service provider exposes. However, since the IETF does > not mandate > >how to implement an ALTO server, an implementer could obviously also > come up with a huge all-in-one ALTO server implementation that realizes > many other > >functions beyond providing the ALTO service itself. > > >I could add a sentence like "Monitoring the benefits of an ALTO > service could also be realized by third parties. In this case, insight > into ALTO data > >may require a trust relationship between the monitoring system > operator and the network service provider offering an ALTO service." > Would this address > >your comment? > > </MSC> > > [Qin]:Yes, thanks. OK > >>2. Section 4.3 says: > " > >>An NSP offering ALTO could consider the impact on (or integration > with) traffic > >> engineering and the deployment of a monitoring service to observe > the > >> effects of ALTO operations. > " > > >>Traffic engineering is related to monitoring ALTO performance? I > think you may gather traffic engineering information from routing > protocol, these > >>traffic engineering information can be used to reflect network > performance and provide fine granularity of cost map or network map? > However it has > >>nothing to do with performance of ALTO? > >>Performance of ALTO is more about how many request or response can be > handled, > > <MSC> > >This sentence is taken from Section 16.1.4 of the ALTO protocol spec > (http://tools.ietf.org/html/draft-ietf-alto-protocol-25#page-81). > Actually, the > >whole scope of Section 3.4 in draft-ietf-alto-deployment is explained > in Section 16.1.4 of the ALTO protocol spec. Given that explanation > there, I am > >not sure if I understand your question and/or concern. Specifically, > the whole section 3.4 of draft-ietf-alto-deployment deals NOT with how > to retrieve > >routing protocol information (including e.g. TE information from IGP) > and how to put it into network and cost maps. This is discussed in > Section 3.2.1. > > [Qin]: The confusion comes from the title of section 3.4 of draft-ietf- > alto-deployments-08,i.e.," Monitoring the Performance of ALTO " since > ALTO performance is more about resource oriented objective, e.g, > CPU,memory, request/responses to be handled, ALTO performance has > nothing to do traffic oriented objective. Well, as I already explained, ALTO expands as Application-layer Topology Optimization, which is an abstract concept, not its implementation in a server. In my understanding the term "Performance of ALTO" is *NOT* the same like "Performance of an ALTO server/service", and the text in that section explains that multiple times. Having said this, I already offered to change this term. > >Let me try to explain how I understand draft-ietf-alto-protocol-25: > This sentence deals with understanding what impact the ALTO guidance > has on > >applications, and whether the ALTO service operation is efficient and > effectively improving "better-than-random" resource selections by > applications. I > >understand here "traffic engineering" in a very general way. For > instance, an NSP offering an ALTO service for P2P applications would > typically expect > >that the amount of aggregated inter-domain traffic is reduced. > > [Qin]: The goal of Traffic engineering is to facilitate efficient and > reliable network operation while simultaneously optimization network > resource > Utilization and traffic performance. Inter-domain traffic engineering > in some cases can be used to reduce traffic between domain, but in most > of cases, > Traffic engineering will be use for traffic performance optimization or > load balancing of the traffic. I am not sure traffic localization is > related to > Traffic engineering. If it is, I think it is one aspect of traffic > performance optimization. Thus, what do you suggest to change specifically in draft-ietf-alto-deployments-09? I only copied the sentence from draft-ietf-alto-protocol-25. I think your definition of traffic engineering "facilitate efficient and reliable network operation while simultaneously optimization network resource Utilization and traffic performance" perfectly describes the key objectives of deploying an ALTO server, e.g., for P2P overlay optimization: - "Network efficiency" is improved (e.g., by less inter-domain traffic, less peering costs, etc.) - "Network reliability" is improved (e.g., by less risk of congesting the network by suboptimal P2P overlay topologies) - "Network utilization" is optimized (e.g., by less network load on core links) - "Traffic performance" is improved (e.g., higher P2P download throughput With all aspects of your definition of "traffic engineering", the sentence in draft-ietf-alto-protocol-25 "Thus, an ALTO Service Provider should consider impacts on (or integration with) traffic engineering and the deployment of a monitoring service to observe the effects of ALTO operations. Note that ALTO-specific monitoring and metrics are discussed in [I-D.ietf-alto-deployments]." Makes perfectly sense. What do I miss? > >If ALTO is effective, the NSP can probably slow down upgrades of > Internet-peering-point capacities; this is traffic engineering, but > that kind of > >information may not be carried in, or retrieved from routing > protocols, and it may not be used to calculate ALTO maps. > > [Qin]: If you are referring to bandwidth capability, it definitely > carried using routing protocol, e.g.,ISIS-TE, OSPF-TE, > Can you tell me what kind of information you are referred to? I might be wrong, I think the root cause of our mis-understanding is that you apparently assume that "traffic engineering" is equivalent to RSVP-TE, ISIS-TE, OSPF-TE, etc. An Internet Service Provider can run an entire IP network without enabling ISIS-TE, OSPF-TE, BGP-TE, etc., right? For instance, if that network doesn't use any MPLS? In that case, as far as I understand routing protocols, no "bandwidth capabilities" will be carried in the routing protocols, right? That ISP can still optimize routing protocol link weights, modify the routing import/export policies, turn up/down interfaces, etc. In other words, an ALTO service provider has many means to influence how traffic is routed in his network, and he doesn't have to rely on ISIS-TE, OSPF-TE, etc. for that. Also, please note that bandwidth information can come from a centralized inventory system/database instead of ISIS-TE, OSPF-TE, etc. As example, have a look at draft-farrkingel-pce-abno-architecture-07 Section 2.3.1.8.1. And I might be wrong, but the SDN research community argues right now that one build traffic engineering solutions without using RSVP-TE, ISIS-TE, OSPF-TE, BGP-TE, etc. In summary: "traffic engineering" != data in routing protocols, right? > > If traffic is kept local in access network domains, the capacity > planning and traffic engineering of core network links can be adapted, > because they > >may have less utilization, e.g., bandwidth allocations of core links > could be adapt. This can be part of the typical mid-/long-term capacity > planning > >processes of a network service provider. > > >Furthermore, please note the ALTO protocol spec has another section > (Section 16.2.5. Performance Management) that describes examples for > system metrics > >characterizing the performance of an ALTO server (e.g., number of > requests, CPU/memory utilization, etc.). > > [Qin]: I think systems metrics described in example of section 16.2.5 > of ALTO protocol spec are the only metrics that characterize the > performance of an ALTO server and make sense to me. > > >If you struggle with the term "ALTO performance" in the headline, we > could perhaps better align the section headlines to draft-ietf-alto- > protocol-25, > >i.e.: > > >OLD: > >3.4. Monitoring the Performance of ALTO > >3.4.1. Supervising the Benefits of ALTO > >3.4.2. How to Monitor ALTO Performance > > >NEW: > >3.4. Monitoring the Impact of ALTO > >3.4.1. Supervising the Benefits of ALTO > >3.4.2. How to Observe the Impact of ALTO > > >Would this address your question? > > [Qin]:Not enough, if you change " Performance of ALTO " to " Impact of > ALTO ", I am wondering what is impacted by the ALTO server, then > I think the last two bullets in the section 3.4.2 should be moved to > somewhere else or removed,i.e., > " > o ALTO system performance: As mentioned in [I-D.ietf-alto- > protocol], > there are a number of interesting parameters that can be measured > at an ALTO server, including the Requests and responses for each > service listed in a Information Directory (total counts and size > in bytes) or the CPU and memory utilization. Also, the > characteristics of the ALTO maps can be monitored as well, e.g., > regarding the frequency of ALTO map updates, the number of PIDs, > or the ALTO map sizes (in-memory size, encoded size, number of > entries). > > o ALTO service utilization: Of potential interet can be the share > of > applications or customers that actually use an offered ALTO > service, i.e., the degree of adoption. I do not agree that a removal makes sense. There are ALTO system performance metrics and ALTO service utilization metrics that allow characterizing the "impact" of ALTO on a network. For instance, assume that an ISP runs several access networks, but he suspects that ALTO is not effectively used in one of them (e.g., because there is much less traffic locality in this access network than in others). How would he probably dig into that problem? If he runs the ALTO server, he could easily look into the distribution of requests and their mapping to access networks (ALTO system performance data, right?). If he observes that from one access network only few requests arrive, he can confirm that ALTO has hardly has any "impact" in that specific access network. Maybe there is a connectivity problem to the ALTO server in that network. Or maybe there is another problem, e.g., the users in those network don't opt-in to using said ALTO server. In any case, in this example, the NSP has used ALTO system performance data to understand why the ALTO deployment and its "impact" is not as efficient as expected. I do not claim that the existing text is already perfect. Specifically, the text may not be comprehensive right now. But please suggest additional specific text if you think that the wording is not clear, instead of asking for removal of monitoring methods that are already introduced and motivated in draft-ietf-alto-protocol. Michael > " > I don't think ALTO system performance is impacted by using a ALTO > sever. > ALTO service utilization seems just show the popularity of ALTO service > deployment. > > If you still use the old title for the section 3.4,i.e.," Monitoring > the Performance of ALTO ", > I think the first two bullets and the last bullet should be removed. > > > >>3. Section 3.4.2 says: > " > >>Total amount and distribution of traffic: ALTO enables ISPs to > >>influence and localize traffic of applications that use the ALTO > >>service. An ISP may therefore be interested in analyzing the > >> impact on the traffic, i.e., whether network traffic patterns are > >>shifted. For instance, if ALTO shall be used to reduce the inter- > >> domain P2P traffic, it makes sense to evaluate the total amount of > >>inter-domain traffic of an ISP. > " > >>How traffic distribution is related to ALTO performance monitoring, I > can not understand this. > > <MSC> > >Again, please refer to Section 16.1.4 in draft-ietf-alto-protocol-25. > That document states "Deployment of an ALTO Server may shift network > traffic > >patterns.". This paragraph just explains examples how one can monitor > whether ALTO achieves the shift of traffic distribution that an NSP > wants to > >achieve by ALTO. > > [Qin]: See above. > > </MSS> > > > >>4. Section 3.4.2 says: > " > > >>Application performance: The objective of ALTO is improve > >> application performance. ALTO can be used by very different > types > >> applications, with different communication characteristics and > >> requirements. For instance, if ALTO guidance achieves traffic > >> localization, one would expect that applications achieve a > higher > >> throughput and/or smaller delays to retrieve data. > Application- > >> specific performance characteristics (e.g., video or audio > >> quality) can be useful as well. In addition, selected > statistics > >> from the TCP/IP stack in hosts can be useful, e.g., the number > of > >> retransmitted TCP segments. > " > >>It is not clear to me whether you measure traffic performance or ALTO > performance? > >>I think application performance you described here has nothing to do > with ALTO performance. > >>Only ALTO system performance is related to ALTO performance. I feel > you mix traffic performance > >>With ALTO performance. What am I missing? > > <MSC> > >The draft explains the categories "total amount and distribution of > traffic", "application performance", and "ALTO system performance", and > they all > >refer to different aspects. For the headline, I was looking for a > short term that encompasses all different aspects, and I came up with > "ALTO > >performance" (obviously, this is not the ALTO system performance). If > you expand it as "application-layer topology optimization performance", > in my > >understanding "ALTO performance" refers to how efficient and effective > the application topologies are optimized by deploying an ALTO service. > > >However, as already mentioned, if the term "ALTO performance" should > be confusing, I could use the term "impact of ALTO" instead of "ALTO > performance", > >i.e., rename the section to "3.4.2. How to Observe the Impact of > ALTO". In that case, it should be clear that it deals with much more > than ALTO system > >performance, "traffic performance", etc. Would this solve your > question? > > [Qin]:See above. > > </MSC> > > Thanks > > Michael _______________________________________________ alto mailing list [email protected] https://www.ietf.org/mailman/listinfo/alto
