Hi,
In order to sort out this discussion regarding terminology, I have now ...
1. Reverted the main headline of the section to what it was in -08 and before
("Monitoring ALTO"),
2. Aligned the wording with draft-ietf-alto-protocol-26 Section 16.1.4 and
Section 16.2.5, under the assumption that the content and wording therein is WG
consensus
3. Added the desired sentence on third parties and trust issues to Section 3.4.1
4. Separated the discussion of "impact" and "performance" into two sections, as
suggested in this thread
Below is a copy of the updated, pre-10 text.
Please let me know if I missed something.
Thanks
Michael
*** draft-ietf-alto-deployments-010pre ***
3.4. Monitoring ALTO
3.4.1. Impact and Observation on Network Operation
ALTO presents a new opportunity for managing network traffic by
providing additional information to clients. In particular, the
deployment of an ALTO Server may shift network traffic patterns, and
the potential impact to network operation can be large. An ISP
providing ALTO may want to assess the benefits of ALTO as part of the
management and operations (cf. [I-D.ietf-alto-protocol]). For
instance, the ISP might be interested in understanding whether the
provided ALTO maps are effective, and in order to decide whether an
adjustment of the ALTO configuration would be useful. Such insight
can be obtained from a monitoring infrastructure. An NSP offering
ALTO could consider the impact on (or integration with) traffic
engineering and the deployment of a monitoring service to observe the
effects of ALTO operations. The measurement of impacts can be
challenging because ALTO-enabled applications may not provide related
information back to the ALTO Service Provider.
To construct an effective monitoring infrastructure, the ALTO Service
Provider should decide how to monitor the performance of ALTO and
identify and deploy data sources to collect data to compute the
performance metrics. In certain trusted deployment environments, it
may be possible to collect information directly from ALTO clients.
It may also be possible to vary or selectively disable ALTO guidance
for a portion of ALTO clients either by time, geographical region, or
some other criteria to compare the network traffic characteristics
with and without ALTO. Monitoring an ALTO service could also be
realized by third parties. In this case, insight into ALTO data may
require a trust relationship between the monitoring system operator
and the network service provider offering an ALTO service.
The required monitoring depends on the network infrastructure and the
use of ALTO, and an exhaustive description is outside the scope of
this document.
3.4.2. Measurement of the Impact
ALTO realizes an interface between the network and applications.
This implies that an effective monitoring infrastructure may have to
deal with both network and application performance metrics. This
document does not comprehensively list all performance metrics that
could be relevant, nor does it formally specify metrics.
The impact of ALTO can be classified regarding a number of different
criteria:
o Total amount and distribution of traffic: ALTO enables ISPs to
influence and localize traffic of applications that use the ALTO
service. An ISP may therefore be interested in analyzing the
impact on the traffic, i.e., whether network traffic patterns are
shifted. For instance, if ALTO shall be used to reduce the inter-
domain P2P traffic, it makes sense to evaluate the total amount of
inter-domain traffic of an ISP. Then, one possibility is to study
how the introduction of ALTO reduces the total inter-domain
traffic (inbound and/our outbound). If the ISPs intention is to
localize the traffic inside his network, the network-internal
traffic distribution will be of interest. Effectiveness of
localization can be quantified in different ways, e.g., by the
load on core routers and backbone links, or by considering more
advanced effects, such as the average number of hops that traffic
traverses inside a domain.
o Application performance: The objective of ALTO is improve
application performance. ALTO can be used by very different types
applications, with different communication characteristics and
requirements. For instance, if ALTO guidance achieves traffic
localization, one would expect that applications achieve a higher
throughput and/or smaller delays to retrieve data. Application-
specific performance characteristics (e.g., video or audio
quality) can be useful as well. In addition, selected statistics
from the TCP/IP stack in hosts can be useful, e.g., the number of
retransmitted TCP segments.
Of potential interest can also be the share of applications or
customers that actually use an offered ALTO service, i.e., the
adoption of the service.
Monitoring statistics can be aggregated, averaged, and normalized in
different ways. This document does not mandate specific ways how to
calculate metrics.
3.4.3. System and Service Performance
A number of interesting parameters can be measured at the ALTO
server. [I-D.ietf-alto-protocol] suggests certain ALTO-specific
metrics to be monitored:
o Requests and responses for each service listed in a Information
Directory (total counts and size in bytes).
o CPU and memory utilization
o ALTO map updates
o Number of PIDs
o ALTO map sizes (in-memory size, encoded size, number of entries)
This data characterizes the workload, the system performance as well
as the map data. Obviously, such data will depend on the
implementation and the actual deployment of the ALTO service.
3.4.4. Monitoring Infrastructure
Understanding the impact of ALTO may require interaction between
different systems, operating at different layers. Some information
discussed in the preceding sections is only visible to an ISP, while
application-level performance can hardly be measured inside the
network. It is possible that not all information of potential
interest can directly be measured, either because no corresponding
monitoring infrastructure or measurement method exists, or because it
is not easily accessible.
One way to quantify the benefit of deploying ALTO is to measure
before and after enabling the ALTO service. In addition to passive
monitoring, some data could also be obtained by active measurements,
but due to the resulting overhead, the latter should be used with
care. Yet, in all monitoring activities an ALTO service provider has
to take into account that ALTO Clients are not bound to ALTO Server
guidance as ALTO is only one source of information, and any
measurement result may thus be biased.
Potential sources for monitoring the use of ALTO include:
o Network Operations, Administration, and Maintenance (OAM) systems:
Many ISPs deploy OAM systems to monitor the network traffic, which
may have insight into traffic volumes, network topology, and
bandwidth information inside the management area. Data can be
obtained by SNMP, Netflow, IP Flow Information Export (IPFIX),
syslog, etc.
o Applications/clients: Relevant data could be obtained by
instrumentation of applications.
o ALTO server: If available, log files or other statistics data
could be analyzed.
o Other application entities: In several use cases, there are other
application entities that could provide data as well. For
instance, there may be centralized log servers that collect data.
In many ALTO use cases some data sources are located within an ISP
while some other data is gathered at application level. Correlation
of data would require a collaboration agreement between the ISP and
an application owner, including agreements of data interchange
formats, methods of delivery, etc. In practice, such a collaboration
may not be possible in all use cases of ALTO, because the monitoring
data can be sensitive, and because the interacting entities may have
different priorities. Details of how to build an over-arching
monitoring system for evaluating the benefits of ALTO are outside the
scope of this memo.
_______________________________________________
alto mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/alto
