Re: [alto] Opsdir last call review of draft-ietf-alto-cost-calendar-16
Thank you so much, Jürgen! The WG decision is to use a single unit and hence to allow scaling below the chosen unit, we have to deal with float. As much as we thought 1 second is typically the smallest, from my understanding, some use cases from a major app developer is already below this. We will revise the sentences to be reasonable and check with you to finalize say by Wednesday the latest. Thanks again, Richard On Mon, Feb 24, 2020 at 4:55 AM Schönwälder, Jürgen < j.schoenwael...@jacobs-university.de> wrote: > I will remove all closed issues and comment further inline. > > On Fri, Feb 21, 2020 at 03:54:30PM -0500, Y. Richard Yang wrote: > > > > > > - Since you write "SHOULD at least IEEE 754 double-precision > floating > > > > > point", does this not make the IEEE 754 reference a normative > > > > > reference? > > > > > > > > > > > > - [[SR]] The text follows RFC7285, section 11.3.2.3, in which > IEEE > > > 754 > > > > is an informative reference and cited with a SHOULD. > > > > > > > > > > On second thought, I wonder whether you really want to mandate how a > > > value is 'stored'. Perhaps you wanted to require 'SHOULD use at least > > > support IEEE 754 double-precision floating point [IEEE.754.2008] > > > precision'. > > > > > > Note that several lines up you suggested to change to integer, but I > > > am not sure you wanted that - please double check that you get this > > > right. Anyway, since you refer to properties of a concrete formats > > > defined in another document, it seems to me the reference is actually > > > normative. > > > > > > > > > In general, I find the double precision requirement a bit irritating > > > given the large range of numbers doubles can represent (roughly from > > > 2.2x10^-308 to 1.8x10^308). And yet, many practically useful numbers > > > such as 0.1 can't be represented exactly as floating point numbers, > > > i.e., there is always rounding involved, making comparisons rather > > > complicated. > > > > > > Often people want to have doubles in the hope that this maximizes the > > > chances of not getting hit by rounding issues but the sad truth is that > > > floating point numbers remain dangerous. > > > > > > > > > > > Totally agree. It is interesting that you use the 0.1 second example. I > > assume that you have the classical Patriot 0.1 second rounding bug in > mind ( > > https://www.viva64.com/en/b/0445/), which is quite close to our setting, > > where 0.1 is their interval size, although I do not foresee that ALTO > cost > > calendar will be used in such a setting :-) > > > > I believe that the initial design used only integers, with a unit (s, ms, > > ns, ...) to avoid the floating-point issue (instead of 0.1 sec, it is 100 > > ms), and then the feedback and WG decision is to use generic floating > > points to have a more uniform, single unit (second) design. One proposal, > > which I am not sure others like, is to use a smaller unit, say ms, and > then > > most likely we will deal with only integers (in the range of [-(2**53)+1, > > (2**53)-1]) and can still handle smaller units should the setting arise.. > > The downside is that there might be more zeros if the interval is longer > > (60 sec -> 6 ms). What do you think? > > > > Another possibility is to keep the current design but add some text to > > discuss the precision issue. A Proposal is the following: > > > > "time-interval-size": > > > > * is the duration of an ALTO Calendar time interval in seconds. > > A "time-interval-size" value contains a JSONNumber. ALTO > > Servers SHOULD use at least IEEE 754 double-precision floating > > point [IEEE.754.2008] to store this value. Example values are: > > 300 , 7200, meaning that each Calendar value applies on a time > > interval that lasts 5 minutes and 2 hours, respectively. > > > > => > > > > "time-interval-size": > > > > * is the duration of an ALTO Calendar time interval in seconds. > > A "time-interval-size" value contains a JSONNumber. Example > > values > > are: 300 , 7200, meaning that each Calendar value applies on a > time > > interval that lasts 5 minutes and 2 hours, respectively. Since > the > > interval size can be smaller than 1 second (e.g., 100 ms), the > > value > > specified can also be a floating point (e.g., 0.1). Both ALTO > > clients and > > servers should be aware of potential issues caused by the > precision > > issues caused by using floating point numbers. For example, > > the server may have an internal, integer representation in ms, > and > > store > > 100 as an int (0.1 sec) in the local storage; the sever sends > "0.1" > > as > > the interval size to the client; the client may use a > float/double > > to store > > the JSON number, which may not represent 0.1 precisely. To > improve > > interoperability, the server and the client SHOULD use IEEE
Re: [alto] Opsdir last call review of draft-ietf-alto-cost-calendar-16
I will remove all closed issues and comment further inline. On Fri, Feb 21, 2020 at 03:54:30PM -0500, Y. Richard Yang wrote: > > > > - Since you write "SHOULD at least IEEE 754 double-precision floating > > > > point", does this not make the IEEE 754 reference a normative > > > > reference? > > > > > > > > > - [[SR]] The text follows RFC7285, section 11.3.2.3, in which IEEE > > 754 > > > is an informative reference and cited with a SHOULD. > > > > > > > On second thought, I wonder whether you really want to mandate how a > > value is 'stored'. Perhaps you wanted to require 'SHOULD use at least > > support IEEE 754 double-precision floating point [IEEE.754.2008] > > precision'. > > > > Note that several lines up you suggested to change to integer, but I > > am not sure you wanted that - please double check that you get this > > right. Anyway, since you refer to properties of a concrete formats > > defined in another document, it seems to me the reference is actually > > normative. > > > > > > In general, I find the double precision requirement a bit irritating > > given the large range of numbers doubles can represent (roughly from > > 2.2x10^-308 to 1.8x10^308). And yet, many practically useful numbers > > such as 0.1 can't be represented exactly as floating point numbers, > > i.e., there is always rounding involved, making comparisons rather > > complicated. > > > > Often people want to have doubles in the hope that this maximizes the > > chances of not getting hit by rounding issues but the sad truth is that > > floating point numbers remain dangerous. > > > > > > > Totally agree. It is interesting that you use the 0.1 second example. I > assume that you have the classical Patriot 0.1 second rounding bug in mind ( > https://www.viva64.com/en/b/0445/), which is quite close to our setting, > where 0.1 is their interval size, although I do not foresee that ALTO cost > calendar will be used in such a setting :-) > > I believe that the initial design used only integers, with a unit (s, ms, > ns, ...) to avoid the floating-point issue (instead of 0.1 sec, it is 100 > ms), and then the feedback and WG decision is to use generic floating > points to have a more uniform, single unit (second) design. One proposal, > which I am not sure others like, is to use a smaller unit, say ms, and then > most likely we will deal with only integers (in the range of [-(2**53)+1, > (2**53)-1]) and can still handle smaller units should the setting arise. > The downside is that there might be more zeros if the interval is longer > (60 sec -> 6 ms). What do you think? > > Another possibility is to keep the current design but add some text to > discuss the precision issue. A Proposal is the following: > > "time-interval-size": > > * is the duration of an ALTO Calendar time interval in seconds. > A "time-interval-size" value contains a JSONNumber. ALTO > Servers SHOULD use at least IEEE 754 double-precision floating > point [IEEE.754.2008] to store this value. Example values are: > 300 , 7200, meaning that each Calendar value applies on a time > interval that lasts 5 minutes and 2 hours, respectively. > > => > > "time-interval-size": > > * is the duration of an ALTO Calendar time interval in seconds. > A "time-interval-size" value contains a JSONNumber. Example > values > are: 300 , 7200, meaning that each Calendar value applies on a time > interval that lasts 5 minutes and 2 hours, respectively. Since the > interval size can be smaller than 1 second (e.g., 100 ms), the > value > specified can also be a floating point (e.g., 0.1). Both ALTO > clients and > servers should be aware of potential issues caused by the precision > issues caused by using floating point numbers. For example, > the server may have an internal, integer representation in ms, and > store > 100 as an int (0.1 sec) in the local storage; the sever sends "0.1" > as > the interval size to the client; the client may use a float/double > to store > the JSON number, which may not represent 0.1 precisely. To improve > interoperability, the server and the client SHOULD use IEEE 754 > double-precision floating point [IEEE.754.2008] to store this > value, in unit >of seconds. > > I am a bit ambivalent of the last sentence though. This is something the WG has to discuss and decide. If the WG can find agreement that there is a smallest sensible unit (e.g., milliseconds or microseconds), then I personally would find it reasonable to implement this specification by storing values in an integral type of this smallest sensible unit internally (if the WG does not want to use integral numbers of that type). Hence, for me personally, a statement how implementations SHOULD store values goes too far as this is an implementation concern. /js -- Juergen Schoenwaelder
Re: [alto] Opsdir last call review of draft-ietf-alto-cost-calendar-16
Dear Jürgen, Sorry for the late reply, as our lead author is off without a stable connection. Please see below. On Wed, Feb 19, 2020 at 4:38 AM Schönwälder, Jürgen < j.schoenwael...@jacobs-university.de> wrote: > Dear authors, > > I will comment inline... > > On Tue, Feb 18, 2020 at 06:39:04PM -0500, Y. Richard Yang wrote: > > Dear Jürgen, > > > > Thank you so much for the careful reviews. The lead author of the > document > > is Sabine and the response below is formulated by her and then discussed > > among the authors. Since she is off this week, so let me send the > response > > on her behalf to make progress. > > > > Her response for an item will start with [[SR]] and end with > > > > Please see below. > > > > On Tue, Feb 11, 2020 at 4:49 AM Jürgen Schönwälder via Datatracker < > > nore...@ietf.org> wrote: > > > > > Reviewer: Jürgen Schönwälder > > > Review result: Has Issues > > > > > > I am not an ALTO expert so keep this in mind in case some of my > > > comments have little meaning for ALTO experts. > > > > > > - If multiple clients retrieve information about costs in the future > > > and take independent scheduling decisions based on that information, > > > it might turn out that a "cheap" time turns into "expensive" time, > > > i.e., the result may not be an improvement of QoE but rather the > > > opposite. How is this dealt with? I see this issue touched on in > > > some places and the suggestion seems to be the advice to implement > > > SSE. But should ALTO servers make promises long into the future if > > > clients do not implement SSE? > > > > > > I am concerned that announcements of "cheap" times can easily turn > > > into instabilities if many clients start to opt for the announced > > > "cheap" times and I think this deserves to be discussed explicitly > > > in the Operational Considerations section. > > > > > > > > Good comment! We will do some reorganization to more explicitly, > > systematically discuss this issue. In particular, we plan to make > explicit > > the following considerations in the Operations Considerations section > > (Section 8): > > > > PROPOSE TEXT: > > It is important that both the operator of the network and the operator of > > the applications consider both the feedback aspect and the > prediction-based > > (uncertainty) aspect of using the cost calendar. Consider the cost > calendar > > as a traffic-aware map service (e.g., Google map). Using the service > > without considering its own effect, a large fleet can turn a > not-congested > > road into a congested one; a large number of individual cars each > choosing > > a road with light traffic ("cheap link") can also result in congestion or > > result in a less optimal global outcome (e.g., the Braess's Paradox > > [citation]); an accident may happen and hence cause unexpected delays. > > > > Although individual network operators and application operators can > choose > > their own approaches to address the aforementioned issues, this document > > recommends the following considerations. First, a typical approach to > > reducing instability and handling uncertainty is to ensure timely update > of > > information. Hence, the "ALTO Incremental Updates Using Server-Sent > Events > > (SSE)" Service [draft-ietf-alto-incr-update-sse] SHOULD be used to update > > the calendar faster if supported by both the server and the client." > > Second, Although there are theoretical analysis [RoughgardenThesis] and > > Internet based evaluations [SIGCOMM2013SelfishRouting] showing that > > uncoordinated behaviors may not result in substantial sub-optimal > > results, when a network operator updates the cost calendar, and when an > > application reacts to the update, they should still consider the > potential > > feedback effects. > > Thanks. I was looking for something like this. > > Great. We will add the text. > > > - p6: "time zone (in UTC)" ?? do you mean "time zone offset relative > > > to UTC"? After reading, I think this is misleading. Perhaps you > > > wanted to say "start time (including time zone)" > > > > > > - [[SR]] > > As the time zone in the provided start time is always UTC+0, would the > > following update be OK?: > > > > - remove text item "time zone (in UTC)" > > - update the "calendar-start-date" description in the next paragraph as > > follows: > > > > OLD > > "calendar-start-date": specifying when the Calendar starts, that is to > > which date the first value of the Cost Calendar is applicable. > > > > NEW > > "calendar-start-date": specifying when the Calendar starts, that is to > > which date the first value of the Cost Calendar is > > applicable. The reference time zone for the provided time values is UTC.. > > - > > Is this not 'calendar-start-time' later on? Or are calendar-start-date > and calendar-start-time two different things? > > Good point. I believe that they are the same. Let us unify. > If times are always in UTC, then why not say: > >
Re: [alto] Opsdir last call review of draft-ietf-alto-cost-calendar-16
Dear authors, I will comment inline... On Tue, Feb 18, 2020 at 06:39:04PM -0500, Y. Richard Yang wrote: > Dear Jürgen, > > Thank you so much for the careful reviews. The lead author of the document > is Sabine and the response below is formulated by her and then discussed > among the authors. Since she is off this week, so let me send the response > on her behalf to make progress. > > Her response for an item will start with [[SR]] and end with > > Please see below. > > On Tue, Feb 11, 2020 at 4:49 AM Jürgen Schönwälder via Datatracker < > nore...@ietf.org> wrote: > > > Reviewer: Jürgen Schönwälder > > Review result: Has Issues > > > > I am not an ALTO expert so keep this in mind in case some of my > > comments have little meaning for ALTO experts. > > > > - If multiple clients retrieve information about costs in the future > > and take independent scheduling decisions based on that information, > > it might turn out that a "cheap" time turns into "expensive" time, > > i.e., the result may not be an improvement of QoE but rather the > > opposite. How is this dealt with? I see this issue touched on in > > some places and the suggestion seems to be the advice to implement > > SSE. But should ALTO servers make promises long into the future if > > clients do not implement SSE? > > > > I am concerned that announcements of "cheap" times can easily turn > > into instabilities if many clients start to opt for the announced > > "cheap" times and I think this deserves to be discussed explicitly > > in the Operational Considerations section. > > > > > Good comment! We will do some reorganization to more explicitly, > systematically discuss this issue. In particular, we plan to make explicit > the following considerations in the Operations Considerations section > (Section 8): > > PROPOSE TEXT: > It is important that both the operator of the network and the operator of > the applications consider both the feedback aspect and the prediction-based > (uncertainty) aspect of using the cost calendar. Consider the cost calendar > as a traffic-aware map service (e.g., Google map). Using the service > without considering its own effect, a large fleet can turn a not-congested > road into a congested one; a large number of individual cars each choosing > a road with light traffic ("cheap link") can also result in congestion or > result in a less optimal global outcome (e.g., the Braess's Paradox > [citation]); an accident may happen and hence cause unexpected delays. > > Although individual network operators and application operators can choose > their own approaches to address the aforementioned issues, this document > recommends the following considerations. First, a typical approach to > reducing instability and handling uncertainty is to ensure timely update of > information. Hence, the "ALTO Incremental Updates Using Server-Sent Events > (SSE)" Service [draft-ietf-alto-incr-update-sse] SHOULD be used to update > the calendar faster if supported by both the server and the client." > Second, Although there are theoretical analysis [RoughgardenThesis] and > Internet based evaluations [SIGCOMM2013SelfishRouting] showing that > uncoordinated behaviors may not result in substantial sub-optimal > results, when a network operator updates the cost calendar, and when an > application reacts to the update, they should still consider the potential > feedback effects. Thanks. I was looking for something like this. > > - p6: "time zone (in UTC)" ?? do you mean "time zone offset relative > > to UTC"? After reading, I think this is misleading. Perhaps you > > wanted to say "start time (including time zone)" > > > > - [[SR]] > As the time zone in the provided start time is always UTC+0, would the > following update be OK?: > > - remove text item "time zone (in UTC)" > - update the "calendar-start-date" description in the next paragraph as > follows: > > OLD > "calendar-start-date": specifying when the Calendar starts, that is to > which date the first value of the Cost Calendar is applicable. > > NEW > "calendar-start-date": specifying when the Calendar starts, that is to > which date the first value of the Cost Calendar is > applicable. The reference time zone for the provided time values is UTC. > - Is this not 'calendar-start-time' later on? Or are calendar-start-date and calendar-start-time two different things? If times are always in UTC, then why not say: The calendar-start-(date|time) is always expressed in UTC. This way the reader does not have to think about what a reference time zone might be. > > - p7: broken sentence > > > >The extensions in this document and encode requests and responses > >using JSON [RFC8259]. > > > > - [[SR]] > yes, thanks: "and" should be removed. New text will be: > "The extensions in this document encode requests and responses using JSON > [RFC8259]." > - OK > > - p7: editorial nit > > > > OLD > > > > this
Re: [alto] Opsdir last call review of draft-ietf-alto-cost-calendar-16
Dear Jürgen, Thank you so much for the careful reviews. The lead author of the document is Sabine and the response below is formulated by her and then discussed among the authors. Since she is off this week, so let me send the response on her behalf to make progress. Her response for an item will start with [[SR]] and end with Please see below. On Tue, Feb 11, 2020 at 4:49 AM Jürgen Schönwälder via Datatracker < nore...@ietf.org> wrote: > Reviewer: Jürgen Schönwälder > Review result: Has Issues > > I am not an ALTO expert so keep this in mind in case some of my > comments have little meaning for ALTO experts. > > - If multiple clients retrieve information about costs in the future > and take independent scheduling decisions based on that information, > it might turn out that a "cheap" time turns into "expensive" time, > i.e., the result may not be an improvement of QoE but rather the > opposite. How is this dealt with? I see this issue touched on in > some places and the suggestion seems to be the advice to implement > SSE. But should ALTO servers make promises long into the future if > clients do not implement SSE? > > I am concerned that announcements of "cheap" times can easily turn > into instabilities if many clients start to opt for the announced > "cheap" times and I think this deserves to be discussed explicitly > in the Operational Considerations section. > > Good comment! We will do some reorganization to more explicitly, systematically discuss this issue. In particular, we plan to make explicit the following considerations in the Operations Considerations section (Section 8): PROPOSE TEXT: It is important that both the operator of the network and the operator of the applications consider both the feedback aspect and the prediction-based (uncertainty) aspect of using the cost calendar. Consider the cost calendar as a traffic-aware map service (e.g., Google map). Using the service without considering its own effect, a large fleet can turn a not-congested road into a congested one; a large number of individual cars each choosing a road with light traffic ("cheap link") can also result in congestion or result in a less optimal global outcome (e.g., the Braess's Paradox [citation]); an accident may happen and hence cause unexpected delays. Although individual network operators and application operators can choose their own approaches to address the aforementioned issues, this document recommends the following considerations. First, a typical approach to reducing instability and handling uncertainty is to ensure timely update of information. Hence, the "ALTO Incremental Updates Using Server-Sent Events (SSE)" Service [draft-ietf-alto-incr-update-sse] SHOULD be used to update the calendar faster if supported by both the server and the client." Second, Although there are theoretical analysis [RoughgardenThesis] and Internet based evaluations [SIGCOMM2013SelfishRouting] showing that uncoordinated behaviors may not result in substantial sub-optimal results, when a network operator updates the cost calendar, and when an application reacts to the update, they should still consider the potential feedback effects. > - p6: "time zone (in UTC)" ?? do you mean "time zone offset relative > to UTC"? After reading, I think this is misleading. Perhaps you > wanted to say "start time (including time zone)" > > - [[SR]] As the time zone in the provided start time is always UTC+0, would the following update be OK?: - remove text item "time zone (in UTC)" - update the "calendar-start-date" description in the next paragraph as follows: OLD "calendar-start-date": specifying when the Calendar starts, that is to which date the first value of the Cost Calendar is applicable. NEW "calendar-start-date": specifying when the Calendar starts, that is to which date the first value of the Cost Calendar is applicable. The reference time zone for the provided time values is UTC. - > - p7: broken sentence > >The extensions in this document and encode requests and responses >using JSON [RFC8259]. > > - [[SR]] yes, thanks: "and" should be removed. New text will be: "The extensions in this document encode requests and responses using JSON [RFC8259]." - > - p7: editorial nit > > OLD > > this document extends: the IRD, the ALTO > > NEW > > this document extends the IRD and the ALTO > - [[SR]] thanks, and we will update. > - I am a bit concerned about changing the type of json fields, i.e., > from a number to an array of numbers. The ALTO specifications may > allow this but is there a certain risk of implementations not being > smart in handling this correctly? Were alternatives considered that > do not change the type of existing json fields? Given the explicit > text why this is OK from an ALTO perspective, it might be that the > WG did iterate on this (sorry if I raise a closed issue again). > > - [[SR]] Indeed the ALTO WG did
[alto] Opsdir last call review of draft-ietf-alto-cost-calendar-16
Reviewer: Jürgen Schönwälder Review result: Has Issues I am not an ALTO expert so keep this in mind in case some of my comments have little meaning for ALTO experts. - If multiple clients retrieve information about costs in the future and take independent scheduling decisions based on that information, it might turn out that a "cheap" time turns into "expensive" time, i.e., the result may not be an improvement of QoE but rather the opposite. How is this dealt with? I see this issue touched on in some places and the suggestion seems to be the advice to implement SSE. But should ALTO servers make promises long into the future if clients do not implement SSE? I am concerned that announcements of "cheap" times can easily turn into instabilities if many clients start to opt for the announced "cheap" times and I think this deserves to be discussed explicitly in the Operational Considerations section. - p6: "time zone (in UTC)" ?? do you mean "time zone offset relative to UTC"? After reading, I think this is misleading. Perhaps you wanted to say "start time (including time zone)" - p7: broken sentence The extensions in this document and encode requests and responses using JSON [RFC8259]. - p7: editorial nit OLD this document extends: the IRD, the ALTO NEW this document extends the IRD and the ALTO - I am a bit concerned about changing the type of json fields, i.e., from a number to an array of numbers. The ALTO specifications may allow this but is there a certain risk of implementations not being smart in handling this correctly? Were alternatives considered that do not change the type of existing json fields? Given the explicit text why this is OK from an ALTO perspective, it might be that the WG did iterate on this (sorry if I raise a closed issue again). - p9: can a time-interval-size be negative? can it be zero? - p9: what does 'at least equal to 1' mean? Do you mean 'a positive integer number of values'? - p14: I am confused about the time zone aspect. The time zone pops up in section 3.2 but it is a bit unclear what is meant there (see above). On page 14, there is a statement about a 'reference time zone' (what is this?) and then there is text about HTTP header fields, and I am lost how that format relates to the rest of the document. Was the idea to say that calendar-start-time uses the HTTP header time format? Then say this where calendar-start-time is defined. And why do you use the HTTP format and not RFC 3339 format? Perhaps this data format for date and time is popular in ALTO and hence it makes sense, I guess I do not know enough about it. Anyway, the time zone seems to be part of the calendar-start-time - if so this was not clear while reading top-down. And if there is free choice for the data and time format, I would find RFC 3339 probably a good robust alternative. - p17: Why is this SHOULD needed? Because you want to ensure that there is always a value that can be used 'right now'? What about values that are stale, i.e., calendar-start-time is way in the past? [...] The value provided for the "calendar- start-time" attribute SHOULD NOT be later than the request date. - p17: I am confused by the description of "repeated". Is the intent to say that the calendar value array simply repeats N times? Am I right in assuming that if "repeated" is not present, it defaults to 1? Or does "repeated" always have to be present? (Similarly for number-of-intervals, can it be absent and then it defaults to 1? - In which sense is draft-ietf-alto-incr-update-sse normative given that the current text says 'can be used' but does not mandate its use? In which sense is RFC 8259 normative? Is it required to make ALTO clients and servers switch to RFC 8259? The Operational Considerations section says "RECOMMENDED" to switch. But how does an implementation work that does both RFC7285 (w/o UTF-8) and RFC8259 (with UTF-8)? I think a clearer message that ALTO clients and servers implementing this extension must comply to RFC 8259 is desirable. - Since you write "SHOULD at least IEEE 754 double-precision floating point", does this not make the IEEE 754 reference a normative reference? Similarly, if you import the data format from HTTP/1.1, does this not make the HTTP/1.1 RFC a normative reference? ___ alto mailing list alto@ietf.org https://www.ietf.org/mailman/listinfo/alto