Re: Discrete interpolation attribute
Hi there, Still mumbling about interpolation options ... IMO, many options may decrease clarity. As far the options are sufficient to the pursued function, I prefer to stay low with options. I think that the most usual implementation of rounding is is half-up-simmetric, also known as half-away-from-zero: 0.5 --> 1.0 1.5 --> 2.0 -0.5 --> -1.0 At least, this is what ISO/IEC 9899 states for the C math library. Note that, because DAVE-ML asks for monotonically increasing breakpoints, "up" refers both to position and value. It's a little precisation but may be useful for clarity (after all free us from the simmetry problem). Round the half point to the nearest even (or odd) integer is the IEEE 754 (aka Banker's): 0.5 --> 0.0 1.5 --> 2.0 This is because rounding always up may bias the results, an issue in statistical applications or when lot of values are added. But, as Randy says, it's hard to see how we can use it here, as for the even test we have to consider not the values but the positions of the breakpoints, and this seems not very intuitive. So, I suggest half-up for the nearest option of the discrete interpolation method. IMO two options (floor and nearest) are more than sufficient. In the other cases, I think that is fair to ask the modeller to redefine the function (by rearranging the whole table, that includes both values and breakpoint set, as Dan highlighted). I bet nobody will need an half-even nearest, but an half-down nearest may be a possible request and a ceil is probable. Because ceil is probable (but my personal feeling is that is not very intuitive), and because breakpoint set may be shared by different tables I agree that the three options (ceil, floor and nearest) may be a more friendly choice. Last few points, and maybe more interesting than my above chatter :) The current documentation does not define which is the default when the value for interpolation is not specified. I suppose linear interpolation is the default. When extrapolation is admitted for one or both sides (default is ...) what method is used? The same for interpolation seems intuitive. In this case how do behave discrete methods? Ceil for min and floor for max (or nearest wrt +/-infinite). Being the interpolation method associated to the independent variables, when functions have more than one independent variable, it is possible to specify different methods? Is it common/useful? Ciao, Giovanni.
Re: Discrete interpolation attribute
I'd like to point out that the "nearest neighbour" option may not always do what a user intends. This is because DAVE-ML gridded data need not be uniformly spaced. Dividing a non-uniform grid at the half way point between successive breakpoints will bias the interval relative to the breakpoint. See below -- < <-- nearest neighbour data -< o oo <-- breakpoints 1 23 This is a true "nearest neighbour" division (allowing for my rather primitive ascii art :), but the difference in breakpoint spacing leads to the middle output value not being centred on breakpoint 2. If the user wants to adjust these biases, it would be better to use either the floor or ceiling forms to explicitly set the output value changes. This is therefore another argument in favour of the three options. Regards, Dan Newman
Re: Discrete interpolation attribute
this might be obvious, but since we are being thorough, need to define (for the "nearest" option) what happens when the data falls exactly midway between the breakpoints, as there are several possibilities. I believe the IEEE standard for rounding floating numbers is to round to even (lowest bit zero), but it's hard to see how we make that useful here. But it is best to avoid a scheme that introduces a bias, like "always round up in a tie". On the other hand it may not make a significant difference, but at least should be defined. Randy
Re: Discrete interpolation attribute
After discussing this around the office, we would be happy to see the three different attributes of nearest, floor and ceiling included. It would clear up any ambiguity and would be simple to implement support within the Janus software with no significant change in performance. Giovanni, you are right in saying that as long as it is clear what an attribute represents the data can be structured appropriately. With that though, minimising the amount of premanipulation of data is ideal. Once changes to the dtd are finalised, we will be updating Janus and re-releasing it. In the meantime, I am preparing a release package for the current dtd implementation that should be available by end next week. Geoff -Original Message- From: Giovanni A. Cignoni [mailto:[EMAIL PROTECTED] Sent: Tuesday, 3 July 2007 12:11 AM To: Bruce Jackson; Brian, Geoff Cc: simstds@larc.nasa.gov; Hill, Shane Subject: Re: [fs] Re: Discrete interpolation attribute > So now I'm considering the following, and would appreciate any > feedback from the community: > > Instead of a single 'discrete' attribute, provide for three: > > 'nearest' which performs as Geoff described above (not how my diagram > is drawn) 'floor' which holds the previous dependent value until the > next breakpoint is reached (which matches my diagram above) 'ceiling' > which rounds up the next dependent value upon passing each breakpoint. > > This seems to be the most flexible, but perhaps also the most complex > and may not be worth the effort. >From my perspective, first of all it's a matter of ambiguosness of the modelling language. The really important thing is to agree on the way the function is calculated. A minimal solution is to allow just one attribute. If the way the table is interpreted is clear for all users, the language is safe. For instance, if the modeller first intention was to use a "nearest", but the language features the "floor", then the modeller has to arrange a little the table. The important thing is the modeller being aware that the language offers "floor". A little more work, but not an actual obstacle. The proposed solution of three attributes is more friendly. It costs a bit more because a fully compliant implementation of the language must offers the three ways of discrete interpretation of the table, but actually it doesn't seems a big costs - quadraticSpline and cubicSpline options are more demanding for the implementor :) Ciao, Giovanni Cignoni. IMPORTANT: This email remains the property of the Australian Defence Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT 1914. If you have received this email in error, you are requested to contact the sender and delete the email.
Re: [fs] Re: Discrete interpolation attribute
So now I'm considering the following, and would appreciate any feedback from the community: Instead of a single 'discrete' attribute, provide for three: 'nearest' which performs as Geoff described above (not how my diagram is drawn) 'floor' which holds the previous dependent value until the next breakpoint is reached (which matches my diagram above) 'ceiling' which rounds up the next dependent value upon passing each breakpoint. This seems to be the most flexible, but perhaps also the most complex and may not be worth the effort. From my perspective, first of all it's a matter of ambiguosness of the modelling language. The really important thing is to agree on the way the function is calculated. A minimal solution is to allow just one attribute. If the way the table is interpreted is clear for all users, the language is safe. For instance, if the modeller first intention was to use a "nearest", but the language features the "floor", then the modeller has to arrange a little the table. The important thing is the modeller being aware that the language offers "floor". A little more work, but not an actual obstacle. The proposed solution of three attributes is more friendly. It costs a bit more because a fully compliant implementation of the language must offers the three ways of discrete interpretation of the table, but actually it doesn't seems a big costs - quadraticSpline and cubicSpline options are more demanding for the implementor :) Ciao, Giovanni Cignoni.
Re: Discrete interpolation attribute
On Jun 19, 2007, at 5:33 PM, Brian, Geoff wrote: From: Bruce Jackson [mailto:[EMAIL PROTECTED] Sent: Tuesday, 19 June 2007 3:40 AM To: Giovanni A. Cignoni Cc: simstds@larc.nasa.gov Subject: Re: Discrete interpolation attribute (was New version of DAVEfunc DTD 1.9b3) On Jun 18, 2007, at 1:28 PM, Giovanni A. Cignoni wrote: The main change to the DTD is addition of 'discrete' as a supported interpolation method for tabular data, yielding a stair- step response from such a function. This came from a suggestion from Geoff Brian of Australia's DSTO. "Discrete" interpolation is the method also known as "nearest neighbor"? Being x in [a, b], f(x) is f(a) or f(b) depending on x being nearer to a or b. Correct? Thanks in advance, ciao, Giovanni Cignoni. This is a good question, Giovanni. I see that we need to be much more rigorous in our definition of 'discrete' as it applies to these tables. I had assumed the interpretation would be as follows (I don't think this is 'nearest-neighbor')... In the case of a one-dimensional function, if the independentVarPts are defined as [a, b, c, d] and an arbitrary griddedTable points are defined as [8.5, 9.0, 9.5, 10.0] the function f(x) would be evaluated as shown below: f(x) ^ | 10.0 - |o || || 9.5 - | o--o | | | | 9.0 - | o--o | | | | 8.5 - | o--o | |--> X | | | | a b c d so the independent values state where the function changes value. Nearest-neighbor would put the transitions exactly between the independent break points. I'd appreciate any feedback on this topic, especially from Geoff Brian who is apparently making use of this 'extension' to DAVE-ML. -- Bruce I also agree that this is a good question. There are times when interrogating data where it is necessary to return either a truncated value, as you describe Bruce, or the nearest neighbour. Thus a single 'discrete' attribute may not have been the best suggestion. This is something we had overlooked in suggesting such an attribute. At the risk of fattening the dtd with unnecessary options, having attributes for both these cases would be useful. At present, the data tables I am currently using the 'discrete' attribute for are more akin to nearest-neighbour. Regards Geoff So now I'm considering the following, and would appreciate any feedback from the community: Instead of a single 'discrete' attribute, provide for three: 'nearest' which performs as Geoff described above (not how my diagram is drawn) 'floor' which holds the previous dependent value until the next breakpoint is reached (which matches my diagram above) 'ceiling' which rounds up the next dependent value upon passing each breakpoint. This seems to be the most flexible, but perhaps also the most complex and may not be worth the effort. What say the combined intelligence of artisans? -- Bruce
RE: Re: Discrete interpolation attribute (was New version of DAVEfunc DTD 1.9b3)
The discrete option, as we originally intended it, was a nearest grid-point
option. The relevant code snippet from the Janus interpreter:
* allow for order 0 linear interpolation, i.e. take nearest discrete
* value on the grid.
*/
if ( 0 == functionIndependentVariableOrder_[ index ][ i ] ) {
if ( 0.5 >= frac[ i ] ) {
frac[ i ] = 0.0;
} else {
frac[ i ] = 1.0;
}
}
We only use discrete where the inputs are meant to be one of the breakpoints.
The "nearest" interpretation ensures we get the right answer if there is some
issue with a data precision-related error (though we would normally use integer
inputs anyway). However, I can also see the utility of a "threshhold" approach
as you have described (eg, ISA properties as a function of pressure altitude).
You could always shift the breakpoints to achieve the same thing, or use
MathML. I agree it needs to be clearly documented.
Regards, Rob
Rob Curtin
Ball Solutions Group
on contract to Air Vehicles Division,
Defence Science & Technology Organisation
Tel: +61 (0)3 9626 7730
Fax: +61 (0)3 9626 7705
[EMAIL PROTECTED]
-Original Message-
From: Giovanni A. Cignoni [mailto:[EMAIL PROTECTED]
Sent: Tuesday, 19 June 2007 10:17 PM
To: simstds@larc.nasa.gov
Subject: Re: Re: Discrete interpolation attribute (was New version of DAVEfunc
DTD 1.9b3)
> so the independent values state where the function changes value.
>
> Nearest-neighbor would put the transitions exactly between the
> independent break points.
So at the end is just truncation vs approximation, but being in sync with names
is anyway important :)
Ciao,
Giovanni.
IMPORTANT: This email remains the property of the Australian Defence
Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT
1914. If you have received this email in error, you are requested to contact
the sender and delete the email.
RE: Discrete interpolation attribute (was New version of DAVEfunc DTD 1.9b3)
I also agree that this is a good question. There are times when interrogating data where it is necessary to return either a truncated value, as you describe Bruce, or the nearest neighbour. Thus a single 'discrete' attribute may not have been the best suggestion. This is something we had overlooked in suggesting such an attribute. At the risk of fattening the dtd with unnecessary options, having attributes for both these cases would be useful. At present, the data tables I am currently using the 'discrete' attribute for are more akin to nearest-neighbour. Regards Geoff -Original Message- From: Bruce Jackson [mailto:[EMAIL PROTECTED] Sent: Tuesday, 19 June 2007 3:40 AM To: Giovanni A. Cignoni Cc: simstds@larc.nasa.gov Subject: Re: Discrete interpolation attribute (was New version of DAVEfunc DTD 1.9b3) On Jun 18, 2007, at 1:28 PM, Giovanni A. Cignoni wrote: >> The main change to the DTD is addition of 'discrete' as a supported >> interpolation method for tabular data, yielding a stair- step >> response from such a function. This came from a suggestion from >> Geoff Brian of Australia's DSTO. > > "Discrete" interpolation is the method also known as "nearest > neighbor"? > Being x in [a, b], f(x) is f(a) or f(b) depending on x being nearer to > a or b. Correct? > > Thanks in advance, ciao, > Giovanni Cignoni. > This is a good question, Giovanni. I see that we need to be much more rigorous in our definition of 'discrete' as it applies to these tables. I had assumed the interpretation would be as follows (I don't think this is 'nearest-neighbor')... In the case of a one-dimensional function, if the independentVarPts are defined as [a, b, c, d] and an arbitrary griddedTable points are defined as [8.5, 9.0, 9.5, 10.0] the function f(x) would be evaluated as shown below: f(x) ^ | 10.0 - |o || || 9.5 - | o--o | | | | 9.0 - | o--o | | | | 8.5 - | o--o | |--> X | | | | a b c d so the independent values state where the function changes value. Nearest-neighbor would put the transitions exactly between the independent break points. I'd appreciate any feedback on this topic, especially from Geoff Brian who is apparently making use of this 'extension' to DAVE-ML. -- Bruce IMPORTANT: This email remains the property of the Australian Defence Organisation and is subject to the jurisdiction of section 70 of the CRIMES ACT 1914. If you have received this email in error, you are requested to contact the sender and delete the email.
Re: Re: Discrete interpolation attribute (was New version of DAVEfunc DTD 1.9b3)
so the independent values state where the function changes value. Nearest-neighbor would put the transitions exactly between the independent break points. So at the end is just truncation vs approximation, but being in sync with names is anyway important :) Ciao, Giovanni.
Re: Discrete interpolation attribute (was New version of DAVEfunc DTD 1.9b3)
On Jun 18, 2007, at 1:28 PM, Giovanni A. Cignoni wrote: The main change to the DTD is addition of 'discrete' as a supported interpolation method for tabular data, yielding a stair- step response from such a function. This came from a suggestion from Geoff Brian of Australia's DSTO. "Discrete" interpolation is the method also known as "nearest neighbor"? Being x in [a, b], f(x) is f(a) or f(b) depending on x being nearer to a or b. Correct? Thanks in advance, ciao, Giovanni Cignoni. This is a good question, Giovanni. I see that we need to be much more rigorous in our definition of 'discrete' as it applies to these tables. I had assumed the interpretation would be as follows (I don't think this is 'nearest-neighbor')... In the case of a one-dimensional function, if the independentVarPts are defined as [a, b, c, d] and an arbitrary griddedTable points are defined as [8.5, 9.0, 9.5, 10.0] the function f(x) would be evaluated as shown below: f(x) ^ | 10.0 - |o || || 9.5 - | o--o | | | | 9.0 - | o--o | | | | 8.5 - | o--o | |--> X | | | | a b c d so the independent values state where the function changes value. Nearest-neighbor would put the transitions exactly between the independent break points. I'd appreciate any feedback on this topic, especially from Geoff Brian who is apparently making use of this 'extension' to DAVE-ML. -- Bruce
