Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-12 Thread Gerald van den Boogaart 
Dear Tom Andrews,

I have to object to your last mail.

In the rare case, where we would like to predict the local value at some 
location on a second earth from observations of a first earth,  we should 
follow your suggestions. 

However, when we are in the situation of kriging, we typically do not assume 
that we change the local geology between measurement and prediction and thus 
the important difference is not the difference of the kriging predictor from 
the expectation or of the difference of expectation and realized value, but 
the difference of predicted and realized value. 

I added some  inline comments to the mail also.

Best regards,
Gerald v.d. Boogaart


Am Donnerstag, 12. Oktober 2006 11:51 schrieb tom andrews:
>   Dear List
>
>   Let us consider theoretical model of stationary random function
>   with some correlation function.
>   Model and correlation function are out of any doubts.
>   First generation of outcome values does not have to cover
>   the second and go on.
>   It means that "true" outcome value from first generation
>   at some coordinate does not have to cover the "true" outcome
>   value from second generation at the same coordinate and go on.
>
>   In practice we see and try to estimate only one generation.
>

So this is what kriging and geostatistics in general tries to do. If you would 
like to predict another generation you either need space-time kriging, where 
you instruct kriging to estimate the next generation. But spatial kriging is 
not applicable in this situation.  

>   In model point of view we should rather to express
>   kriging variance by
>   a) var( estimate(x)-Z'(x) ) where Z' can generate all outcome values
>   of random variable at some coordinate x
>   not by
>   b) var( estimate(x)-Z(x) ) where Z generates only one "true" outcome
> value at some coordinate x

You say "should" express kriging variance. However that would mean changing 
definitions. Call this object an Andrews variance and we can discusse the 
usefulness of Andrews variance. 


>   I see that 1.96 catches 95% of probability in case b but not in a
>   (except mean estimation in case a).

It catches 95% of the marginal probability and 95% of the "next generation". 
It catches more than 95% on the same generation. However for the next 
generation you can have by far shorter prediction intervals using the adequat 
method (e.g. estimation of mean and variance, rather than kriging) 

>
>   My thesis (right side of kriging variance does not lie):
>
>   a) In statistics we have the variance E{ [E{V}-V ]^2 }  where E{V} is
>   expected VALUE and V is random VARIABLE
>
>   b) Kriging variance in fact is E{ [S{V}-V]^2 } where S{V} is spread VALUE
>   and V is random VARIABLE

S(V) is the prediction of the random variable, not a spread value, but in it 
self random.


>   c) IF S{V}  =  E{V} then E{ [S{V}-V]^2 }  =  E{ [E{V}-V]^2 } = sigma^2
>   IF S{V} <> E{V} then E{ [S{V}-V]^2 } <> E{ [E{V}-V]^2 } = sigma^2

Typically the kriging variance is smaller than sigma^2. Which is a good 
feature not a bad one. 

>   d) We should forget about kriging variance in the case of interpolation

No. According to your argument, we must forget about kriging variances in case 
of predicting a second earth,


>   e) We should analyze our interpolation results out of area of interest
>  (to not meet known values with default zero value of kriging
> variance). If kriging variance is equal to sigma^2 (sigma^2  is a
> multiplier in the term of correlation function so we can only analyze
> variance ratio) then we know mean value of V.

This is a way to predict the mean of the field.
>  If kriging variance tends to zero it means that predicted value
>  "disappears" in the tail of distribution of random variable V.

Not really. 

>   All considerations follow ordinary kriging variance.
>
>   P.S.
>   Gaussian noise is unpredictable (spread value is unpredictable).
>   Has only mean and variance. Gaussian noise in linear drift also has
>   only mean and variance cause detrended gaussian noise in drift has only
>   constant mean and variance.

You forgot the word "white" here. Gaussian white noise is unpredictable. 

Best regards,
Gerald 
>
>   Best Regards
> tom
>
>
> -
> Talk is cheap. Use Yahoo! Messenger to make PC-to-Phone calls.  Great rates
> starting at 1¢/min.


+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and 
"unsubscribe ai-geostats" in the message body. DO NOT SEND 
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-12 Thread tom andrews 
  Dear List     Let us consider theoretical model of stationary random function   with some correlation function.  Model and correlation function are out of any doubts.  First generation of outcome values does not have to cover  the second and go on.  It means that "true" outcome value from first generation   at some coordinate does not have to cover the "true" outcome  value from second generation at the same coordinate and go on.     In practice we see and try to estimate only one generation.      In model point of view we should rather to express  kriging variance by  a) var( estimate(x)-Z'(x) ) where Z' can generate all outcome values   of random variable at some coordinate x   not by  b) var( estimate(x)-Z(x) ) where Z generates only one
 "true" outcome value  at some coordinate x     I see that 1.96 catches 95% of probability in case b but not in a   (except mean estimation in case a).     My thesis (right side of kriging variance does not lie):     a) In statistics we have the variance E{ [E{V}-V ]^2 }  where E{V} is       expected VALUE and V is random VARIABLE      b) Kriging variance in fact is E{ [S{V}-V]^2 } where S{V} is spread VALUE       and V is random VARIABLE     c) IF S{V}  =  E{V} then E{ [S{V}-V]^2 }  =  E{ [E{V}-V]^2 } = sigma^2       IF S{V} <> E{V} then E{ [S{V}-V]^2 } <> E{ [E{V}-V]^2 } = sigma^2      d) We should forget about kriging variance in the case of interpolation  
    e) We should analyze our interpolation results out of area of interest      (to not meet known values with default zero value of kriging variance).     If kriging variance is equal to sigma^2 (sigma^2  is a multiplier in the      term of correlation function so we can only analyze variance ratio)      then we know mean value of V.      If kriging variance tends to zero it means that predicted value     "disappears" in the tail of distribution of random variable V.      All considerations follow ordinary kriging variance.     P.S.  Gaussian noise is unpredictable (spread value is unpredictable).   Has only mean and variance. Gaussian noise in linear drift also has  only mean and variance cause detrended gaussian noise in drift
 has only   constant mean and variance.      Best Regards    tom 
		Talk is cheap. Use Yahoo! Messenger to make PC-to-Phone calls.  Great rates starting at 1¢/min.

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-11 Thread Gerald van den Boogaart 
Dear Tom,

I do not fully understand your question. However I will try to give an as good 
as possible answer.

Am Dienstag, 10. Oktober 2006 10:46 schrieb tom andrews:
>   Dear List
>
>   For me kriging variance may convey distribution of random variable
>   (the same for each random variable).

The variance of the field is the same for each Z(x), but the Kriging variance 

sigma^2_Krige (x) = var( estimate(x)-Z(x) )  

is different for each location x. 

>   Ok, I'm not going to argue what kriging variance really is.
>
>   I can imagine mean (central value) of gaussian distribution +/- sigma.

Ok we have two Gaussian distributions of Z(x) here: 
The marginal one with variance of the field 

and 
the conditional one (given the surrounding observations) 
This has mean estimate(x) and Variance Z(x)

>   I can imagine that sigma can be multiplied by quantile of standard
> gaussian distribution, I see in my mind how confidence intervals are
> growing up to e.g. 95% of total area under the curve.

Ok, thats the image for the marginal one.

>   Next I see "poor" spread value in the tail of gaussian distribution
>   with some error (e.g. error of estimation) equal to e.g. 0,1 sigma.
>   But I can not imagine how these error intervals somewhere in the tail of
>   gaussian distribution and equal to +/- 0,1 sigma grow up to 95% of total
>   area under the curve when multiplied by 1,96.

Whats a poor spread value here?  I don't understand that.

However probably you see a kriging estimate in the tail of the Gaussian 
distribution, with a small 1.96 sigma^2_Krige=0.196 sigma around it and 
wonder, why it should catch 95% of probability of the marginal distribution. 

It does not do this. It catches 95% of the conditional distribution. So if the 
Z(x) is very low estimate(x) will in most cases be very low either. And if 
Z(x) is very high estimate(x) will be very high either and therefore the 
short moving intervall will catch the true value in 95% of the cases.

This is because the kriging estimate is not fixed and uses informations from 
the neighbouring locations to follow the true value of x. 

Let us consider my two feet. It is very difficult to predict the location of 
my left foot. A 95% confidence intervall at least has to contain my office, 
my home and the street between. A 99,9% confidence intervall probabily has to 
contain whole Europe.  However consider you know the position of my right 
foot. A 95% confidence region for my left foot of can easily be limited to 
50cm around this position. And this is true  although I could clearly do a 
splits or haveing a cruel accident.  

A 95% confidence limit for the surface of the earth would range from 0 to 
maybe 1500m. However if i give you a measurement of altitude from a random 
point of the earth saying it is 60m and I ask you what would be a 95% 
condidence limit for the altitude of a location 100m east of it. Most 
probably you would give me something like 25m to 75m. And similar (probabily 
with a larger spread since you assume more spread in the montains (i.e. 
heteroskedastisity)) if the given altitude where 5657m. 

Back for a moment to the kriging variance: If I would ask the same question 
1km east instead of 100m of the observed point your intervall would probably 
have considerably more spread and likewise it would have a different kriging 
variance. 


Best regards,
Gerald 
>



>   Could somebody help me?
>
>   Best Regards
> tom
>
>
> -
> Do you Yahoo!?
>  Everyone is raving about the  all-new Yahoo! Mail.

-- 
-
Prof. Dr. K. Gerald v.d. Boogaart
Professor als Juniorprofessor fuer Statistik
http://www.math-inf.uni-greifswald.de/statistik/  

B�ro: Franz-Mehring-Str. 48, 1.Etage rechts
e-mail: [EMAIL PROTECTED]
phone:  00+49 (0)3834/86-4621
fax:00+49 (0)3834/86-4615   (Institut)

paper-mail:
Ernst-Moritz-Arndt-Universitaet Greifswald
Institut f�r Mathematik und Informatik
Jahnstr. 15a
17487 Greifswald
Germany
--

+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and 
"unsubscribe ai-geostats" in the message body. DO NOT SEND 
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-10 Thread Raimon Tolosana 
Oh, I'm not sure, but I have the feeling that, in this discussion, we 
are mixing the non-conditional distribution (a.k.a. the global 
distribution) with the conditional (local) one... and by the way, 
dismissing confidence intervals using simple kriging variance for 
gaussian distributions is equivalent to dismissing *prediction* 
intervals (NOT *estimation* intervals of the mean) for a standard 
regression Y=a+bX... the length of both do not depend on the "data 
value" (the X of a regression, the observed values in SK), but I hope 
that noone will doubt of the fitness of confidence intervals for a 
simple regression...


Regards
Raimon


En/na tom andrews ha escrit:

Dear List
 
For me kriging variance may convey distribution of random variable

(the same for each random variable).
Ok, I'm not going to argue what kriging variance really is.
 
I can imagine mean (central value) of gaussian distribution +/- sigma.
I can imagine that sigma can be multiplied by quantile of standard 
gaussian

distribution, I see in my mind how confidence intervals are growing up to
e.g. 95% of total area under the curve.
Next I see "poor" spread value in the tail of gaussian distribution
with some error (e.g. error of estimation) equal to e.g. 0,1 sigma.
But I can not imagine how these error intervals somewhere in the tail of
gaussian distribution and equal to +/- 0,1 sigma grow up to 95% of total
area under the curve when multiplied by 1,96.
 
Could somebody help me?
 
Best Regards

  tom


Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail. 
 



+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-10 Thread tom andrews 
  Dear List     For me kriging variance may convey distribution of random variable   (the same for each random variable).   Ok, I'm not going to argue what kriging variance really is.      I can imagine mean (central value) of gaussian distribution +/- sigma.  I can imagine that sigma can be multiplied by quantile of standard gaussian  distribution, I see in my mind how confidence intervals are growing up to  e.g. 95% of total area under the curve.  Next I see "poor" spread value in the tail of gaussian distribution  with some error (e.g. error of estimation) equal to e.g. 0,1 sigma.  But I can not imagine how these error intervals somewhere in the tail of   gaussian distribution and equal to +/- 0,1 sigma grow up to 95% of total   area under the curve when multiplied by 1,96. 
    Could somebody help me?      Best Regards    tom  
		Do you Yahoo!? Everyone is raving about the  all-new Yahoo! Mail.

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-05 Thread Gerald van den Boogaart 
Dear philosophers,

of the order of the dwarf.
>   Prediction intervals have no meaning.

>   Suppose that ordinary kriging predicts that I am a dwarf (or giant)
>   with the height H.
>   be small (it means that predicted value "comes from" the tail of
>   distribution not that estimation is "good").
...
>
Predition intervals are defined as random intervals (random because data 
dependent) containing the unkown but true value with a given high 
probability. 

So any prediction method can predict an intervall of small values altough the 
value is high. However it is quite improbabile, that the true value is not 
contained in the prediction intervall. 
And to me this is a nice meaning of a prediction interval. 

And at least for Gaussian fields we can prove: Conditioned on what ever height 
you have, the conditional probability that your height is in the predictive 
intervall is 95%. 

The true problem about prediction intervals with kriging is that they depend 
on the variogram (which is estimated and might be wrong) and do not model 
local heteroskedastisity of the process, if we forgot to model it (e.g. by 
failing to do lognormal approaches) and do not model deviations from 
normality, when we apply the most simple methods (e.g. when we forget to 
model that e.g. by histogram reproduction). 

So, my conclusion is that we should invest more in thinking on our own models 
and applications, rather than blaming the most simple model  (e.g. ordinary 
kriging) for beeing simplicistic.

Best regards,
Gerald v.d. Boogaart

Am Donnerstag, 5. Oktober 2006 15:40 schrieb tom andrews:
> Dear Mario
>
>   I agree with You.
>
>   Suppose that ordinary kriging predicts that I am a dwarf (or giant)
>   with the height H.
>   Since for e.g. gaussian distribution holds
>   P(H)=0
>   we have to introduce
>   P(H - s < h < H + s) = p
>   where s is a square root of kriging variance and p is an area under
>   the gaussian curve for the interval (H-s,H+s) in its tail.
>   Knowing a total area under the curve we can compute probability.
>   No matter, am I a dwarf or not, the probability of being a dwarf
>   (with some height tolerance) is not high so kriging variance will
>   be small (it means that predicted value "comes from" the tail of
>   distribution not that estimation is "good").
>   Prediction intervals have no meaning.
>
>
>   Suppose that ordinary kriging predicts that I have a mean height.
>   Now, kriging variance (minimized error variance) is the estimator of
>   variance of random variable (sigma^2) and reaches maximum.
>   Probability that I have a mean height +/- sigma is high and known
>   but it is only property of distribution of random variable.
>   Any prediction intervals have no meaning too.
>
>
>
>   Best Regards
>  tom
>
>
> -
> Want to be your own boss? Learn how on  Yahoo! Small Business.

-- 
-
Prof. Dr. K. Gerald v.d. Boogaart
Professor als Juniorprofessor fuer Statistik
http://www.math-inf.uni-greifswald.de/statistik/  

B�ro: Franz-Mehring-Str. 48, 1.Etage rechts
e-mail: [EMAIL PROTECTED]
phone:  00+49 (0)3834/86-4621
fax:00+49 (0)3834/86-4615   (Institut)

paper-mail:
Ernst-Moritz-Arndt-Universitaet Greifswald
Institut f�r Mathematik und Informatik
Jahnstr. 15a
17487 Greifswald
Germany
--

+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and 
"unsubscribe ai-geostats" in the message body. DO NOT SEND 
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-05 Thread Dan Cornford 

Tom, list,

  hmmm  I worry that you do not fully understand probability theory 
(not meaning to be rude ...) - or maybe I have not understood your post!


The situation that p(x) = 0  is true for all continuous random 
variables, but this is not a problem since we typically (for continuous 
random variables) consider, as you suggest, the probability of a small 
interval or range, which is just the integral of the probability 
distribution function over this range (= area under the curve). So far 
so good - this is basic probability theory.


Your dwarf analogy is very bad however!

Edzer said it correctly when he pointed out that kriging, but in general 
all statistics, is fundamentally based on some model, and assumptions. 
This is true, in my perception, of all rigorous approaches to problem 
solving in general (I am not meaning to start a debate here) - the key 
thing is that you state and test your assumptions, and clearly define 
your model. The debate is then, is your model appropriate and are your 
assumptions valid - in real applications this MUST always be a debate!


If your variable really follows a Gaussian distribution (or multivariate 
Gaussian in the standard geostatistical case) then this will be a good 
model and confidence intervals, expressed as +/- standard deviation are 
good (but note these are confidence intervals about the mean - your 
prediction!). If you want to look at the probability of another value 
(not the mean) then you have to do something else - this is not what 
kriging variance is! But it will give you the numbers to compute the 
probability of being in any range, or exceeding a value or whatever else 
(from the Gaussian distribution function) BUT ONLY IF your model is 
correct and your assumptions are also good.


Unless I am missing something very subtle in your arguments then there 
is no issue here!


By the way, thanks to all for a good discussion of the merit and 
demerits of simulation as opposed to prediction!


cheers

Dan

tom andrews wrote:

Dear Mario

I agree with You.

Suppose that ordinary kriging predicts that I am a dwarf (or giant)
with the height H.   
Since for e.g. gaussian distribution holds

P(H)=0
we have to introduce
P(H - s < h < H + s) = p
where s is a square root of kriging variance and p is an area under
the gaussian curve for the interval (H-s,H+s) in its tail.
Knowing a total area under the curve we can compute probability.
No matter, am I a dwarf or not, the probability of being a dwarf
(with some height tolerance) is not high so kriging variance will
be small (it means that predicted value "comes from" the tail of
distribution not that estimation is "good").
Prediction intervals have no meaning.


Suppose that ordinary kriging predicts that I have a mean height.
Now, kriging variance (minimized error variance) is the estimator of
variance of random variable (sigma^2) and reaches maximum.
Probability that I have a mean height +/- sigma is high and known
but it is only property of distribution of random variable.
Any prediction intervals have no meaning too.



Best Regards
   tom 



Want to be your own boss? Learn how on Yahoo! Small Business. 



--
Dr Dan Cornford  | tel: +44 (0)121 204 3451
NCRG and Computer Science| email: [EMAIL PROTECTED]
Aston University | web: www.ncrg.aston.ac.uk/~cornfosd
Birmingham B4 7ET, UK|

+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

RE: AI-GEOSTATS: kriging variance and accuracy

2006-10-05 Thread tom andrews 
Dear MarioI agree with You.Suppose that ordinary kriging predicts that I am a dwarf (or giant)  with the height H.     Since for e.g. gaussian distribution holds   P(H)=0  we have to introduce  P(H - s < h < H + s) = p  where s is a square root of kriging variance and p is an area under   the gaussian curve for the interval (H-s,H+s) in its tail.   Knowing a total area under the curve we can compute probability.   No matter, am I a dwarf or not, the probability of being a dwarf   (with some height tolerance) is not high so kriging variance will   be small (it means that predicted value "comes from" the tail of   distribution not that estimation is "good").  Prediction intervals have no meaning.  Suppose that ordinary kriging predicts that I have a mean height.  Now, kriging variance (minimized error variance) is the estimator of   variance of random variable
 (sigma^2) and reaches maximum.   Probability that I have a mean height +/- sigma is high and known   but it is only property of distribution of random variable.  Any prediction intervals have no meaning too. Best Regards     tom 
	
	
		Want to be your own boss? Learn how on  Yahoo! Small Business.

Re: AI-GEOSTATS: kriging variance and accuracy

2006-10-02 Thread Syed Shibli 
Conditional simulations have their place, whether Gaussian-based or indicator-based. If there is a transfer function, for instance a multiphase flow simulator acting on a distribution of transmissibilities mapped using conditional simulation, then a set of realizations provides a useful basis for some uncertainty analyses, e.g. getting a distribution of hydrocarbon recovery numbers.

For such transmissibilities, which is notoriously non-stationary (e.g. where fractured rock dominates a permeability field), and where a global variogram model may not indicate the connectivity of extreme values, then there is probably no better recourse than to model a set of indicator variograms provided there is sufficient data to obtain them. In this case even Gaussian-based simulations, which still invoke some sort of second order stationarity and a global variogram model, will be of limited application.

Every technique is a double edged sword, so I guess it is a question of "know what you're doing and when"!

Regards,

Syed Shibli

On 2 Oct 2006, at 23:42, Mario Rossi wrote:

Dear all,
The purpose of the paper (and also of this discussion) is to make people understand about a clear fact: the kriging variance does not depend on local data, only indirectly through a global variogram model. It's a factual statement, not an attack.
 
There may be cases when you don't want to have the uncertainty model depend locally on data, but I have yet to find those cases in my practice (long one at that).
 
My point is that many people don't realize that the KV is not data dependent (locally), and they should. I've certainly proven to myself many times over the extent to which the KV fails to provide a measurement of local accuracy.
Cheers,
 
 
Mario

"Heuvelink, Gerard" <[EMAIL PROTECTED]> wrote:
Dear all,

For those interested, I have asked Gregoire to put the paper that Edzer
refers to on the ai-geostats documents section.

It was written in a time that the kriging variance was heavily attacked.
We felt the need to react to the biased views expressed at the time.

One interesting outcome of the paper is that it is not necessarily a
good thing to let the measure for interpolation accuracy strongly depend
on the degree of local spatial variation. Apparently, one can be too
'data-charged'!

As usual, I agree with everything that Edzer writes.

Gerard

Gerard B.M. Heuvelink
Environmental Sciences Group
Wageningen University and Research Centre
P.O. Box 47
6700 AA Wageningen
The Netherlands

tel +31 317 474628 / 485208
email [EMAIL PROTECTED]
http://www.sil.wur.nl/UK/

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Edzer J. Pebesma
Sent: vrijdag 29 september 2006 17:13
To: Gustavo G. Pilger
Cc: Kerry Ritter; ai-geostats@jrc.it
Subject: Re: AI-GEOSTATS: kriging variance and accuracy

Dear Kerry and Gustavo,

The kriging variance is a perfect measure for estimation uncertainty as 
long as a second order stationary model is a good representation of the
data under study. Obviously, if variability and/or spatial correlation
varies over the field of interest and you have sufficient data to 
characterize this, or e.g. do a non-linear transform such as a 
log-transform to correct for a proportional effect, than you can and
will do better when taking this into account.

In my opinion papers such as those by Journel and Rossi have
over-shouted their point, and have ignored that for many cases a second
order stationary random field is a suitable model, if not the only
possible.

The argument that after rejecting the kriging variance, conditional
simulation is suddenly needed as the solution get some measure of 
uncertainty is invalid: if you create a large enough set of conditional
Gaussian simulations, their mean value equals the kriging mean and their

variance equals the kriging variance. Nothing is gained, only an 
expensive approximation of something rather cheap is obtained.

You will not find many papers that make this point, as the only point is

that someone else is wrong. Not many people like to write such stuff.
Below is a reference that may be hard to get (but you can google for the

first author). I for instance didn't enjoy writing this email.

Best regards,
--
Edzer

Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging
variance a proper measure of interpolation error? In: Proceedings of the

fifth International Symposium on Spatial Accuracy Assessment in Natural
Resources and Environmental Sciences (eds. G. Hunter and K. Lowell).
Melbourne: RMIT University, 179-186.

Gustavo G. Pilger wrote:
> Hi,
>
> Indeed the kriging variance is only semi-variogram and spatial data
> configuration dependent. The kriging variance is calculated taking
> into account only the geometry of the samples, i.e. their spatial
> arrangement and the semi-variogram. Basically, kriging variance do not

> take i

RE: AI-GEOSTATS: kriging variance and accuracy

2006-10-02 Thread Mario Rossi 
Dear all,  The purpose of the paper (and also of this discussion) is to make people understand about a clear fact: the kriging variance does not depend on local data, only indirectly through a global variogram model. It's a factual statement, not an attack.      There may be cases when you don't want to have the uncertainty model depend locally on data, but I have yet to find those cases in my practice (long one at that).     My point is that many people don't realize that the KV is not data dependent (locally), and they should. I've certainly proven to myself many times over the extent to which the KV fails to provide a measurement of local accuracy.  Cheers,        Mario  "Heuvelink, Gerard" <[EMAIL PROTECTED]> wrote:  Dear all,For those interested, I have asked Gregoire to put the paper that Edzerrefers to on the ai-geostats documents section.It was written in a time that the kriging variance was heavily attacked.We felt the need to react to the biased views expressed at the time.One interesting outcome of the paper is that it is not necessarily agood thing to let the measure for interpolation accuracy strongly dependon the degree of local spatial variation. Apparently, one can be too'data-charged'!As usual, I agree with everything that Edzer writes.GerardGerard B.M. HeuvelinkEnvironmental Sciences GroupWageningen University and Research CentreP.O. Box 476700 AA WageningenThe Netherlandstel +31 317 474628 / 485208email [EMAIL PROTECTED]http://www.sil.wur.nl/UK/-Original Message-From: [EMAIL PROTECTED]
 [mailto:[EMAIL PROTECTED] OnBehalf Of Edzer J. PebesmaSent: vrijdag 29 september 2006 17:13To: Gustavo G. PilgerCc: Kerry Ritter; ai-geostats@jrc.itSubject: Re: AI-GEOSTATS: kriging variance and accuracyDear Kerry and Gustavo,The kriging variance is a perfect measure for estimation uncertainty as long as a second order stationary model is a good representation of the data under study. Obviously, if variability and/or spatial correlation varies over the field of interest and you have sufficient data to characterize this, or e.g. do a non-linear transform such as a log-transform to correct for a proportional effect, than you can and will do better when taking this into account.In my opinion papers such as those by Journel and Rossi have over-shouted their point, and have ignored that for many cases a second order stationary random field is a suitable model, if not the
 onlypossible.The argument that after rejecting the kriging variance, conditional simulation is suddenly needed as the solution get some measure of uncertainty is invalid: if you create a large enough set of conditional Gaussian simulations, their mean value equals the kriging mean and theirvariance equals the kriging variance. Nothing is gained, only an expensive approximation of something rather cheap is obtained.You will not find many papers that make this point, as the only point isthat someone else is wrong. Not many people like to write such stuff. Below is a reference that may be hard to get (but you can google for thefirst author). I for instance didn't enjoy writing this email.Best regards,--EdzerHeuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging variance a proper measure of interpolation error? In: Proceedings of thefifth International Symposium on
 Spatial Accuracy Assessment in Natural Resources and Environmental Sciences (eds. G. Hunter and K. Lowell). Melbourne: RMIT University, 179-186.Gustavo G. Pilger wrote:> Hi,>> Indeed the kriging variance is only semi-variogram and spatial data > configuration dependent. The kriging variance is calculated taking > into account only the geometry of the samples, i.e. their spatial > arrangement and the semi-variogram. Basically, kriging variance do not> take into account the value of the samples, but only their location > (and the semi-variogram), consequently ignoring the local variability.> Therefore this parameter is not appropriate to measure uncertainty. > For this purpose you should consider the use of conditional simulation> methods.>> I wrote some papers about this subject some years ago. For exemple:>> PILGER, Gustavo G.; COSTA, Joao
 Felipe Coimbra Leite; KOPPE, Jair > Carlos, 2001. Additional Samples: Where they Should be Located?. > Natural Resources Research, New York, v. 10, n. 3, p. 197-207.>> I can send you a copy if you wish.>> I hope this helps you.>> Cheers.> <><><><><><><><><><><><><><><><><><><><><><><>> Gustavo G. Pilger, Mining Engineer, MSc, PhD> Senior Geostatistician> MBR - Brazil> <><><><><><><><><><><><><><><><><><><><><><><>> Hi. I just read through Journel and Rossi's 1999 paper, "When do we>> need a trend model in Kriging". In the appendix they say "A kriging>>
 variance is but a variogram-model dependent ranking of data>> configurations; being data-value independent, it is generally not a>> measure of local accuracy...This fact is unfortunately not yet fully>> appreciated by some practitioners". Can someone explain the>> implications

RE: AI-GEOSTATS: kriging variance and accuracy

2006-10-02 Thread Heuvelink, Gerard 
Dear all,

For those interested, I have asked Gregoire to put the paper that Edzer
refers to on the ai-geostats documents section.

It was written in a time that the kriging variance was heavily attacked.
We felt the need to react to the biased views expressed at the time.

One interesting outcome of the paper is that it is not necessarily a
good thing to let the measure for interpolation accuracy strongly depend
on the degree of local spatial variation. Apparently, one can be too
'data-charged'!

As usual, I agree with everything that Edzer writes.

Gerard

Gerard B.M. Heuvelink
Environmental Sciences Group
Wageningen University and Research Centre
P.O. Box 47
6700 AA Wageningen
The Netherlands

tel +31 317 474628 / 485208
email [EMAIL PROTECTED]
http://www.sil.wur.nl/UK/

-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
Behalf Of Edzer J. Pebesma
Sent: vrijdag 29 september 2006 17:13
To: Gustavo G. Pilger
Cc: Kerry Ritter; ai-geostats@jrc.it
Subject: Re: AI-GEOSTATS: kriging variance and accuracy

Dear Kerry and Gustavo,

The kriging variance is a perfect measure for estimation uncertainty as 
long as a second order stationary model is a good representation of the 
data under study. Obviously, if variability and/or spatial correlation 
varies over the field of interest and you have sufficient data to 
characterize this, or e.g. do a non-linear transform such as a 
log-transform to correct for a proportional effect, than you can and 
will do better when taking this into account.

In my opinion papers such as those by Journel and Rossi have 
over-shouted their point, and have ignored that for many cases a second 
order stationary random field is a suitable model, if not the only
possible.

The argument that after rejecting the kriging variance, conditional 
simulation is suddenly needed as the solution get some measure of 
uncertainty is invalid: if you create a large enough set of conditional 
Gaussian simulations, their mean value equals the kriging mean and their

variance equals the kriging variance. Nothing is gained, only an 
expensive approximation of something rather cheap is obtained.

You will not find many papers that make this point, as the only point is

that someone else is wrong. Not many people like to write such stuff. 
Below is a reference that may be hard to get (but you can google for the

first author). I for instance didn't enjoy writing this email.

Best regards,
--
Edzer

Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging 
variance a proper measure of interpolation error? In: Proceedings of the

fifth International Symposium on Spatial Accuracy Assessment in Natural 
Resources and Environmental Sciences (eds. G. Hunter and K. Lowell). 
Melbourne: RMIT University, 179-186.

Gustavo G. Pilger wrote:
> Hi,
>
> Indeed the kriging variance is only semi-variogram and spatial data 
> configuration dependent. The kriging variance is calculated taking 
> into account only the geometry of the samples, i.e. their spatial 
> arrangement and the semi-variogram. Basically, kriging variance do not

> take into account the value of the samples, but only their location 
> (and the semi-variogram), consequently ignoring the local variability.

> Therefore this parameter is not appropriate to measure uncertainty. 
> For this purpose you should consider the use of conditional simulation

> methods.
>
> I wrote some papers about this subject some years ago. For exemple:
>
> PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair 
> Carlos, 2001.  Additional Samples: Where they Should be Located?. 
> Natural Resources Research, New York, v. 10, n. 3, p. 197-207.
>
> I can send you a copy if you wish.
>
> I hope this helps you.
>
> Cheers.
> <><><><><><><><><><><><><><><><><><><><><><><>
> Gustavo G. Pilger, Mining Engineer, MSc, PhD
> Senior Geostatistician
> MBR - Brazil
> <><><><><><><><><><><><><><><><><><><><><><><>
>
>
>
>> Hi.  I just read through Journel and Rossi's 1999 paper, "When do we
>> need a trend model in Kriging".  In the appendix they say "A kriging
>> variance is but a variogram-model dependent ranking of data
>> configurations; being data-value  independent, it is generally not a
>> measure of local accuracy...This fact is unfortunately not yet fully
>> appreciated by some practitioners".  Can someone explain the
>> implications of this in terms of determining cost-efficiency analysis
>> for sample designs?  Specifically, can we use kriging variance
>> estimates across poten

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-30 Thread Edzer J. Pebesma 

Mario, you use some tricky reasoning here:

Mario Rossi wrote:


Edzer,
the kriging variance is not data-dependent, which means that, no 
matter how perfectly the second order assumption applies to your 
domain, on a point by point or block by block basis, the kriging 
variance cannot be used to give you confidence intervals on individual 
estimated points.


Here you make a statement which is not true. (First, the primary 
question was whether the kriging variance is possibly a valid measures 
of uncertainty, which is true; second you introduce CI, which indeed can 
be obtained from kriging prediction and variance if the the multivariate 
normal model assumption holds, as you acknowledge below. Kriging 
variances and kriging prediction intervals are two different things)


The reason is that, for most variables, the estimation error is 
dependent on sample values, a property called heteroscedasticity; the 
only exception is if the errors are Gaussian.


Here you told us why your first statement is false.

 
Using the KV to provide local uncertainties (point-by-point) can only 
be done if the error distribution is homoscedastic,a property that 
only the Gaussian dist. has; in practice, error distributions tend to 
be far from Gaussian, even if the original variable

is quasi-Gaussian at the univariate (histogram) level.


I strongly doubt whether no non-Gaussian second order stationary 
functions exist.


 
In cases where you averaging up your points to large (very large) 
blocks, then the use of the KV becomes more acceptable (realistic) 
because the Central Limit Theorem will help in making those errors 
"more" Gaussian-like.


I agree with this, for the case of non-Gaussian variables.


Hope this helps,


Yes and no, if I'm concerned. It feels like you want to forbid any 
multi-Gaussian assumption; I believe that people should realise what 
they assume, but that they should decide for themselves.


Best regards,
--
Edzer

 
Regards,
 
 
Mario
 


*/"Edzer J. Pebesma" <[EMAIL PROTECTED]>/* wrote:

Dear Kerry and Gustavo,

The kriging variance is a perfect measure for estimation
uncertainty as
long as a second order stationary model is a good representation
of the
data under study. Obviously, if variability and/or spatial
correlation
varies over the field of interest and you have sufficient data to
characterize this, or e.g. do a non-linear transform such as a
log-transform to correct for a proportional effect, than you can and
will do better when taking this into account.

In my opinion papers such as those by Journel and Rossi have
over-shouted their point, and have ignored that for many cases a
second
order stationary random field is a suitable model, if not the only
possible.

The argument that after rejecting the kriging variance, conditional
simulation is suddenly needed as the solution get some measure of
uncertainty is invalid: if you create a large enough set of
conditional
Gaussian simulations, their mean value equals the kriging mean and
their
variance equals the kriging variance. Nothing is gained, only an
expensive approximation of something rather cheap is obtained.

You will not find many papers that make this point, as the only
point is
that someone else is wrong. Not many people like to write such stuff.
Below is a reference that may be hard to get (but you can google
for the
first author). I for instance didn't enjoy writing this email.

Best regards,
--
Edzer

Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging
variance a proper measure of interpolation error? In: Proceedings
of the
fifth International Symposium on Spatial Accuracy Assessment in
Natural
Resources and Environmental Sciences (eds. G. Hunter and K. Lowell).
Melbourne: RMIT University, 179-186.

Gustavo G. Pilger wrote:
> Hi,
>
> Indeed the kriging variance is only semi-variogram and spatial data
> configuration dependent. The kriging variance is calculated taking
> into account only the geometry of the samples, i.e. their spatial
> arrangement and the semi-variogram. Basically, kriging variance
do not
> take into account the value of the samples, but only their location
> (and the semi-variogram), consequently ignoring the local
variability.
> Therefore this parameter is not appropriate to measure uncertainty.
> For this purpose you should consider the use of conditional
simulation
> methods.
>
> I wrote some papers about this subject some years ago. For exemple:
>
> PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair
> Carlos, 2001. Additional Samples: Where they Should be Located?.
> Natural Resources Research, New York, v. 10, n. 3, p. 197-207.
>
> I can send you a copy if you wish.
>
> I hope this helps you.
>
> Cheers.
> <

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Mario Rossi 
Edzer,  the kriging variance is not data-dependent, which means that, no matter how perfectly the second order assumption applies to your domain, on a point by point or block by block basis, the kriging variance cannot be used to give you confidence intervals on individual estimated points.  The reason is that, for most variables, the estimation error is dependent on sample values, a property called heteroscedasticity; the only exception is if the errors are Gaussian.   Using the KV to provide local uncertainties (point-by-point) can only be done if the error distribution is homoscedastic,a property that only the Gaussian dist. has; in practice, error distributions tend to be far from Gaussian, even if the original variable   is quasi-Gaussian at the univariate (histogram) level.      In cases where you averaging up your points to large (very large) blocks, then the use of the KV becomes
 more acceptable (realistic) because the Central Limit Theorem will help in making those errors "more" Gaussian-like.  Hope this helps,     Regards,        Mario     "Edzer J. Pebesma" <[EMAIL PROTECTED]> wrote:  Dear Kerry and Gustavo,The kriging variance is a perfect measure for estimation uncertainty as long as a second order stationary model is a good representation of the data under study. Obviously, if variability and/or spatial correlation varies over the field of interest and you have sufficient data to characterize this, or e.g. do a non-linear transform such as a log-transform to correct for a proportional effect, than you can and will do better when taking this into account.In my opinion
 papers such as those by Journel and Rossi have over-shouted their point, and have ignored that for many cases a second order stationary random field is a suitable model, if not the only possible.The argument that after rejecting the kriging variance, conditional simulation is suddenly needed as the solution get some measure of uncertainty is invalid: if you create a large enough set of conditional Gaussian simulations, their mean value equals the kriging mean and their variance equals the kriging variance. Nothing is gained, only an expensive approximation of something rather cheap is obtained.You will not find many papers that make this point, as the only point is that someone else is wrong. Not many people like to write such stuff. Below is a reference that may be hard to get (but you can google for the first author). I for instance didn't enjoy writing this email.Best
 regards,--EdzerHeuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging variance a proper measure of interpolation error? In: Proceedings of the fifth International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences (eds. G. Hunter and K. Lowell). Melbourne: RMIT University, 179-186.Gustavo G. Pilger wrote:> Hi,>> Indeed the kriging variance is only semi-variogram and spatial data > configuration dependent. The kriging variance is calculated taking > into account only the geometry of the samples, i.e. their spatial > arrangement and the semi-variogram. Basically, kriging variance do not > take into account the value of the samples, but only their location > (and the semi-variogram), consequently ignoring the local variability. > Therefore this parameter is not appropriate to measure uncertainty. > For this purpose you
 should consider the use of conditional simulation > methods.>> I wrote some papers about this subject some years ago. For exemple:>> PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair > Carlos, 2001. Additional Samples: Where they Should be Located?. > Natural Resources Research, New York, v. 10, n. 3, p. 197-207.>> I can send you a copy if you wish.>> I hope this helps you.>> Cheers.> <><><><><><><><><><><><><><><><><><><><><><><>> Gustavo G. Pilger, Mining Engineer, MSc, PhD> Senior Geostatistician> MBR - Brazil>
 <><><><><><><><><><><><><><><><><><><><><><><>> Hi. I just read through Journel and Rossi's 1999 paper, "When do we>> need a trend model in Kriging". In the appendix they say "A kriging>> variance is but a variogram-model dependent ranking of data>> configurations; being data-value independent, it is generally not a>> measure of local accuracy...This fact is unfortunately not yet fully>> appreciated by some practitioners". Can someone explain the>> implications of this in terms of determining cost-efficiency analysis>> for sample designs? Specifically, can we use kriging variance>> estimates across potential sampling grids, (from modeled variograms>> estimated from say a pilot study) to estimate the
 variability>> associated with different sampling densities/configurations. In>> addition, can someone provide some references that address this topic. Thanks,>> Kerry>> +>> + To post a message to the list, send it to ai-geostats@jrc.it>> + To unsubscribe, send email to majordomo@ jrc.it with no subject and>> "unsubscribe ai-geostats" in the message body. DO NOT SEND>>

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Yetta Jager 
One way to assess the validity of kriging variance estimates (and 
semivariogram model)
is by cross-validation (removing points, estimating values, and 
calculating the resulting

deviation from the known value).

+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Edzer J Pebesma 



Ashton Shortridge wrote:

My own take differs a bit from Edzer's. 


...

However, if one's application is concerned with joint uncertainties, for 
example, the joint probability that an entire region is above a certain 
height, than simulation appears to be necessary.


Yours,

Ashton Shortridge
 

Ashton, I agree completely. I didn't mean to say that conditional 
simulation is unnecessary in general, your example is a strong case.

--
Edzer

+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

RE: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Pierre Goovaerts 
Hello,
 
I have discussed the use of stochastic simulation and kriging to model
local versus spatial uncertainty in the following paper:
Goovaerts, P. 2001. Geostatistical modelling of uncertainty in soil science. 
Geoderma, 103: 3-26. <http://www.terraseer.com/training/geostats/geoder01.pdf>  
 
As Ashton pointed out, stochastic simulation is now trendy and several times
I came across papers where the authors spent a great deal of time generating
a bunch of realizations before simply taking the average of all the maps..
which is clearly a waste of CPU time and memory... As Edzer mentioned,
the mean and variance of Gaussian simulated values will give back the kriging
estimate and variance. This is, however, not the case if the sample histogram
is asymmetric and data need to be normal score transformed... You can still use
multiGaussian kriging but the derivation of the kriging estimate and variance 
in the original
space needs some work (see Saito, H. and P. Goovaerts. 2000. Geostatistical 
<http://home.att.ne.jp/grape/geostat/PAPER/EST01.pdf> 
interpolation of positively skewed and censored data in a dioxin contaminated 
site. 
Environmental Science & Technology, vol.34, No.19: 4228-4235. ), 
In this case, it is usually more straightforward for users with no coding skill 
to generate 
a bunch of realizations and compute statistics from the set of back-transformed 
simulated values.
 
In general, stochastic simulation is useful to model uncertainty over spatial 
supports that
are larger than the measurement support (i.e. upscaling), as well as to 
characterize
the uncertainty prevailing at several locations simultaneously (e.g. 
uncertainty about the
occurrence of a string of low or high values).
 
Edzer made a good point that the kriging variance is derived from a model, and 
we
shouldn't blame the variance as an inadequate measure of uncertainty whenever we
use a model that is inappropriate for our data... On the other hand, the 
kriging variance
has too often been portrayed as the magic recipe that will reveal where to 
collect additional
samples.. while it simply tells you to sample in sparsely sampled areas...
 
Pierre
 
Pierre Goovaerts
Chief Scientist at BioMedware Inc.
Courtesy Associate Professor, University of Florida
President of PGeostat LLC
 
Office address: 
516 North State Street
Ann Arbor, MI 48104
Voice: (734) 913-1098 (ext. 8)
Fax: (734) 913-2201 
http://home.comcast.net/~goovaerts/ 



From: [EMAIL PROTECTED] on behalf of Ashton Shortridge
Sent: Fri 9/29/2006 12:47 PM
To: Edzer J. Pebesma
Cc: Gustavo G. Pilger; Kerry Ritter; ai-geostats@jrc.it
Subject: Re: AI-GEOSTATS: kriging variance and accuracy



My own take differs a bit from Edzer's. He is of course correct that the use
of conditional simulation to identify uncertainty at a location is at best an
inefficient way to accomplish this. I've reviewed articles that did this. I
don't know if it's ignorance or just the trendy nature of simulation.

However, if one's application is concerned with joint uncertainties, for
example, the joint probability that an entire region is above a certain
height, than simulation appears to be necessary.

Yours,

Ashton Shortridge

On Friday 29 September 2006 11:12 am, Edzer J. Pebesma wrote:
> Dear Kerry and Gustavo,
>
> The kriging variance is a perfect measure for estimation uncertainty as
> long as a second order stationary model is a good representation of the
> data under study. Obviously, if variability and/or spatial correlation
> varies over the field of interest and you have sufficient data to
> characterize this, or e.g. do a non-linear transform such as a
> log-transform to correct for a proportional effect, than you can and
> will do better when taking this into account.
>
> In my opinion papers such as those by Journel and Rossi have
> over-shouted their point, and have ignored that for many cases a second
> order stationary random field is a suitable model, if not the only
> possible.
>
> The argument that after rejecting the kriging variance, conditional
> simulation is suddenly needed as the solution get some measure of
> uncertainty is invalid: if you create a large enough set of conditional
> Gaussian simulations, their mean value equals the kriging mean and their
> variance equals the kriging variance. Nothing is gained, only an
> expensive approximation of something rather cheap is obtained.
>
> You will not find many papers that make this point, as the only point is
> that someone else is wrong. Not many people like to write such stuff.
> Below is a reference that may be hard to get (but you can google for the
> first author). I for instance didn't enjoy writing this email.
>
> Best regards,
> --
> Edzer
>
> Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging
> variance a proper measure

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Ashton Shortridge 
My own take differs a bit from Edzer's. He is of course correct that the use 
of conditional simulation to identify uncertainty at a location is at best an 
inefficient way to accomplish this. I've reviewed articles that did this. I 
don't know if it's ignorance or just the trendy nature of simulation.

However, if one's application is concerned with joint uncertainties, for 
example, the joint probability that an entire region is above a certain 
height, than simulation appears to be necessary.

Yours,

Ashton Shortridge

On Friday 29 September 2006 11:12 am, Edzer J. Pebesma wrote:
> Dear Kerry and Gustavo,
>
> The kriging variance is a perfect measure for estimation uncertainty as
> long as a second order stationary model is a good representation of the
> data under study. Obviously, if variability and/or spatial correlation
> varies over the field of interest and you have sufficient data to
> characterize this, or e.g. do a non-linear transform such as a
> log-transform to correct for a proportional effect, than you can and
> will do better when taking this into account.
>
> In my opinion papers such as those by Journel and Rossi have
> over-shouted their point, and have ignored that for many cases a second
> order stationary random field is a suitable model, if not the only
> possible.
>
> The argument that after rejecting the kriging variance, conditional
> simulation is suddenly needed as the solution get some measure of
> uncertainty is invalid: if you create a large enough set of conditional
> Gaussian simulations, their mean value equals the kriging mean and their
> variance equals the kriging variance. Nothing is gained, only an
> expensive approximation of something rather cheap is obtained.
>
> You will not find many papers that make this point, as the only point is
> that someone else is wrong. Not many people like to write such stuff.
> Below is a reference that may be hard to get (but you can google for the
> first author). I for instance didn't enjoy writing this email.
>
> Best regards,
> --
> Edzer
>
> Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging
> variance a proper measure of interpolation error? In: Proceedings of the
> fifth International Symposium on Spatial Accuracy Assessment in Natural
> Resources and Environmental Sciences (eds. G. Hunter and K. Lowell).
> Melbourne: RMIT University, 179-186.
>
> Gustavo G. Pilger wrote:
> > Hi,
> >
> > Indeed the kriging variance is only semi-variogram and spatial data
> > configuration dependent. The kriging variance is calculated taking
> > into account only the geometry of the samples, i.e. their spatial
> > arrangement and the semi-variogram. Basically, kriging variance do not
> > take into account the value of the samples, but only their location
> > (and the semi-variogram), consequently ignoring the local variability.
> > Therefore this parameter is not appropriate to measure uncertainty.
> > For this purpose you should consider the use of conditional simulation
> > methods.
> >
> > I wrote some papers about this subject some years ago. For exemple:
> >
> > PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair
> > Carlos, 2001.  Additional Samples: Where they Should be Located?.
> > Natural Resources Research, New York, v. 10, n. 3, p. 197-207.
> >
> > I can send you a copy if you wish.
> >
> > I hope this helps you.
> >
> > Cheers.
> > <><><><><><><><><><><><><><><><><><><><><><><>
> > Gustavo G. Pilger, Mining Engineer, MSc, PhD
> > Senior Geostatistician
> > MBR - Brazil
> > <><><><><><><><><><><><><><><><><><><><><><><>
> >
> >> Hi.  I just read through Journel and Rossi's 1999 paper, "When do we
> >> need a trend model in Kriging".  In the appendix they say "A kriging
> >> variance is but a variogram-model dependent ranking of data
> >> configurations; being data-value  independent, it is generally not a
> >> measure of local accuracy...This fact is unfortunately not yet fully
> >> appreciated by some practitioners".  Can someone explain the
> >> implications of this in terms of determining cost-efficiency analysis
> >> for sample designs?  Specifically, can we use kriging variance
> >> estimates across potential sampling grids, (from modeled variograms
> >> estimated from say a pilot study) to estimate the variability
> >> associated with different sampling densities/configurations.  In
> >> addition, can someone provide some references that address this topic.
> >>
> >> Thanks,
> >> Kerry
> >> +
> >> + To post a message to the list, send it to ai-geostats@jrc.it
> >> + To unsubscribe, send email to majordomo@ jrc.it with no subject and
> >> "unsubscribe ai-geostats" in the message body. DO NOT SEND
> >> Subscribe/Unsubscribe requests to the list
> >> + As a general service to list users, please remember to post a summary
> >> of any useful responses to your questions.
> >> + Support to the forum can be found at http://www.ai-geostats.org/
> >
> > +
> > + To post a message to the list, send it to ai-

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Kerry Ritter 
Thank you all for your replies.  Based on Edzer's comments, it seems 
that if I have adequately taken care to insure second order stationarity 
then I can use the kriging variance.  If I choose ordinary kriging I 
model residuals after detrending/transforming the data. It seems that if 
I do this incorrectly then my kriging errors will be off.   If I use 
universal kriging will I fair any better? 
-Kerry

Edzer J. Pebesma wrote:


Dear Kerry and Gustavo,

The kriging variance is a perfect measure for estimation uncertainty 
as long as a second order stationary model is a good representation of 
the data under study. Obviously, if variability and/or spatial 
correlation varies over the field of interest and you have sufficient 
data to characterize this, or e.g. do a non-linear transform such as a 
log-transform to correct for a proportional effect, than you can and 
will do better when taking this into account.


In my opinion papers such as those by Journel and Rossi have 
over-shouted their point, and have ignored that for many cases a 
second order stationary random field is a suitable model, if not the 
only possible.


The argument that after rejecting the kriging variance, conditional 
simulation is suddenly needed as the solution get some measure of 
uncertainty is invalid: if you create a large enough set of 
conditional Gaussian simulations, their mean value equals the kriging 
mean and their variance equals the kriging variance. Nothing is 
gained, only an expensive approximation of something rather cheap is 
obtained.


You will not find many papers that make this point, as the only point 
is that someone else is wrong. Not many people like to write such 
stuff. Below is a reference that may be hard to get (but you can 
google for the first author). I for instance didn't enjoy writing this 
email.


Best regards,
--
Edzer

Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging 
variance a proper measure of interpolation error? In: Proceedings of 
the fifth International Symposium on Spatial Accuracy Assessment in 
Natural Resources and Environmental Sciences (eds. G. Hunter and K. 
Lowell). Melbourne: RMIT University, 179-186.


Gustavo G. Pilger wrote:


Hi,

Indeed the kriging variance is only semi-variogram and spatial data 
configuration dependent. The kriging variance is calculated taking 
into account only the geometry of the samples, i.e. their spatial 
arrangement and the semi-variogram. Basically, kriging variance do 
not take into account the value of the samples, but only their 
location (and the semi-variogram), consequently ignoring the local 
variability. Therefore this parameter is not appropriate to measure 
uncertainty. For this purpose you should consider the use of 
conditional simulation methods.


I wrote some papers about this subject some years ago. For exemple:

PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair 
Carlos, 2001.  Additional Samples: Where they Should be Located?. 
Natural Resources Research, New York, v. 10, n. 3, p. 197-207.


I can send you a copy if you wish.

I hope this helps you.

Cheers.
<><><><><><><><><><><><><><><><><><><><><><><>
Gustavo G. Pilger, Mining Engineer, MSc, PhD
Senior Geostatistician
MBR - Brazil
<><><><><><><><><><><><><><><><><><><><><><><>




Hi.  I just read through Journel and Rossi's 1999 paper, "When do we
need a trend model in Kriging".  In the appendix they say "A kriging
variance is but a variogram-model dependent ranking of data
configurations; being data-value  independent, it is generally not a
measure of local accuracy...This fact is unfortunately not yet fully
appreciated by some practitioners".  Can someone explain the
implications of this in terms of determining cost-efficiency analysis
for sample designs?  Specifically, can we use kriging variance
estimates across potential sampling grids, (from modeled variograms
estimated from say a pilot study) to estimate the variability
associated with different sampling densities/configurations.  In
addition, can someone provide some references that address this topic.

Thanks,
Kerry
+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and
"unsubscribe ai-geostats" in the message body. DO NOT SEND
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary
of any useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/




+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and 
"unsubscribe ai-geostats" in the message body. DO NOT SEND 
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a 
summary of any useful responses to your questions.

+ Support to the forum can be found at http://www.ai-geostats.org/



+
+ To post a message to the list,

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Edzer J. Pebesma 

Dear Kerry and Gustavo,

The kriging variance is a perfect measure for estimation uncertainty as 
long as a second order stationary model is a good representation of the 
data under study. Obviously, if variability and/or spatial correlation 
varies over the field of interest and you have sufficient data to 
characterize this, or e.g. do a non-linear transform such as a 
log-transform to correct for a proportional effect, than you can and 
will do better when taking this into account.


In my opinion papers such as those by Journel and Rossi have 
over-shouted their point, and have ignored that for many cases a second 
order stationary random field is a suitable model, if not the only possible.


The argument that after rejecting the kriging variance, conditional 
simulation is suddenly needed as the solution get some measure of 
uncertainty is invalid: if you create a large enough set of conditional 
Gaussian simulations, their mean value equals the kriging mean and their 
variance equals the kriging variance. Nothing is gained, only an 
expensive approximation of something rather cheap is obtained.


You will not find many papers that make this point, as the only point is 
that someone else is wrong. Not many people like to write such stuff. 
Below is a reference that may be hard to get (but you can google for the 
first author). I for instance didn't enjoy writing this email.


Best regards,
--
Edzer

Heuvelink, G.B.M. and E.J. Pebesma, 2002, Is the ordinary kriging 
variance a proper measure of interpolation error? In: Proceedings of the 
fifth International Symposium on Spatial Accuracy Assessment in Natural 
Resources and Environmental Sciences (eds. G. Hunter and K. Lowell). 
Melbourne: RMIT University, 179-186.


Gustavo G. Pilger wrote:

Hi,

Indeed the kriging variance is only semi-variogram and spatial data 
configuration dependent. The kriging variance is calculated taking 
into account only the geometry of the samples, i.e. their spatial 
arrangement and the semi-variogram. Basically, kriging variance do not 
take into account the value of the samples, but only their location 
(and the semi-variogram), consequently ignoring the local variability. 
Therefore this parameter is not appropriate to measure uncertainty. 
For this purpose you should consider the use of conditional simulation 
methods.


I wrote some papers about this subject some years ago. For exemple:

PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair 
Carlos, 2001.  Additional Samples: Where they Should be Located?. 
Natural Resources Research, New York, v. 10, n. 3, p. 197-207.


I can send you a copy if you wish.

I hope this helps you.

Cheers.
<><><><><><><><><><><><><><><><><><><><><><><>
Gustavo G. Pilger, Mining Engineer, MSc, PhD
Senior Geostatistician
MBR - Brazil
<><><><><><><><><><><><><><><><><><><><><><><>




Hi.  I just read through Journel and Rossi's 1999 paper, "When do we
need a trend model in Kriging".  In the appendix they say "A kriging
variance is but a variogram-model dependent ranking of data
configurations; being data-value  independent, it is generally not a
measure of local accuracy...This fact is unfortunately not yet fully
appreciated by some practitioners".  Can someone explain the
implications of this in terms of determining cost-efficiency analysis
for sample designs?  Specifically, can we use kriging variance
estimates across potential sampling grids, (from modeled variograms
estimated from say a pilot study) to estimate the variability
associated with different sampling densities/configurations.  In
addition, can someone provide some references that address this topic.

Thanks,
Kerry
+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and
"unsubscribe ai-geostats" in the message body. DO NOT SEND
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary
of any useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/



+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and 
"unsubscribe ai-geostats" in the message body. DO NOT SEND 
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a 
summary of any useful responses to your questions.

+ Support to the forum can be found at http://www.ai-geostats.org/


+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

Re: AI-GEOSTATS: kriging variance and accuracy

2006-09-29 Thread Gustavo G. Pilger 

Hi,

Indeed the kriging variance is only semi-variogram and spatial data  
configuration dependent. The kriging variance is calculated taking  
into account only the geometry of the samples, i.e. their spatial  
arrangement and the semi-variogram. Basically, kriging variance do not  
take into account the value of the samples, but only their location  
(and the semi-variogram), consequently ignoring the local variability.  
Therefore this parameter is not appropriate to measure uncertainty.  
For this purpose you should consider the use of conditional simulation  
methods.


I wrote some papers about this subject some years ago. For exemple:

PILGER, Gustavo G.; COSTA, Joao Felipe Coimbra Leite; KOPPE, Jair  
Carlos, 2001.  Additional Samples: Where they Should be Located?.  
Natural Resources Research, New York, v. 10, n. 3, p. 197-207.


I can send you a copy if you wish.

I hope this helps you.

Cheers.
<><><><><><><><><><><><><><><><><><><><><><><>
Gustavo G. Pilger, Mining Engineer, MSc, PhD
Senior Geostatistician
MBR - Brazil
<><><><><><><><><><><><><><><><><><><><><><><>




Hi.  I just read through Journel and Rossi's 1999 paper, "When do we
need a trend model in Kriging".  In the appendix they say "A kriging
variance is but a variogram-model dependent ranking of data
configurations; being data-value  independent, it is generally not a
measure of local accuracy...This fact is unfortunately not yet fully
appreciated by some practitioners".  Can someone explain the
implications of this in terms of determining cost-efficiency analysis
for sample designs?  Specifically, can we use kriging variance
estimates across potential sampling grids, (from modeled variograms
estimated from say a pilot study) to estimate the variability
associated with different sampling densities/configurations.  In
addition, can someone provide some references that address this topic.

Thanks,
Kerry
+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and
"unsubscribe ai-geostats" in the message body. DO NOT SEND
Subscribe/Unsubscribe requests to the list
+ As a general service to list users, please remember to post a summary
of any useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/



+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/

AI-GEOSTATS: kriging variance and accuracy

2006-09-28 Thread Kerry Ritter 
Hi.  I just read through Journel and Rossi's 1999 paper, "When do we 
need a trend model in Kriging".  In the appendix they say "A kriging 
variance is but a variogram-model dependent ranking of data 
configurations; being data-value  independent, it is generally not a 
measure of local accuracy...This fact is unfortunately not yet fully 
appreciated by some practitioners".  Can someone explain the 
implications of this in terms of determining cost-efficiency analysis 
for sample designs?  Specifically, can we use kriging variance estimates 
across potential sampling grids, (from modeled variograms estimated from 
say a pilot study) to estimate the variability associated with different 
sampling densities/configurations.  In addition, can someone provide 
some references that address this topic.


Thanks,
Kerry
+
+ To post a message to the list, send it to ai-geostats@jrc.it
+ To unsubscribe, send email to majordomo@ jrc.it with no subject and "unsubscribe 
ai-geostats" in the message body. DO NOT SEND Subscribe/Unsubscribe requests to the 
list
+ As a general service to list users, please remember to post a summary of any 
useful responses to your questions.
+ Support to the forum can be found at http://www.ai-geostats.org/