Hi Dan
Thank you for your stimulating reply.
I'm aware (and obsessed by it!), we talked about
this in some past emails, of the indetermination
principle behind the concept of trend/residuals
decomposition and this time my question is really basic (and maybe stupid!).
Well, I'm working on topographic data and I need to analyze
their roughness and in general their short range
spatial variability, so I need to remove
the effect of long range fluctuations. In this
case, as I said, I'm lucky because of
I know what I want to filter out. I tried
different methods like local polynomials and
averaging with moving windows. Giving a look to
the resulting trend surfaces I see, qualitatively, that
also with moving averages I obtain "good" results
and these will improve further if instead of doing simple
moving window means I use a simple kernel, like
the one cited in the paper (that practically means that, in my trend
estimation, for a certain distance around the
center of the window I give the same weight to
all samples and beyond this distance the weight
decay). Having seen this (that, as I said, it is
not a big new!) my question is: which kernel?
But your reply on this open an important point
when working with topographic data.
How to evaluate the quality of the derived trend surface?
And above all, how can be evaluated the expert qualitative
judgement coming from the observation of reality
or better of the true "realization" (that for
many other spatial parameters is not
available)?
Sorry for the too long mail
Sebastiano
At 10.49 02/02/2010, Cornford, Dan wrote:
Sebastiano,
I am struggling to understand why you are
interested in doing trend + residual
separation? There can be no unique
decomposition of a data set into trend and
residual, it is a judgement about what model
you feel is most appropriate given your prior
beliefs and observations (evidence). The only
thing you can say about the model is to
validate it on out of sample data (even as a
Bayesian I say this!). So in a sense there is
no correct decomposition, and any decomposition
is valid (so long as it is correctly
implemented maybe that is your question?).
Are some decompositions better than others?
Well yes they are likely to be, but this
largely depends on your data (and the completeness of the overall model).
In terms of your original question about the
shape of the kernel there is no overall theory
that I am aware of different kernels will have
different properties in terms of the function
classes that they represent (e.g.
differentiability, frequency response /
characteristic length scales). Kernel families
will have different null spaces which might or
might not be important for your specific
application and what you want to find out.
Im not sure if this is terribly helpful
but I
think it is the reality everything depends on
your data and your judgement (prior).
Conditional on those you get a model and you
need to validate this model carefully
then you are OK.
cheers
Dan
-------------------------------------------
Dr Dan Cornford
Senior Lecturer, Computer Science and NCRG
Aston University, Birmingham B4 7ET
www:
<http://wiki.aston.ac.uk/DanCornford/>http://wiki.aston.ac.uk/DanCornford/
tel: +44 (0)121 204 3451
mob: 07766344953
-------------------------------------------
----------
From: owner-ai-geost...@jrc.ec.europa.eu
[mailto:owner-ai-geost...@jrc.ec.europa.eu] On Behalf Of seba
Sent: 02 February 2010 08:39
To: Pierre Goovaerts
Cc: ai-geostats@jrc.it
Subject: Re: AI-GEOSTATS: moving averages and trend
Hi Pierre
I think that for my task factorial kriging is a little bit
too much sophisticated (nevertheless, is there any open source or
free implementation of it ??? I remember that it
is implemented in Isatis.....).
I have an exhaustive and regularly spaced data set (i.e. a grid) and I need
to calculate locally the spatial variability of the residual surface or better
I would like to calculate the spatial
variability of the high frequency component.
Here I'm lucky because I know exactly what I
want to see and what I need to filter out.
In theory, using (overlapping) moving window
averages (but here it seems better to use some more complex kernel)
one should be able to filter out the short range
variability (characterized by an
eventual variogram range within the window size???).
Seeing the problem from another perspective, in the case of a perfect
sine wave behavior, I should be able to filter out spatial
variability components with wave lengths up to the window size.
But maybe there is something flawed in my
reasoning....so feedback is appreciated!
Bye
Sebastiano
At 16.27 01/02/2010, you wrote:
well Factorial Kriging Analysis allows you to tailor the filtering weights
to the spatial patterns in your data. You can use the same filter size but
different kriging weights depending on whether you want to estimate
the local or regional scales of variability.
Pierre
2010/2/1 seba
<<mailto:sebastiano.trevis...@libero.it> sebastiano.trevis...@libero.it>
Hi José
Thank you for the interesting references. I'm going to give a look!
Bye
Sebastiano
At 15.46 01/02/2010, José M. Blanco Moreno wrote:
Hello again,
I am not a mathematician, so I never worried too
much on the theoretical reasons. You may be able
to find some discussion on this subject in
Eubank, R.L. 1999. Nonparametric Regression and
Spline Smoothing, 2a ed. M. Dekker, New York.
You may be also interested on searching
information in and related to (perhaps citing)
this work: Altman, N. 1990. Kernel smoothing of
data with correlated errors. Journal of the
American Statistical Association, 85: 749-759.
En/na seba ha escrit:
Hi José
Thank you for your reply.
Effectively I'm trying to figure out the theoretical reasons for their use.
Bye
Sebas
--
Pierre Goovaerts
Chief Scientist at BioMedware Inc.
3526 W Liberty, Suite 100
Ann Arbor, MI 48103
Voice: (734) 913-1098 (ext. 202)
Fax: (734) 913-2201
Courtesy Associate Professor, University of Florida
Associate Editor, Mathematical Geosciences
Geostatistician, Computer Sciences Corporation
President, PGeostat LLC
710 Ridgemont Lane
Ann Arbor, MI 48103
Voice: (734) 668-9900
Fax: (734) 668-7788
<http://goovaerts.pierre.googlepages.com/>http://goovaerts.pierre.googlepages.com/
