Re: [R] need help with smooth.spline

roger koenker Fri, 05 Mar 2004 00:56:55 -0800

Another option for fitting smoothing spline models to data with abrupt changes in derivative is the total variation penalty methods that are incorporated into the R package "nprq" . The models there are fitting piecewise linear models both univariate and bivariate components are allowed, but the roughness penalty is total variation of the derivative, or gradient in the bivariate case, so sharp kinks are permitted.


url:        www.econ.uiuc.edu/~roger                Roger Koenker
email   [EMAIL PROTECTED]                       Department of Economics
vox:    217-333-4558                            University of Illinois
fax:    217-244-6678                            Champaign, IL 61820

On Mar 4, 2004, at 8:30 AM, Liaw, Andy wrote:

Hi Carlisle,

If I understand you correctly, the problem is smooth.spline() not handling sharp jump(s), right? If so, it's probably easier to try something that can handle such features. Wavelet `denoising' (as opposed to `smoothing', and available in the wavethresh package) is well known for being able to handle abrupt changes (very `spatially adaptive'). Other things you might consider are mars() in the `mda' package (which fits splines in an adaptive fashion) and locfit() in the `locfit' package. For locfit, you will want to specify local smoothing parameter selection, via a call like

locfit(..., alpha=c(0, 0, 2), acri="cp")
You might need to play with the `2' a bit to get the right amount of
smoothing.  The details are in Loader's book `Local regression and
Likelihood'.
HTH,
Andy
From: W. C. Thacker

Andy,

As the data are often noisy, smoothing splines should be appropriate.
The first example profile shows an isothermal (constant temperature)
layer in the upper ocean followed by a sharp thermocline (large
temperature gradient), but there are relatively few observations
defining this sharp transition.  In this case simple linear
interpolation works fine, but smooth.spline() with all defaults gives
an absolutely absurd value in the isothermal layer.  With all.knots =
TRUE, the values in the isothermal layer are much better but still
peculiar.
Given the sampling and the data, is it possible to get smooth.spline()
do better?  If so, would that adversely impact its performance for
other cases?  (There are thousands of profiles.)  If not, is there a
simp[le way to select cases that smooth.spline() should not be
expected to handle, so they can be treated separately?
Thanks,

Carlisle

"Liaw, Andy" wrote:
If you really want interpolation, should you be using
spline() rather than
smooth.spline()? The later is for smoothing data observed
with noise, not
for interpolation.

Andy

From: W. C. Thacker

Dear R listers,

When using smooth.spline to interpolate data, results are
generally
good. However, some cases produce totally unreasonable results.
The data are values of pressure, temperature, and salinity from a
probe that is lowered into the ocean, and the objective is to
interpolate temperature and salinity to specified
pressures. While
smooth.spline provides excellent values at the observed pressures,
there are cases when the values at the desired pressures are
unusable.  A dataframe with four such profiles, indicated
by values of
id, is attached. My target values for pressure are
seq(25,1600,25),
but 1:500 is also interesting.

Setting all.knots = TRUE helps, but it would be nice to
be able to do
better.

Any suggestions?

Thanks,

Carlisle

version
         _
platform sparc-sun-solaris2.9
arch     sparc
os       solaris2.9
system   sparc, solaris2.9
status
major    1
minor    8.0
year     2003
month    10
day      08
language R
--

William Carlisle Thacker
Atlantic Oceanographic and Meteorological Laboratory
4301 Rickenbacker Causeway, Miami, Florida 33149 USA
Office: (305) 361-4323           Fax: (305) 361-4392
"Too many have dispensed with generosity
     in order to practice charity."     Albert Camus
______________________________________________
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
--------------------------------------------------------------
----------------
Notice: This e-mail message, together with any
attachments,...{{dropped}}
______________________________________________
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html

--
William
Carlisle Thacker
Atlantic Oceanographic and Meteorological Laboratory
4301 Rickenbacker Causeway, Miami, Florida 33149 USA
Office: (305) 361-4323           Fax: (305) 361-4392
"Too many have dispensed with generosity
     in order to practice charity."     Albert Camus
----------------------------------------------------------------------- ------- Notice: This e-mail message, together with any attachments,...{{dropped}}

______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html

______________________________________________
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html

Re: [R] need help with smooth.spline

Reply via email to