On Wednesday 01 July 2009, roger koenker wrote:
> It's not clear to me whether you are looking for an exploratory tool
> or something more like formal inference.  For the former, it seems
> that estimating a few weighted quantiles would be quite useful.  at
> least
> it is rather Tukeyesque.  While I'm appealing to authorities, I can't
> resist recalling that Galton's "invention of correlation" paper:  Co-
> relations
> and their measurement, Proceedings of the Royal Society, 1888-89,
> used medians and interquartile ranges.
>

Thanks Roger. 

We are interested in a formal inference type of comparison. I have found that 
weighted density plots to be a helpful exploratory graphic-- but ultimately 
we would like to (as objectively as possible) determine if distributions are 
different. 

Unfortunately the distributions are very poorly behaved. Once such example can 
be seen here:
http://169.237.35.250/~dylan/temp/ACTIVITY-method_comparision.pdf

I was imagine using something like the number of 'significant' differences in 
n-quantiles would be used to determine which treatment group was most 
different than the control group.

Cheers,
Dylan


> url:    www.econ.uiuc.edu/~roger            Roger Koenker
> email    rkoen...@uiuc.edu            Department of Economics
> vox:     217-333-4558                University of Illinois
> fax:       217-244-6678                Urbana, IL 61801
>
> On Jul 1, 2009, at 2:48 PM, Dylan Beaudette wrote:
> > Thanks Roger. Your comments were very helpful. Unfortunately, each of
> > the 'groups' in this example are derived from the same set of data,
> > two of
> > which were subsets-- so it is not that unlikely that the weighted
> > medians
> > were the same in some cases.
> >
> > This all leads back to an operation attempting to answer the question:
> >
> > Of the 2 subsetting methods, which one produces a distribution most
> > like the
> > complete data set? Since the distributions are not normal, and there
> > are
> > area-weights involved others on the list suggested quantile-
> > regression. For a
> > more complete picture of how 'different' the distributions are, I
> > have tried
> > looking at the differences between weighted quantiles: (0.1, 0.25,
> > 0.5, 0.75,
> > 0.9) as a more complete 'description' of each distribution.
> >
> > I imagine that there may be a better way to perform this comparison...
> >
> > Cheers,
> > Dylan
> >
> > On Tuesday 30 June 2009, roger koenker wrote:
> >> Admittedly this seemed quite peculiar....  but if you look at the
> >> entrails
> >> of the following code you will see that with the weights the first
> >> and
> >> second levels of your x$method variable have the same (weighted)
> >> median
> >> so the contrast that you are estimating SHOULD be zero.  Perhaps
> >> there is something fishy about the data construction that would have
> >> allowed us to anticipate this?  Regarding the "fn" option, and the
> >> non-uniqueness warning,  this is covered in the (admittedly obscure)
> >> faq on quantile regression available at:
> >>
> >>    http://www.econ.uiuc.edu/~roger/research/rq/FAQ
> >>
> >> # example:
> >> library(quantreg)
> >>
> >> # load data
> >> x <- read.csv(url('http://169.237.35.250/~dylan/temp/test.csv'))
> >>
> >> # with weights
> >> summary(rq(sand ~ method, data=x, weights=area_fraction, tau=0.5),
> >> se='ker')
> >>
> >> #Reproduction with more convenient notation:
> >>
> >> X0 <- model.matrix(~method, data = x)
> >> y <- x$sand
> >> w <- x$area_fraction
> >> f0 <- summary(rq(y ~ X0 - 1, weights = w),se = "ker")
> >>
> >> #Second reproduction with orthogonal design:
> >>
> >> X1 <- model.matrix(~method - 1, data = x)
> >> f1 <- summary(rq(y ~ X1 - 1, weights = w),se = "ker")
> >>
> >> #Comparing f0 and f1 we see that they are consistent!!  How can that
> >> be??
> >> #Since the columns of X1 are orthogonal estimation of the 3
> >> parameters
> >> are separable
> >> #so we can check to see whether the univariate weighted medians are
> >> reproducible.
> >>
> >> s1 <- X1[,1] == 1
> >> s2 <- X1[,2] == 1
> >> g1 <- rq(y[s1] ~ X1[s1,1] - 1, weights = w[s1])
> >> g2 <- rq(y[s2] ~ X1[s2,2] - 1, weights = w[s2])
> >>
> >> #Now looking at the g1 and g2 objects we see that they are equal and
> >> agree with f1.
> >>
> >>
> >> url:    www.econ.uiuc.edu/~roger            Roger Koenker
> >> email    rkoen...@uiuc.edu            Department of Economics
> >> vox:     217-333-4558                University of Illinois
> >> fax:       217-244-6678                Urbana, IL 61801
> >>
> >> On Jun 30, 2009, at 3:54 PM, Dylan Beaudette wrote:
> >>> Hi,
> >>>
> >>> I am trying to use quantile regression to perform weighted-
> >>> comparisons of the
> >>> median across groups. This works most of the time, however I am
> >>> seeing some
> >>> odd output in summary(rq()):
> >>>
> >>> Call: rq(formula = sand ~ method, tau = 0.5, data = x, weights =
> >>> area_fraction)
> >>> Coefficients:
> >>>                  Value    Std. Error t value  Pr(>|t|)
> >>> (Intercept)        45.44262  3.64706   12.46007  0.00000
> >>> methodmukey-HRU     0.00000  4.67115    0.00000  1.00000
> >>>                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >>>
> >>> When I do not include the weights, I get something a little closer
> >>> to a
> >>> weighted comparison of means, along with an error message:
> >>>
> >>> Call: rq(formula = sand ~ method, tau = 0.5, data = x)
> >>> Coefficients:
> >>>                  Value    Std. Error t value  Pr(>|t|)
> >>> (Intercept)        44.91579  2.46341   18.23318  0.00000
> >>> methodmukey-HRU     9.57601  9.29348    1.03040  0.30380
> >>> Warning message:
> >>> In rq.fit.br(x, y, tau = tau, ...) : Solution may be nonunique
> >>>
> >>>
> >>> I have noticed that the error message goes away when specifying
> >>> method='fn' to
> >>> rq(). An example is below. Could this have something to do with
> >>> replication
> >>> in the data?
> >>>
> >>>
> >>> # example:
> >>> library(quantreg)
> >>>
> >>> # load data
> >>> x <- read.csv(url('http://169.237.35.250/~dylan/temp/test.csv'))
> >>>
> >>> # with weights
> >>> summary(rq(sand ~ method, data=x, weights=area_fraction, tau=0.5),
> >>> se='ker')
> >>>
> >>> # without weights
> >>> # note error message
> >>> summary(rq(sand ~ method, data=x, tau=0.5), se='ker')
> >>>
> >>> # without weights, no error message
> >>> summary(rq(sand ~ method, data=x, tau=0.5, method='fn'), se='ker')



-- 
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Reply via email to