Dr. Harrell,
Thank you for your response. I had noted, and appreciate, your perspective on
ROC in past listserv entries and am glad to have an opportunity to delve a
little deeper.
I (and, I think, Jose Daniel Anadon, the original poster of this question) have
a predictive model for the presence of, say, animal_X. This is a spatial model
that can be represented on maps and is based on known locations where animal_X
is present and (usually) known locations where animal_X is absent. Output of
the analysis (using any number of analytic routines, including logit,
randomForest, maximum entropy, mahalanobis distance...) is a full map where
every spot on the map has a probability that that particular location has the
appropriate habitat for animal_x.
This output can be visualized by just using a color scale (perhaps blue for low
probability to red for high probability), BUT, there are times when we want to
apply a cutoff to this probability output and create a product where we can say
either "yes, animal_X habitat is predicted here" or "no, animal_X habitat is
not predicted here."
Note this is the final analytic step. There are no later anaylsis steps and so
(possibly) adjustments for multiple comparisons do not come into play.
Indeed, it seems that using a standard process to find a threshold reduces the
arbitrariness of the probabiliity color scale (at what probability do we set
'red'? at what probability do we set 'blue'?).
Are there alternative approaches that reduce the drawbacks you allude to?
How would you turn a surface of probabilities into a binary surface of yes-no?
Thank you for your time.
Sincerely,
Tim Howard
Ecologist
New York Natural Heritage Program
>>> Frank E Harrell Jr <[EMAIL PROTECTED]> 03/31/06 11:20 AM >>>
Choosing cutoffs is frought with difficulties, arbitrariness,
inefficiency, and the necessity to use a complex adjustment for multiple
comparisons in later analysis steps unless the dataset used to generate
the cutoff was so large as could be considered infinite.
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
